Analysing an algo’s performance

After a client trades an algo, we always ask ourselves “how did it perform” and “did it meet client expectations”. Even with the advent of independent best execution services, algo providers need to analyse their trades to make sure they are meeting the design objectives. Objectively “How did it perform” can be to some extent answered by standard metrics. Slippage vs Risk Transfer Price is often the first point of call. After this it will depend on the type of algo, market conditions and what we think the client was trying to achieve.

Analysing an  algo’s performance
By Tom Appleton, Executive Director, e-FX Algo Execution, Crédit Agricole CIB

If it’s a passive execution then we will be looking at how much executed passively, what the spread capture is, if there was any market impact from the placements, time to fill etc. If it is an aggressive algo we will be looking at fill ratio, market impact curves after trades, slippage, cost of retry. If it is an illiquid pair we might also be looking at bid ask compression and signalling risk.

Smoking gun

Sometimes, there is a smoking gun, and its easy to identify. For example, if selling in a downward trending market, it is easy to demonstrate the price before, during and after the trade and show that the outcome was relative to market performance overall. The output of the analysis dictates what we do next to improve, what parameters to adjust. When its not so obvious, you need to delve more deeply into what's happening in the ECNs, and policing the liquidity pools.

Answering the question “how did it perform” and even more subjectively “did it meet client expectations” means we need to understand what our client was trying to achieve with the order. 

For example, if a client says I want the algo to be passive then an indication of the type of duration expected helps when checking performance. If opening a position and the price is going in the clients favour there may be a desire to leave more risk on the table – ie trade more slowly and allow the algo to benefit from price improvement. In this scenario, if an algo completes in 10 minutes, but the expected duration was an hour, it doesn’t matter that the algo looks good on all metrics – it failed the fundamental objective and we have think carefully about why that happened.

However obviously this has to be analysed correctly for each client – for most clients, if an algo reacts opportunistically to favourable conditions, this is exactly what they want. Choosing the right algo to match execution objectives is key here, as is the ability to step back and adjust the algo if necessary if the market changes.

If a client says I want the algo to be passive then an indication of the type of duration expected helps when checking performance
If a client says I want the algo to be passive then an indication of the type of duration expected helps when checking performance

More experienced algo users will generally choose their algo and provider with this in mind and will have a specific set of criteria to determine whether the execution was good or not, either done in house or by submitting orders to independent Best Execution services. Less experienced algo users may need more help in understanding the different characteristics of algos so they can better meet their execution objectives. At this point we provide our own objective analysis to the clients.

Ultimately, even with a vast array of execution analytics that help improve performance often the best feedback is from the client and whether or not they got what they feel is a good result at that time.