If it’s a passive execution then we will be looking at how much executed passively, what the spread capture is, if there was any market impact from the placements, time to fill etc. If it is an aggressive algo we will be looking at fill ratio, market impact curves after trades, slippage, cost of retry. If it is an illiquid pair we might also be looking at bid ask compression and signalling risk.
Sometimes, there is a smoking gun, and its easy to identify. For example, if selling in a downward trending market, it is easy to demonstrate the price before, during and after the trade and show that the outcome was relative to market performance overall. The output of the analysis dictates what we do next to improve, what parameters to adjust. When its not so obvious, you need to delve more deeply into what's happening in the ECNs, and policing the liquidity pools.
Answering the question “how did it perform” and even more subjectively “did it meet client expectations” means we need to understand what our client was trying to achieve with the order.
For example, if a client says I want the algo to be passive then an indication of the type of duration expected helps when checking performance. If opening a position and the price is going in the clients favour there may be a desire to leave more risk on the table – ie trade more slowly and allow the algo to benefit from price improvement. In this scenario, if an algo completes in 10 minutes, but the expected duration was an hour, it doesn’t matter that the algo looks good on all metrics – it failed the fundamental objective and we have think carefully about why that happened.
However obviously this has to be analysed correctly for each client – for most clients, if an algo reacts opportunistically to favourable conditions, this is exactly what they want. Choosing the right algo to match execution objectives is key here, as is the ability to step back and adjust the algo if necessary if the market changes.
More experienced algo users will generally choose their algo and provider with this in mind and will have a specific set of criteria to determine whether the execution was good or not, either done in house or by submitting orders to independent Best Execution services. Less experienced algo users may need more help in understanding the different characteristics of algos so they can better meet their execution objectives. At this point we provide our own objective analysis to the clients.
Ultimately, even with a vast array of execution analytics that help improve performance often the best feedback is from the client and whether or not they got what they feel is a good result at that time.