The Financial Markets Standards Board (FMSB) has published a transparency draft Statement of Good Practice (SoGP) looking at the application of a model risk management framework to algorithms. The FMSB has built up a body of Standards and Statements of Good Practice over time, prioritising those areas where FMSB member firms consider there is a lack of clarity in the standards of behaviour expected of market participants, or a lack of understanding of the issues relevant to a product or transaction type, or evidence of poor conduct. SoGPs do not form part of FMSB Standards, and they are not subject to FMSB’s adherence framework. Rather, they reflect FMSB’s view of what constitutes good or best practice in the areas covered by the SoGPs in question. Member Firms are expected, and other firms are invited, to consider their own practices in light of the relevant SoGP and make any changes to such practices that they deem to be appropriate.
Good Practice Statements
This SoGP sets out nine GPSs relevant to the application of a model risk management framework to Algos:
1. Model indentification
Good Practice Statement 1:
Firms should examine their Algos to identify any methods that constitute a Model.
2. Model Risk Tiers
Good Practice Statement 2:
Firms should categorise each of their Models associated with Algos into risk-based tiers to help identify and manage Model Risk.
3. Model testing
Good Practice Statement 3:
Model testing in Algos should, in particular:
i. Assess Model performance under a variety of market conditions, including volatile conditions and scenarios where there is limited and/or poor quality market data; and
ii. Emphasise testing of both the Embedded Controls and the Mitigating Controls as opposed to testing the accuracy of a Model, given that the adverse consequences of Model inaccuracy can be addressed through an effective control framework.
4. Model validation
Good Practice Statement 4:
When considering the Residual Risk and the depth and frequency of the validation of the methodology of a Model used in Algos, firms should take into account all Mitigating Controls.
Good Practice Statement 5:
Model validation activities should be tailored to the context in which such Models are deployed and proportionate to the risks they present. For Algos, Model validation may prioritise reliance upon the effectiveness of Mitigating Controls over Model accuracy.
Good Practice Statement 6:
Independent staff conducting Model validation for Models associated with Algos should be sufficiently knowledgeable of the use of such Models in financial markets.
Good Practice Statement 7:
The nature and frequency of ongoing performance monitoring for Models associated with Algos should (i) be appropriate to the risk-based tier of the Model; and (ii) complement any manual trading supervision of the Algos and the associated continuous objective feedback of Algo or Model performance required in a wholesale markets context. When considering how to respond to any Model issues or errors identified during ongoing performance monitoring, firms should consider, using observed data, if such issue or error is likely to lead to materially adverse outcomes.
5. Model changes
Good Practice Statement 8:
When determining if a change to a Model associated with Algos requires validation, and, if so, the extent of such validation, a firm should consider (i) the materiality of the change in methodology; (ii) the risk-based tier of the Model; (iii) the extent to which the change impacts the Inherent Risk of the Model.
Good Practice Statement 9:
Firms should consider whether Model, or Model change, documentation can be supported with Model source code access.
The FMSB says firms should continue to apply any supervisory expectations and relevant guidance applicable to them and their Models.
The Principles are intended to supplement and assist firms with the practical application of these expectations and guidance for Models used in Algos in a manner that is commensurate with the risks posed by such Models.
Model risk management frameworks
The purpose of this particular SoGP is to support firms in applying model risk management frameworks in a proportionate manner to models deployed in their electronic trading algorithms taking into account the nature, scale and complexity of such models as well as existing systems and risk controls intended to mitigate associated market, conduct, credit and operational risks.
This SoGP addresses a sub-set of issues associated with model risk management and is not intended to detail a comprehensive model risk management framework or to address all risk types. The areas focused on are where market practitioners, including “first line” risk owners and “second line” risk managers, have identified that the nature of model use in electronic trading algorithms merits a differentiated approach compared with other model types. In particular, (see the box for more details) the SoGP considers:
i. Key factors in determining if a method used in an Algo constitutes a model (GPS 1);
ii. Factors influencing the risk-tiering assigned to a model used in Algos, and the impact of mitigating controls in reducing the residual risk of a model (GPS 2);
iii. Key features of model testing for Algos (GPS 3);
iv. Tailoring model risk management activities for models deployed in Algos to the context and purpose for which models are deployed, focusing on model methodology and input accuracy, staffing and ongoing performance monitoring of outputs (GPSs 4-7); and
v. The treatment of material changes to models deployed in Algos from a validation and documentation perspective (GPSs 8-9).
Algorithmic trading risk summary
The table below summarises material risks associated with the deployment of Algos either to fair and effective markets or to Algo operators. The table encapsulates conduct, market, credit, operational and Model Risks. A firm’s risk management frameworks, of which a model risk management framework is one component, will typically be designed to reduce or mitigate such risks, though appropriate calibration and effective application of such frameworks will be key to their effectiveness.
The FMSB is inviting comments on the draft by 22 September 2023. For more information please visit: https://fmsb.com/other-relevant-information/