Oplossingen voor AML-backtesting
Anti-Money Laundering (hereinafter: ‘AML’) is a required component of the financial industry’s ongoing efforts to detect and prevent financial crimes. Most financial institutions rely on rule-based and/or machine learning models to identify suspicious activity. However, the effectiveness of these models is directly tied to the quality of feedback they receive. One key method to refine models, in general, is through backtesting, which evaluates model performance by comparing historical data with model outcomes. This process not only aids in the identification of potential weaknesses, but also allows for adjustments to improve overall performance. For instance, in credit risk models, financial institutions analyze past loan data, including borrower characteristics, loan terms, and repayment histories to validate the accuracy and reliability of models predicting loan default probability.
In theory, backtesting in AML models should allow financial institutions to fine-tune their models, increasing true positives and reducing false positives. However, in practice, they often face significant challenges when it comes to conducting proper backtesting. Why can AML models not benefit from backtesting the same way that, for example, credit models can?
The challenge – lack of feedback on alerts
A significant challenge in the backtesting process is the lack of ground truth for model outcomes. When a transaction monitoring (hereinafter: ‘TM’) model identifies unusual transactions, they are initially reviewed by an internal alert handling team and a compliance department. If deemed ‘unusual’, these alerts are escalated to relevant authorities, such as the Financial Intelligence Unit (hereinafter: ‘FIU’). However, financial institutions rarely receive follow-up information on whether these alerts resulted in investigations, prosecutions, or whether they were ultimately classified as false positives. Additionally, when no feedback is received, this does not automatically justify the position that the alert should ultimately be considered a false positive. Consequently, financial institutions are unable to utilize this crucial feedback as ground truth for the refinement of their models. This limitation results in challenges such as the inability to enhance detection model accuracy, and higher rates of false positives. Regulators like the Dutch central bank (DNB) expect institutions to demonstrate compliance with strict laws, such as the Anti-Money Laundering and Anti-Terrorist Financing Act (Wwft) and the Sanctions Act (SW), but without reliable backtesting, this is certainly challenging.
[....]
