Metrics for Assessing AML Compliance Program

Financial crimes compliance is not easy, and that’s especially true for fintech firms — young, fast growing, and subject to a complicated thicket of anti-money laundering rules. Building an effective compliance program in that environment is not easy, and compliance officers need to tread carefully to get it right.

To that end, in this post we offer five metrics that can help you assess the effectiveness of your compliance program. First, however, let’s unpack why AML performance metrics are so important for your compliance program’s success. Those metrics help a compliance officer do three things:

• Identify where your program needs improvement; 
• Direct your resources to where they’ll have the greatest impact; and
• Demonstrate your program’s effectiveness to senior management, investors, and regulators. 

All three are crucial to your success as a compliance officer. After all, most fintech firms are still trying to establish themselves in the greater financial services world. This means that the firm is still trying to define what its AML compliance program should do, and to provide assurance (to the board, to regulators, to banking partners, to customers) that you’re doing it well.

At the same time, regulators are paying more attention to AML compliance programs. They flag failures in suspicious activity reporting, transaction monitoring, policies and procedures, or training on a regular basis. They want to see clear, compelling evidence that your AML program has been designed to address your firm’s specific risks, and that the program works as intended. 

So the more accurate and penetrating your AML performance metrics are, the better. Now let’s consider five metrics that will give you that sense of program effectiveness you need. 

False Positive Rate in high-volume, low-value transactions.

This is the false positive rate (a transaction flagged as suspicious but later determined to be legitimate) specifically for transactions that are great in absolute number but small in value — say, monthly remittances that U.S. residents might send to family in another country. These transactions tend to follow similar behavior patterns, so it’s harder to distinguish between suspicious and legitimate transactions. 

The false positive rate for this class of transactions can be painfully high (well over 50 percent) because automated transaction monitoring systems often run on generic rules. Those systems end up flagging a large number of these transactions just because they’re so numerous.

To get an accurate reading of your false positive rate, you’ll need to be sure your transaction monitoring system (TMS) collects as much data as possible about each transaction; and then use AI-based analytics tools that are properly calibrated to study that data at scale and then reach the right decisions. 

Alert-to-SAR ratio in a digital environment.

This metric compares how many alerts your transaction monitoring system generates to the number of suspicious activity reports (SARs) you actually file. It’s expressed as a percentage, and the lower that percentage is, the worse your AML program is. For example, if your ratio is 1 percent, that means 99 percent of all suspicious activity alerts your system generates do not need a SAR filing; you’re wasting your team’s time.

Your Alert-to-SAR ratio can be poor for numerous reasons. You might have a calibrated transaction monitoring systems that flag too many transactions; poorly trained staff, who focus on the wrong issues while missing the right ones; or incomplete or poorly integrated data, which leads to erroneous judgments about a transaction.

Compliance officers need to cast a critical eye at this metric, to be sure you accurately understand any problems you might have. The more robust your TMS is, and the more data you can feed into it, the better.

Transaction monitoring effectiveness for real-time payments.

This measures how well your transaction monitoring system can detect and intercept suspicious payments as those transactions happen, since they’re happening in real time. Another objective here is not to disrupt legitimate payments, so your TMS must strike the right balance between speed (to let legitimate payments proceed immediately) and accuracy (to prevent suspicious transactions from going forward).

The best way to understand the effectiveness of your TMS is to look at the rate of false positives — that is, how often your TMS flags a transaction as suspicious, when in fact the payment is legitimate. The higher your rate of false positives, the more disruptive to real-time payments your TMS is.

If your TMS is returning too many false positives, that could be because it isn’t using the right algorithms to evaluate a transaction’s risk, or your AI-driven tools weren’t calibrated properly and aren’t “learning” fast enough to keep pace with transaction volume.

Suspicious Activity Report filing rates for digital transactions.

This refers to how often fintech firms submit SARs to regulatory authorities when they detect potentially suspicious or fraudulent digital transactions. If you filed a SAR for every potentially suspicious transaction you encountered, your filing rate would be 100 percent; if you never filed any, it would be zero. 

You can get a sense of a “normal” SAR filing rate by looking at published industry benchmarks, or at historical data of your own SAR filing rates. Also consider the risk profile of your business and customer base; a higher risk profile (say, more cross-border payments or high-dollar value transactions) should lead to higher SAR filing rates. 

Customer due diligence completion rates.

This metric tracks how many customers who open an account online go through complete due diligence; that is, how many provide all the biographical information you request. The higher that number, the more thorough your customer due diligence (CDD) program is.

Completion rates will differ among various groups. For example,  customers in low-risk categories are likely to complete CDD more often, but customers in high-risk categories will complete it less often, because the higher risk typically means you require more information from them. 

Would-be customers abandon due diligence processes (and therefore cease being sales prospects for your firm) when your CDD processes are complicated or not user-friendly. Aim for streamlined, automated processes whenever possible, and for a completion rate as close to 100 percent as you can get.  

Conclusion

Compliance officers need to know that their AML programs work. That means selecting the right performance metrics to understand how well your program works — and where it might need improvement. The five examples above will provide valuable insights, and then you can get on with the job of making your AML compliance program better.