ACI WorldWide is a leading global payments provider. We sat down to discuss their new, patented Incremental Learning approach to building machine learning models to fight eCommerce fraud.
Incremental learning innovates on traditional fraud models by providing the algorithm with new data every 24 hours–which allows it to spot new fraud patterns as the occur.
Our discussion includes evaluating fraud models for degradation, the flaws in the traditional machine learning model, and more.
1. ACI’s incremental learning is centered on the idea of fraud models degrading over time. How long do merchants’ fraud models last on average? How can you evaluate for degradation, before it becomes apparent via chargebacks?
All models degrade over time. How long the models last depends on a number of variables including the merchant’s industry, purchasing channel, season, etc… We’re seeing frequent changes in fraudulent behavior, more so than before; So it is important to be able to take an approach to ensure that these changes do not lead to model degradation. ACI’s incremental learning approach is able to recognize these small but frequent changes and, if required, make adjustments to the production model without requiring retraining or redeploying the production model.
2. What do you estimate the practical effect of incremental learning to be in terms of chargeback rates and false positive declines versus traditional machine learning models?
Any degradation of model performance will increase the amount of chargebacks incurred. Incremental learning is about maintaining performance at a high quality for longer periods of time without negatively affecting false positive rates. In testing it was shown that within three months our models saw the benefits of incremental learning. This translates into a lot of savings in time and effort for the merchant. Most importantly, it performs better than traditional machine learning methods.
3. You filed for a patent for incremental learning in January. Can you share what makes the technology unique enough that it required one?
The ability to take an existing model and maintain performance for longer periods of time, by actively looking for anomalous behavior incrementally over shorter periods of time is a new approach to tackling the ever-changing fraudulent behaviors that are seen. Incremental learning is autonomous, which moves fraud prevention from predictive to being prescriptive. Having a seamless approach that identifies and adapts to new behaviors without intervention is a step forward in fraud prevention.
4. The traditional machine learning model vaunts itself as benefiting from a long and coordinated order review history across the merchants in the network. Is incremental learning saying that model is fundamentally flawed?
Traditional machine learning approaches are here for the foreseeable future. The power of historical consortium data is fundamental to machine learning, ours as well as every other solution. Where incremental learning fits is to take a traditionally trained model and maintain its performance for longer periods of time. This is done by looking at data over shorter intervals and, if required, make changes to the production model without having to retrain a new model.
5. Incremental learning ‘adjusts to new behaviors’ within every 24-hour time frame. Can you give a hypothetical example of how the model might adjust based on new inputs, and how the adaptation would produce better results than a traditional machine learning model?
Traditionally, machine learning models are trained on large amounts of historical data that is labeled. Each 24-hour period adds new information that is valuable and sometimes critical to the performance a fraud prevention model. The benefit of incremental learning is that you avoid retraining the model on the expanded data set and only require to learn from the last 24 hrs of data. The model in production can then be updated with the new learnings without a requirement to redeploy.
6. Is there a risk that changes seen over just a 24-hour period might cause models to readjust too quickly to an anomaly (say, a single, massive attack by a fraud ring) and risk sub-optimal results in the future? What if the data from two separate 24-hour periods diverges significantly?
Incremental learning does not hastily make decisions and instead gradually morphs to the changes it sees in each 24-hr period. We at ACI also use a multi-layered approach to fraud prevention, where incremental learning is a part of the machine learning layer. These layers, along with our highly skilled and experienced data scientists and risk analysts work together to create strategic rules that protect our customers from singular fraud ring-related events.
7. You write that incremental learning models don’t need to “re-learn everything they already know”. Can you give a practical example of how this is an improvement on standard machine learning models?
The improvement is similar to how we use information ourselves. When we learn something new, we do not have to re-learn everything we knew. We take the additional learnings and use them to make informed decisions. This is what incremental learning brings to machine learning. The new behaviors observed are used in conjunction with what the model had been trained on previously to make better predictions against fraudulent transactions.
8. Is there any performance or storage cost benefit to reducing reliance on historical data?
Storage can be quite expensive and storing large amounts of data for the purposes of retraining models can lead to additional costs that build up over time. Incremental learning does not require the initial dataset once trained. The main concept of online ML algorithms is that each transaction is only used once and then discarded, thus we can achieve both performance and storage cost benefits simultaneously.
9. You cite “[reducing] the number of model deployments required for our customers” as one of incremental learning’s advantages. Can you give an example of when/how a merchant would benefit from this?
Model performance can degrade over time. This degradation can be due to several factors, such as change in fraud behavior observed. Incremental learning keeps model parameters up to date, allowing a reduction in the number of deployments over longer periods. The ability to morph the model gradually based on new learnings from the last 24 hrs is a key component of Incremental Learning. This keeps you from having to retrain and deploy a new model 2-3 additional times each year, saving time and effort of skilled personnel at the merchant.
Jimmy Hennessy
Director of Data Science
ACI Worldwide
https://www.linkedin.com/in/jimmyhennessy/
Mr. Hennessy leads a team of data scientists at ACI who are dedicated to safe guarding merchants against fraud. He is a seasoned veteran with 18 years’ experience within computer and data science with a proven track record of successfully helping clients.