NVIDIA: The Hurdles FIs Must Overcome for AI Implementation

AI and in particular Gen AI, are the buzzwords of today when it comes to digital transformation at banks and other large financial institutions. 

Of course, there’s no harm in displaying excitement at the possibilities Gen AI can unlock for the financial services industry, but the academic discourse is easier to discuss than the technology is to implement. 

We speak to Malcolm deMayo, VP of Global Financial Services at NVIDIA, about the significant hurdles banks must overcome to successfully, effectively and safely implement AI. 

The key challenges to AI adoption for banks 

The challenges to AI adoption at financial institutions are wide-ranging, they cause friction and slow financial firms down. 

For Malcolm, it’s important to first take a step back. After all, AI has been baked into financial services operations for some time – predictive AI and machine learning in the form of early neural nets, like recurrent neural nets (RNNs), have been embedded by early AI adopters. 

Unified data sets

When it comes to the latest AI innovations, the first question banks should ask themselves is how good their data is. 

“Financial institutions are organised around products, which has led to the creation of data silos,” Malcolm says. 

Now, it’s about getting that data into a unified set – a curated dataset that can be accessed and analysed to drive actionable insights. 

“Getting data into a place where you can make a high-quality data set is important for model accuracy, and it is an effort that financial firms are very focused on. They know this challenge and have been working on it for years, but it's one of the challenges slowing them down,” Malcolm adds. 

Is your dataset AI-ready?

It’s all well and good working toward creating a unified data source, but is it AI-ready? For Malcolm, creating a data pipeline consisting of tabular and unstructured data from internal and external sources and then curating the data—deduping, compressing white space, extracting noise, scanning for toxicity, and handling data gaps—are key steps in creating an AI-ready dataset.

“There are a few fundamental ways to improve model accuracy, and the first is to have a high-quality data set,” says Malcolm. It is in this way a unified data set feeds into the delivery of curated data. 

“Model accuracy is influenced by the size of the model,” he continues. “The larger the model in parameters, the better its reasoning skills, and the better its thought, the more accurate it is. The more data it's trained on, the better it is at understanding the world which is described in words. 

“When you have this, you have a large language model, ie, transformers or generative AI.” 

Leveraging open-source, improving efficiency

The advent of open-source models has significantly helped financial institutions in this regard. “These open source models allow firms to not have to go out and build their own foundational model,” adds Malcolm, cutting down on technology investment, time and manpower costs. 

“Jensen Huang describes an open-source model as a University graduate who knows a lot about the world but nothing about your organisation. So banks must train that model on their business data,” Malcolm explains. 

“Customising open-source models to your organisation’s needs is achieved using various techniques, such as supervised fine-tuning and Retrieval Augmented Generation (RAG). This really helps improve model accuracy. 

“The leading firms are working hard at data quality and experimenting with fine-tuning techniques, or partial fine-tuning techniques in combination with RAG to improve model accuracy.

“Once you’ve fine-tuned and validated your model, you can deploy it using RAG. Combining high-quality data, supervised fine-tuning, and RAG enables your model to perform better. This means the model will provide the sought-after answer to prompts (a question by another name)  on the first go more often instead of having to iterate your prompt several times before the model figures out what you are after. It starts to become expensive.

“And so for those firms, as they start experimenting, they realise that model accuracy leads to more efficiency and a better experience.”

This is the next consideration for banks – efficiency, both in terms of time and costs. 

It’s clear that data quality, model accuracy, and model size are all interrelated when it comes to efficiency and cost savings.

Building an AI Factory that supports multi and hybrid cloud

The next challenge for banks is to leverage an AI Factory that enables them to develop and deploy models anywhere. If they have sensitive data, need to prove their workload is resilient, would like to meet customers across clouds, or are concerned about accuracy, cost, or latency – if any of these are true, banks will want to leverage an AI Factory that can live anywhere. “It really requires a firm to have a platform to build models and inference models that's available anywhere in any cloud, on-premise or in colocation (colo),” says Malcolm. 

The regulatory angle

The final challenge for banks is arguably the most challenging – it’s the one area they have the least control over – regulations. 

Of course, financial firms have been integrating new technologies for years and have developed robust lines of defence to ensure any new implementation is fully compliant. 

They have four lines of defence already in place for new technology, especially for predictive AI or machine learning.

“The first line of defence is to ask: Just because we can do it, should we? Every employee in every financial firm is tasked with answering that question and understanding that they must be responsible if they decide to implement any new tech,” continues Malcolm.

With the fiduciary responsibility put on financial firms, this has, in essence, created a ‘table stacks’ process they must all go through.

The next layer of defence is model validation. Proving that the model will do no harm and will generate accurate, fair, and responsible insights and predictions. 

Malcolm says: “Last year was the year of experimentation from leading firms, and they looked at recalibrating their control framework for generative AI, having conversations with regulators around how it works, how varying inputs impact the outputs, and how to explain how a model operates responsibly.”

The third layer of defence is governance, and the fourth is risk management. Banks must ensure they focus on their exposures based on everything happening in the world. 

“That requires understanding beyond just tabular data,” Malcolm notes. “It requires looking at unstructured data, looking at breaking news, looking at fundamental data, looking at social sub-forms and paying attention to satellite images, which is quite a bit of work and requires a lot of experimentation.”

AI is here to stay

So, while there has been a lot of hype around the adoption of Gen AI at banks, Malcolm is definitive: “It’s here and it’s here to stay.”

He concludes: “This is for real. In 2024, we're starting to see production-based large language models pop up, and you're probably going to start seeing more as the year progresses. This will be the year when we start to see models get into production, as AI assistance, computer code generators, process automators, and co-pilots, all requiring a human in the loop to make final decisions, but it will keep growing as we head into 2025.”

**************

Make sure you check out the latest edition of FinTech Magazine and also sign up to our global conference series – FinTech LIVE 2024. 

**************

FinTech Magazine is a BizClik brand.