Regulatory Challenges of AI

Artificial Intelligence (AI), Machine Learning, ‘big data’ and ChatGPT have now reached the public consciousness in a big way, hence headlines such as ‘Central Bank of Ireland blocks ChatGPT for staff as financial services firms grapple with AI’ and ‘China’s Central Bank Warns Against ChatGPT ‘Data Leaks’.

Roland van der Vorst (Head of Innovation at Rabobank) has made the point that the way AI solves things is becoming less and less transparent. We don't know how it got its answers ¹.

He also makes that point that humans seem to be changing, increasingly focusing ‘our thinking on what is directly in front of us: the next suggestion or stimulus.’ ‘Instead of telling and coming up with big stories, we are trained to respond to the quick answer. Our main orientation increasingly seems to be the next step, rather than the broad vista.’ 

The Centre for Economic and Policy Research published a paper about AI and central banks in 2020 ².This paper largely considered AI in the context of regulation of financial markets. However, the challenges it identifies apply more widely.

If AI is used in a regulatory setting, confidence needs to be built around four areas when using AI:-

1. Procyclicality

If AI tools standardise recommendations around the least-worst option, then a homogeneous view emerges. If all AI tools, whoever is using them, does this, then organisations could end up with the same ‘answers’, which increases systemic risk.

2. Unknown-unknowns

The challenge is how to train AI to assess and respond to ‘unknown-unknowns’. It is straightforward to train them to consider events that have happened or even to train them to look at simulated scenarios. History shows, though, that crisis tend to come from the unexpected. The subprime mortgage was an example of a risk that was not foreseen.

Goodhart’s law (1975) says, ‘(a)ny observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.’ The complexity and ever changing nature of the financial systems, and the actions of the ever random human element, will limit the usefulness of AI in a crisis.

3. Trust

If AI is incorporated into decision-making it is likely that over time humans will learn to trust it. Based on achieving given objectives, AI is likely to keep financial systems safe effectively in normal circumstances building trust. In a situation where there are fixed objectives but unlimited complexity, AI may take unexpected decisions that lack humanity. Even in the most extreme and unexpected turn of events, a human can adjust their behaviour based on a shared understanding of values and the environment.

AI won’t be able to because it has no values, only objectives. And its understanding of the environment will not necessarily be intelligible to humans. To compound the problem, it is possible we won’t be able to understand why it has made the decision it has made.

By the time AI has reached the point where it is trusted to make key decisions, it is probable that ‘turning it off’ won’t be possible. It will be embedded across too many critical systems.

4. Optimise against the system

In any system, particularly the financial system, there will be people who are solely focused on short term immediate gain. They may well be armed with their own AI systems. How can a central bank AI system react to a malicious actor? The malicious actor is probably just manipulating one element while the central bank AI has to monitor and manage the whole system. In addition, the central bank AI is likely to be predictable along the lines of its given objectives.

It is already normal to train AI to react randomly in interactions to limit the ability to game it. This may not work for a central bank AI given that it needs to be innately rational and also transparent and fair, which puts it at a disadvantage when it is used for supervision.

This may be less of an issue for monetary policy where constructive ambiguity is accepted. AI is likely to be beneficial to monetary policy because it will manage data collection, policy forecasting and information flows at a much lower cost than with current infrastructure.

Final word

AI offers the opportunity to do some key tasks more efficiently and inexpensively for the banking system - fraud detection, compliance, Know Your Customer (KYC) and Anti-Money Laundering (AML), customer onboarding and risk management. It brings with it a new set of risks and uncertainties that we don’t understand at present. Regulation is one thing, creating a resilient, effective solution over which humans have control is another.

1 - https://fd.nl/tech-en-innovatie/1474767/de-machine-als-mens-de-mens-als-machine

2 - Artificial intelligence as a central banker / CEPR