Notes on Future of SOX Compliance

This week I’m attending the annual user conference for Workiva, maker of software for internal audit and GRC teams. This means, of course, that artificial intelligence is all over the agenda and everyone is talking about how AI will transform internal control and SOX compliance. Let’s review the notes I took.

One point that came up several times was that SOX compliance teams must embrace artificial intelligence at least to some extent, because your external audit firm already is. If you can’t keep up with their AI-enhanced capabilities, you’ll always be on the back foot in conversions about the effectiveness of your internal control.

For example, say you test your internal controls through the time honored approach of taking a sample of transactions and studying whether your internal controls worked as expected. You look at 1,000 transactions and find seven that fell outside normal control parameters, for a failure rate of 0.7 percent. Then along comes your external audit firm with its AI and advanced data analytics, and it sucks up all your transactions and finds a failure rate of 1.3 percent. 

How are you, the SOX compliance leader mired in the sample-and-test approach, supposed to respond to that? You can’t say the audit firm is wrong, because it examined all 1,000 transactions you tested and all the other transactions, too. The auditor’s data and analysis is more thorough than yours — so you’ll be in a far weaker position as you and the auditor spar over control design issues, significant deficiencies, or (eek!) a material weakness.

SOX compliance leaders know all this, of course. In fact, I don’t know any compliance or internal audit leader who still wants to use the traditional sample-and-test approach; you’d all love to adopt data analytics and AI for the comprehensive analysis it can bring. The obstacles are either that you don’t have the budget, or you haven’t quite figured out how to employ these new technologies to maximum effect.

But that day is coming. Traditional approaches to SOX compliance won’t be feasible for much longer. 

Implications for Internal Control

So let’s say you do transform your SOX compliance technology along these lines. Now you can analyze all transactions and identify all exceptions to the control clearly and easily. Can we pause here for a moment to consider what you’ve actually done? 

I would argue that you’re no longer “auditing” internal controls in the traditional sense of the word. That is, you’re not sifting through a sample of transactions to find a failure rate, like the 0.7 percent example we mentioned earlier. You’re identifying all exceptions to the control. That’s good and useful, but it’s not the same as “testing” a control. Instead, you’re identifying all instances where the control did not work.

Several implications flow from that point.

First, you have to figure out what to do with those exceptions. Perhaps the transactions themselves are so small in value that they’re not material and you can ignore them; but that means you need to know what your materiality threshold is. On the other hand, perhaps the exceptions are happening so frequently that even though they are quantitatively immaterial, in total they add up to a qualitatively material control failure. 

My point is that by embracing AI and data analytics, SOX compliance teams will get two tons more visibility into what’s really happening with all your transactions. You need to anticipate what that greater visibility will allow you to see, and what you’ll want to do with all those things you won’t be able to un-see once better technology drags them into the light.

Second, the importance of strong IT general controls will rise, so that nobody can tamper with the IT systems generating your transaction records. 

We’ve talked about the importance of “ITGCs” many times before on this blog. They are the controls that govern the technology you use, and address issues such as who can have privileged access to the system, who can create new users, who can alter existing records, and so forth. 

If we move to an assurance world where all transactions are reviewed automatically by some AI-enhanced system, then that AI system is the control. So the thing that SOX compliance teams need to review is the AI system’s performance and security — which are governed by IT general controls — rather than the actual transactions your company makes. 

Again, most SOX compliance leaders already perceive this assurance future off in the distance, even while we’re still slogging through the morass of traditional controls testing and documentation. Conceptually, none of this is news.

The question is how we get there, to a world where SOX compliance systems can review all transactions automatically and surface all exceptions immediately. It will require budget and planning; but it will also require SOX compliance leaders, external auditors, and even regulators such as the Public Company Accounting Oversight Board to anticipate the new and different questions of assurance that will arise.