More Reality Checks on AI Adoption

Today we have more food for thought about artificial intelligence systems and how they work in corporate organizations: three reports all suggesting that so far, workers and corporations alike aren’t reaping the benefits that AI supposedly delivers. 

This is important because remember, we’re all supposed to love AI. The chief executives all say AI is going to transform every job, and might well eliminate a whole bunch of jobs along the way, and everyone needs to accept those facts of life and get on board the AI train. Your jobs, as compliance officers and internal auditors and risk managers, will be to help usher in that AI transformation, both within your own teams and processes and for the rest of your organization too.

Anyway, that’s what the CEOs say. Now let’s get back to those reports — starting with PwC’s annual CEO survey, where 56 percent of CEOs say that so far their AI investments have had no effect whatsoever on either costs or revenue.

That’s right. According to 4,454 chief executives around the world who participated in PwC’s survey, 56 percent said their AI investments haven’t translated into any appreciable change in revenue or cost at all. See Figure 1, below.

PwC, of course, interpreted this to mean that companies aren’t embracing AI enough. Or, as the firm said in an admirable bit of advisory-speak: “Isolated, tactical AI projects often don’t deliver measurable value. Tangible returns come from enterprise-scale deployment consistent with company business strategy.” 

So after years of AI thinkers advising companies to experiment with small, point-specific use cases, we’re now supposed to boil the ocean? That reminds me of the ERP implementation boondoggles of the 1990s. Sure, companies reaped productivity and growth gains from those efforts eventually (I can already hear plenty of you muttering, “Dude, no we haven’t,” under your breath), but it took years. 

Two other thoughts about the PwC survey. 

First, notice that it only talks about AI investments in terms of revenue and cost. It does not talk about AI risks, which are a huge part of the equation — especially for compliance, audit, security, and legal teams; who will be stuck figuring out how to boil that AI ocean wisely, should your CEO decide to sail into it.

Second, if you read the fine print of Figure 1, you’ll see the disclaimer that the findings don’t include “Don’t Know” responses from CEOs. 

I’ve asked PwC how many Don’t Know responses it received. PwC hasn’t responded yet, and I hope that if Don’t Know answers were statistically significant, PwC would have said so in the original chart. But it underlines the point that perhaps some CEOs are divorced from the realities of how their employees work and what AI actually can or can’t do for them. 

Speaking of which… 

What Employees Say About AI

Our second glimpse of the AI frontier comes from consulting firm Section. It released a report last week finding that senior executives generally believe their AI adoption plans are a success, while almost everyone else in Corporate America believes AI is barely helping them at all. 

Figure 2, below (courtesy of the Wall Street Journal), tells the tale. Forty percent of workers say AI isn’t saving them any time at all in their daily routines; another 27 percent say the savings is less than two hours, and 20 percent more peg the savings at two to four hours. Put another way, 87 percent of workers say AI saves them less than 10 percent of their workweek, assuming a 40-hour week. Which is a huge assumption in the modern corporate workforce. 

Senior executives, on the other hand, see AI the other way around: 19 percent say it saves them at least 12 hours a week, another 24 percent say eight to 12 hours, and 33 percent more say four to eight hours. So 76 percent of senior executives say AI saves them more than 10 percent of their workweek. 

Both findings can be true at the same time. Senior executives make big strategic decisions and then delegate the execution of those decisions to lower-level employees. So if AI can bring more information to the senior execs’ fingertips and help to sharpen their analysis of big challenges, of course the C-suite would think AI is great.

Middle and lower-level employees, however, execute those decisions made by senior people. They follow processes. Sure, mid- and lower-level employees need information and analysis too, but most of the time they’re following established processes to make tactical decisions about how to do something. 

Plus, lower-level employees typically have been doing this work for years. They already know the processes. Their judgment is solid. They don’t need AI to provide advice, as much as they need it to do work — but if you can’t trust AI because it hallucinates or can’t juggle complex, judgment-intensive processes, then you spend more time fixing AI’s mistakes when you could’ve just done the job correctly yourself. 

That brings us to our third look at AI’s shortcomings. 

Losing Time on AI ‘Rework’

HR software giant Workday released a report last week finding that nearly 40 percent of AI’s efficiency gains are immediately lost to “AI rework,” such as correcting errors, rewriting content, and verifying outputs from one-size-fits-all AI tools. Only 14 percent of employees consistently get clear, positive net outcomes from AI. Maybe those 14 percent are all CEOs? I’m not sure.

Workday also found that younger employees (aged 25-34) account for nearly half of folks who spend the most time on AI rework — so perhaps my above quip about CEOs was on the mark after all. 

Plus, the amount of AI rework differs by roles. Employees in IT-related roles devote less time to AI rework, because those employees typically use AI for pattern recognition and data analysis, where AI excels. Employees in HR-related roles, however, devote more time to AI rework, because their work typically involves “people decisions, communications, and compliance-sensitive processes where ‘good enough’ is rarely acceptable,” Workday said. 

That’s especially interesting for compliance and audit crowds, since the compliance function is all about people decisions, communications, and (duh) compliance sensitive processes — so your embrace of AI should proceed slowly, assuming AI even makes sense for your work at all. Audit teams, on the other hand, might be able to use AI much more, because lots of their work does involve pattern recognition and data analysis.

Anyway, altogether these reports suggest that embracing AI within corporate organizations is going to be a long slog, despite CEOs’ daydreams to the contrary. Plan accordingly.