A Closer Look at SOC Audits

Anyone involved in cybersecurity or privacy compliance knows that one handy tool to assess your vendor risks is a SOC audit. Now, at long last, we have a report that explores an important question: Just what do all those SOC audit reports actually examine, anyway?

The report comes from CBiz MHM, a mid-sized accounting and advisory services firm that performs SOC audits for various clients. This week folks in the firm’s IT advisory practice published an analysis of more than 150 SOC 1 and SOC 2 audit reports (How are the two different? We’ll get to that shortly) to determine what issues the reports examine and the level of assurance they provide.

This research is important for several reasons. First, as more companies rely on outside vendors for mission-critical services or allow vendors to connect to your IT systems, the risk increases of a cybersecurity attack coming through your vendors to harm your IT systems. So you need assurance that those vendors have strong security protections themselves, and a SOC audit is one useful way to get that assurance. 

Second, however, is the pesky detail that SOC audits are not all performed according to a single, uniform set of criteria (unlike financial audits, which are). Instead, every SOC audit is based on whatever security and privacy issues are most important to the client seeking a SOC audit or the vendor undergoing one. 

So we can’t breezily declare something like, “Every SOC audit provides assurance for security and data privacy” and assume that statement means the same thing in all cases. It doesn’t. To understand what SOC audits entail, someone needs to do a case-by-case analysis of a large sample of them. 

Which brings us back to CBiz MHM and its analysis of SOC audits.

SOC Audits All Over the Map

The CBiz MHM study examined 116 SOC 1 reports, which assess the financial controls of a service provider; plus another 38 SOC 2 reports, which assess data security and privacy controls. In either type of audit, the number of objectives in a SOC report and the number of controls tested varied hugely.

How hugely? See Figure 1, below, as an example. CBiz MHM found that the average number of controls tested in a SOC 2 report was 98 — but that average came from a range of 38 to 233, which is enormous. Moreover, roughly one-third of SOC 2 audits were at the low end, nearly half were in a middle range, and 18 percent were at the high end.

 

This matters because it reinforces what I mentioned above: that a vendor can’t simply declare, “We have a SOC 2 audit so you, dear client, can rest easy that our security is sound.” 

No, dear client, you can’t. A SOC 2 audit that only focuses on security, and audits a few dozen controls, will be radically different from a SOC 2 audit that addresses security and privacy and process integrity, and tests hundreds of controls. The dear client’s internal audit team needs a clear understanding of that nuance. Otherwise you’re setting yourself up for wasted time at best, or worse, an incorrect sense of your vendors’ cybersecurity risks. Good luck explaining that to the audit committee when an attack happens.

None of this is to say SOC 2 audits are worthless. On the contrary, they can be much more cost-efficient than the alternative: the dear client’s CISO sending a lengthy security questionnaire to the vendor (potentially thousands of questions), which the vendor may or may not answer accurately; or the CISO paying someone to perform an on-site audit of the vendor, which costs lots of money. 

The real issue surfaced in the CBiz MHM study is the importance of knowing how to scope a SOC 2 audit appropriately — so that both the vendor and the client have an assurance document they can actually use, rather than everyone reinventing the cyber assurance wheel over and over again.

I’ve written about that need for proper scoping of SOC 2 audits before. As large corporations continue to rely on more vendors for more tasks, and integrate more suppliers ever more closely into complex supply chains, that need for proper scoping will only increase. 

Productive SOC 2 Conversations

The CBiz MHM people presented their analysis of SOC reports at a recent conference of the Institute of Internal Auditors’ Northeast District, which I attended; and during their presentation they offered other good advice about how to put SOC reports to good use. I took detailed notes. 

First, for vendors considering whether to undergo a SOC audit: consult with advisers to understand your customer service requirements. Get a clear sense of what you believe should be in scope for a SOC audit, based on the services you provide to customers. Then you can guide customers through that conversation, with a “We think this is what matters most” approach. They’ll usually agree with most of what you propose. 

Don’t simply ask customers what they want included in a SOC audit, because they’ll just ask for it all. That puts you in a state of what I call over-compliance, where you’ve spent time and money providing assurance for risks that don’t actually exist in your customer relationship.

Vendors should also look to their peers, to see how many controls they’re testing and documenting in their SOC audits. If your SOC auditor is testing considerably more, your auditor might be taking you for a ride.

Second, for customers reading the SOC reports: always look to see which audit firm has performed the audit. Plenty of small or boutique audit firms claim that they can perform a SOC audit, but some might be quacks. 

Also look to see whether your vendor has any sub-service providers, and whether those sub-service people are in the scope of your SOC report. CBiz MHM found that the average service provider had 2.3 sub-service providers, but 18 percent listed no sub-service providers (which seems implausible to me). 

Or, your vendor might have a non-disclosure agreement with its sub-service provider, so you can’t see that sub’s own SOC report. In that case, review the vendor’s own vendor management program, to see whether that program gives you the warm fuzzies or the cold clammies. 

Last bit of advice: look closely at the time between the end of the SOC audit period and the issuance of the SOC report. Some vendors have SOC reports turned around within several days; others take as long as 18 months. Both are sketchy. A few months from end of audit period to SOC report in your hands is more reasonable.