How SOC 2 auditors test

You are ready for the SOC 2. You have chosen a SOC 2 auditor and you are confident that your controls are working. Now what? Getting audited can be a mysterious and nerve wracking process. It helps to have a general idea of what you can expect. While each auditor has their own nuanced approach and methodology, most of them will follow a similar process.

An auditor’s job is to prove that all of your controls are working as expected, meaning they will test the controls that you have selected. They will test every single control, but will use a method called statistical sampling to limit how much evidence is tested for each control. For example, if you made over 1000 changes to code, they are not going to test all 1000 changes, but will select a random sample. You won’t know which samples they plan to test, so all of your control evidence should be audit-ready. 

Sometimes you can predict exactly what they want, for example, the auditor is guaranteed to test any control that is performed annually. For controls that are performed at some other frequency, they will test via sampling. It is rare that an auditor will test 100% of a population (unless it is an annual control and the sample size is one). Therefore you want each and every control to be operating perfectly - you can’t be sure which samples they will select.

A few weeks before the start of your audit, your auditor will send you a document request list (sometimes called a PBC or provided by client list). This list is based on the controls you have identified for them. As you begin to deliver your documents, they may have follow up questions or request additional bits of evidence. It’s normal to have some back-and-forth communication during this phase.  

Frequency drives audit sample size

The frequency with which each control is performed will influence which documents the auditor requests, and the sample size they will select. If you have a control that is performed only when an event happens (new-hires are on boarded randomly), then that is ad hoc. The auditor will look at each control and how often it is performed, and then extrapolate a sample.

The sampling math happens behind the scenes and gets distilled into a testing approach that may look like this:
This is an example. Each audit firm will have their own approach, but different auditors will have comparable approaches unless you are in a high risk industry or have a high risk profile (meaning you have a history of botching your controls). Higher risk will result in larger sample sizes.

Audit techniques

The auditor will use one or a combination of testing approaches. You can expect to see the following techniques, for example:

• Inquiry: They will ask the control owner to describe how the control works. This is their least preferable method for testing a control, but they use it a lot to ‘spot check’ whether or not control owners know what they’re doing. Don't be surprised if an auditor asks what appears to be a really obvious question. It is a required due diligence step in their testing.  
• Observation: They will watch over your shoulder as you perform a control in order to confirm that the control works. This can be done in person, or virtually. This is especially useful for showing an auditor that a server room door is actually locked or that an error message pops up on a screen when a person without proper access tries to log in.  
• Inspection: They will examine the sample or samples to make sure all the bits and pieces of the control are working and are present as promised. This technique is used a lot when testing samples. For example, were all the steps in a change management process appropriately captured in a change ticket? Were NDA’s signed for a sample of new hires prior to onboarding?
• Reperformance: The auditor will try to re-do the control and see if they get the same result(s). This, along with Inspection, are the most preferable testing methods for auditors. This one is often used to test user access reviews in larger organizations.

Your auditor may also test a few other control characteristics, depending on the nature of the control. They will want to ensure that your controls are:

• Timely — Did the control happen when it was supposed to? You want to perform each control as close as possible to the cadence you have defined for it.  For example, if emergency changes are required to be reviewed within 24 hours of deployment, you want to make sure that the review is indeed in that time frame. 
• Complete and accurate — Did the control cover everything it was supposed to? If you are performing user access reviews, for example, are you positive you have reviewed ALL in scope systems? Are you positive the parameters you used to pull each report did not inadvertently skip a few users?
  
  

Tips for a great audit

There are few helpful things you can do when providing evidence that will ingratiate you to your auditor.

• If your piece of evidence is odd, add a note to your auditor to help them understand any nuances.
• When you take a screenshot, try to capture the date and time stamp from your desktop.  This lets auditors know the screenshot is current.
• When you run a report, include a screenshot of any parameters you used to create the report. Reports that are generated out of a system are ideal.
• Don’t give the auditor more than what they ask for. No busy work!
• If an auditor includes an evidence reference number with their request, add that number to the evidence file name.
• Only answer the questions they ask — no need to embellish. Avoid taking them down a rabbit hole. If they want to know more, they will ask.