6 new metrics for measuring incident response using automation

What can organisations use in order to ensure they maximise the cost-efficiency of their incident response performance?

As security teams redefine their organisation’s security infrastructures to deal with the changing threat landscape, they often need to demonstrate how those changes will positively impact the bottom line of the business.

However, in 2016, the SANS Institute found that 71% of organisations do not have regular metrics or measurements for incident response (IR) performance. While this percentage has decreased in recent years – 58% declared not having metrics in 2017 – many security teams will only see the need to measure their performance following a major cyber incident.

>See also: Security operations: does automation have to mean automatic?

By automating incident response processes, can help companies develop a whole new approach to cybersecurity metrics, reduce costs, increase efficiencies and provide actionable goals for managers, not only strengthening their security operations, but ensuring that they security approach is aligned with the business. The six new metrics include:

1. Cost per incident (CPI)

The CPI metric can be measured as the duration of an incident multiplied by the average hourly rate for a tier one analyst. Many security teams will run that formula through the IR playbook for each phase of an incident from detection – to response and remediation.

>See also: Security automation: boosting IT productivity and network resilience

Using automation, that not only eliminates entire steps or workflows, but delivers acceleration, huge cost savings and improve efficiency as teams can focus on validating and concluding incidents, rather than wasting time on a wild goose chase.

2. Automatic detection versus manual detection

Security teams can establish a baseline for determining the ratio of detections the security stack produces versus the combined number of human detections received.

To figure out the human detections, security teams must determine the number of staff detections, such as an employee recognising that their machine is malfunctioning or an IT admin recognising that a system is performing in unusual ways. Add to this the number of external detections – such as the number of times the security team gets a call from government/IT admins – and the number of detections security operations staff created manually synthesising data from their security stack or SEIM, and this will give organisations a sense of the efficiency of the current system.

>See also: Mobility demands that security walks hand-in-hand with automation

By automating incident response processes, companies can expect the ratio to tilt substantially towards the automation side of the equation, which means better security operations efficiency.

3. Percent investigation versus volume

Security operations can now determine what is slipping through the cracks. By measuring investigations versus alert volume, companies can measure the risk gap in current security operations. With automated incident response processes, the percent of investigations per volume will increase dramatically.

For example, if an organisation is typically performing three investigations for every 100 alerts (3/100 or 3%) and then implements automation, which sees a 10% alert-to-conclusion rate and an additional two investigations (5/10 or 50%), that yields a massive 1,500 percent increase to security operations effectiveness.

4. Ratio of investigation to response

It is in the interest of any organisation to ensure that the security operations team is wasting as little time as possible – and the ratio of investigation to response metric can help determine how many items that were investigated lead to a response workflow going through completion.

>See also: Cyber security predictions for 2018

Automating incident response processes will lead to a convergence of investigations to response, since more investigations are against validated conclusions, rather than merely suspected attacks.

5. Rate of decision

This metric measures the time it takes to make a decision following the generation of an alert. It is not uncommon for ‘analysis paralysis’ and security operations uncertainty to increase dwell times and risk the spread of an attack. It also takes time away from investigating and responding to other attacks that may be happening at the same time.

By measuring the decision rate both before and after implementing automation, the security operations team can demonstrate their nimbleness and increase response capacity without adding scarce people resources.

6. Remediation response vs reimage

This metric measures business disruption. The more surgical, remote responses that are enabled by automating processes, the fewer “big hammer” fixes of reimaging an end-user’s endpoint have to happen – meaning less business disruption and inconvenience for employees. It is easy to see how taking someone’s laptop away for a day, or taking down a payment processing server, can severely disrupt security operations – even when hot backups and clustered failovers are part of the solution.

>See also: The cyber security challenge for retail branch IT

Automating incident response processes avoids such disruptions through the creation of actionable rules that can automatically kick off surgical remediation and deep analysis.

Metrics as enablers

As incident response processes continue to be transformed by automation, proving how it transforms incident response performance will be close to impossible without the enablement of these new metrics.

Put simply, each of these new metrics can demonstrate how the automation of incident response processes can not only strengthen your security infrastructure, but also impact the bottom line.


Sourced from Andrew Bushby, UK director at Fidelis Cybersecurity

Avatar photo

Nick Ismail

Nick Ismail is a former editor for Information Age (from 2018 to 2022) before moving on to become Global Head of Brand Journalism at HCLTech. He has a particular interest in smart technologies, AI and...

Related Topics

Incident Response