CISO Diaries
8 metrics to track cybersecurity performance
How can you know that you’re maximizing your cyber performance plans without tracking them? We share key tracking metrics to measure upskilling and gain board support.
Mags22,
Mar 14
2024
Table of Contents
Cyber performance programs invest in the growth and retention of your team. In cybersecurity, this looks like continuous hands-on upskilling, clear career development paths, and battling burnout and fatigue with engaging initiatives.
But this isn’t just about cybersecurity. It’s about aligning performance with your organization’s mission.
Why tracking cyber performance is important
Management consultant Peter Drucker famously said, “If you can't measure it, you can't manage it.”
How can you know that you’re maximizing your cyber performance plans without tracking them? And more importantly, how will key stakeholders witness growth and invest more budget in your cybersecurity initiatives without evidence that they’re working?
Metrics are the answer. Having the right measurements in place will enable you to optimize upskilling and overall cyber performance.
Monitoring the right metrics also means your team is better equipped to deal with emerging threats.
For example, there could be a new type of ransomware attack exploiting a recently discovered vulnerability (CVE).
You need to quickly assess whether your team, both blue and red, has the current skills to defend against this specific threat.
Tracking the ongoing training and certifications of your team members can give you immediate insight into their readiness and capability to handle such threats.
Academic research states that performance evaluation and benchmarking are a widely used method to identify and adopt best practices as a means to improve performance and productivity.
This methodology is particularly valuable when no objective or engineered standard is available to define efficient and effective performance. Leadership and management must be able to quantitatively define progress and improvement and that begins with understanding the starting point.
Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.
How to track security posture
When reporting to the wider business and C-suite, there’s a higher focus on financial implications and risk. Whilst these metrics can be greatly improved by cyber performance programs, it’s important to track the following and compare improvement throughout the cyber performance process.
After all, your cybersecurity team may have different individual goals and metrics based on their specific job roles, but the wider business will have a broader focus on security posture.
The magic happens when these two align through well-thought-out performance programs.
Recommended read: How to connect cybersecurity learning outcomes to company goals
Preparedness
How well prepared are your organization and cybersecurity team for threats and attacks? Consider the following:
-
False alarms: what’s the number of false positives and false negatives from security monitoring tools? Are these numbers being reduced by refining the monitoring process?
-
Security patches: how many organizational devices are up to date?
-
Testing: how often are disaster recovery and incident response plans tested via hands-on enhanced tabletop exercises? What were the results of the last test?
-
Vulnerabilities: regularly conduct vulnerability assessments and rectify issues.
Increase your preparedness level with HTB Enterprise
Hack The Box can help your teams prepare and mitigate security incidents with our comprehensive hands-on upskilling:
-
New Machines are released weekly featuring the latest CVEs and TTPs, for both offensive and defensive teams.
-
Our Professional Labs feature real-world scenarios to better prepare teams for real incidents.
-
Machines and Labs are highly hands-on and practical, improving muscle memory.
Security incidents
It’s important to track the number of security incidents throughout your cyber performance program, as these should begin to reduce as skills improve. Remember to ask the following questions:
-
Proactive security measures: how many incidents have been prevented due to measures such as endpoint protection, intrusion detection systems, and threat intelligence?
-
Number of security incidents: how many have been detected and resolved in the past month/quarter/year?
-
Downtime: what’s the average downtime as a result of each incident and how is this number being reduced?
-
Incident response plan: are you regularly testing and updating your response plan to ensure it’s still relevant and effective?
Cost per incident
Cybersecurity managers need to regularly track the cost per incident and measure a reduction in cost as employees get more experienced in their roles. The quicker they can respond and rectify an incident, the lower the cost and overall business impact.
Cost per incident covers:
-
Employee overtime.
-
Reduction in productivity.
-
Suspension of regular activities.
-
Loss of communication with customers, leading to lost sales, for example.
-
System downtime.
-
The cost of investigating the attack.
Time to detection and mitigation
With a better understanding of the threat landscape and vulnerabilities in general, are incidents detected and mitigated faster?
We can track this with mean-time metrics:
-
Mean-time-to-detect (MTTD).
-
Mean-time-to-resolve (MTTR).
-
Mean-time-to-contain (MTTC).
How to track cybersecurity performance
1. Metrics to optimize upskilling
To optimize upskilling, we need to have metrics and goals in place that align with both our team and company KPIs.
There’s no one-size-fits-all approach to upskilling metrics, but one thing that remains consistent is outcomes. If we determine exactly what our end goal is, we can assign meaningful metrics.
Before I want to know metrics, I need to know outcomes. Once I know the outcomes, I can gather the correct information.
I want to understand and know what my 6-month or 12-month training plan is for each individual. Normally this will be based on a work role from NIST/NICE. As they progress, they then can obtain higher salaries, new skills, etc. So my outcomes are based on frameworks.
Therefore, if I know I have a junior SOC analyst and I will allow them to have 5 hours a week to train, I want to know how long they are on the platform, when are they on the platform, and what they are doing on the platform.
Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.
Here are some strategies to put in place to track your metrics to optimize upskilling for individuals on teams:
-
Benchmark existing skills: track the percentage of team members achieving a specific score range in a Capture The Flag (CTF) event. This can be broken down into different skill areas (e.g., network security, application security) to identify specific skill gaps.
-
Assign upskilling programs: track the number or percentage of team members enrolled in and completing upskilling programs aligned with industry standards like MITRE ATT&CK or NIST/NICE frameworks. This also provides an incentive for employees to earn more as they progress through industry frameworks.
-
Set measurable goals: specific performance targets based on skill areas. For example, a goal could be to "increase the number of team members proficient in cloud security by 20% within the next six months."
-
Regularly assess and monitor progress: tracking improvements in scores or performance in periodic assessments, such as bi-annual CTF events, tabletop exercises (TTXs), or simulations.
Optimize cybersecurity upskilling with Hack The Box
-
Use our CTFs to benchmark your team’s capabilities and identify skills gaps.
-
Search and assign Labs based on MITRE ATT&CK framework techniques and tactics.
-
Track your team's activity, engagement, completion rate, and skill progression with our activity tracker.
2. Measure the impact of cyber performance
Now that we have metrics in place to optimize and track employee performance, we can measure the impact our cyber performance plans are having both on teams and the organization as a whole.
For organizations, it’s important to train and identify CVEs in a safe environment. This enables cybersecurity teams to put predictive concepts in place and take a proactive approach to security.
Having the right cyber performance plan in place can facilitate this, but you can measure the impact of cyber performance with metrics like:
-
% of time users spend upskilling.
-
% of upskilling program completed/certificates earned.
-
% increased team engagement.
-
% of decreased response time and improved recovery post-incident.
How does this look in action?
Before embarking on your cyber performance program, put together some statistics on data breach costs, for example. Once the upskilling has time to take effect, you can compare these costs from before and after the program. Ideally, they should be lower due to teams containing attacks faster.
Key stakeholders care most about reducing risk and taking care of their employees. If your cyber performance program can achieve this, the changes to your cybersecurity team and posture will be huge.
3. Track career development
As a cybersecurity leader, it’s essential to pay close attention to each individual when it comes to team development. In an industry where talent and knowledge are lacking, we must devote time and resources to the development of our teams.
To do this successfully, individual success metrics must also be decided and measured:
I would compare results to the cyber performance training we mapped out for the individual. Are they on track to meet all training requirements? What are the new skills they learned? Are we getting feedback on user satisfaction? Is the intent of the plan aligned with their actual goals and team goals?
Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.
Individual success benefits an organization as a whole, which requires regular tracking of defined metrics:
As an organization deeply invested in the upskilling of our IT security employees, particularly those in penetration testing and red teaming, we employ a comprehensive approach to track individual success.
Firstly, we monitor the progress of certifications and training. Our employees are encouraged to pursue relevant certifications like OSCP, CRTO, and others. We make 20% of working time available for training and further education. We track the number of certifications obtained, courses completed, or hours dedicated to training.
We also conduct regular skill assessments. These assessments cover various aspects of IT security, from network penetration to social engineering, enabling us to quantitatively measure skill enhancements.
In terms of project performance metrics, we evaluate how effectively our employees identify vulnerabilities, the complexity of the security issues they uncover, the time efficiency in system breaches, and the viability of the security solutions they propose.
Moritz Samrock, Red team manager, Laokoon Security.
Regularly review individual success by tracking the following metrics:
-
Job productivity: are they resolving more tickets? Remediating more vulnerabilities? Set targets for what you’d expect after a certain level of upskilling.
-
Feedback: are they satisfied with the upskilling program? What do managers and colleagues have to say about their improvements?
-
New skill acquisition: track which skills they are upskilling most frequently in.
-
Career development: are they on track to be promoted or move laterally in the company? Adopting more blue or red skills?