CISO Diaries

13 min read

8 metrics to track cybersecurity performance

How can you know that you’re maximizing your cyber performance plans without tracking them? We share key tracking metrics to measure upskilling and gain board support.

Mags22 avatar

Mags22,
Mar 14
2024

Cyber performance programs invest in the growth and retention of your team. In cybersecurity, this looks like continuous hands-on upskilling, clear career development paths, and battling burnout and fatigue with engaging initiatives. 

But this isn’t just about cybersecurity. It’s about aligning performance with your organization’s mission. 

Why tracking cyber performance is important 

Management consultant Peter Drucker famously said, “If you can't measure it, you can't manage it.”

How can you know that you’re maximizing your cyber performance plans without tracking them? And more importantly, how will key stakeholders witness growth and invest more budget in your cybersecurity initiatives without evidence that they’re working? 

Metrics are the answer. Having the right measurements in place will enable you to optimize upskilling and overall cyber performance. 

Monitoring the right metrics also means your team is better equipped to deal with emerging threats. 

For example, there could be a new type of ransomware attack exploiting a recently discovered vulnerability (CVE). 

You need to quickly assess whether your team, both blue and red, has the current skills to defend against this specific threat. 

Tracking the ongoing training and certifications of your team members can give you immediate insight into their readiness and capability to handle such threats.

Academic research states that performance evaluation and benchmarking are a widely used method to identify and adopt best practices as a means to improve performance and productivity.

 

This methodology is particularly valuable when no objective or engineered standard is available to define efficient and effective performance. Leadership and management must be able to quantitatively define progress and improvement and that begins with understanding the starting point.

 

Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.

How to track security posture

Cybersecurity posture checklist
 

When reporting to the wider business and C-suite, there’s a higher focus on financial implications and risk. Whilst these metrics can be greatly improved by cyber performance programs, it’s important to track the following and compare improvement throughout the cyber performance process. 

After all, your cybersecurity team may have different individual goals and metrics based on their specific job roles, but the wider business will have a broader focus on security posture. 

The magic happens when these two align through well-thought-out performance programs. 

Recommended read: How to connect cybersecurity learning outcomes to company goals

Preparedness

How well prepared are your organization and cybersecurity team for threats and attacks? Consider the following:

  • False alarms: what’s the number of false positives and false negatives from security monitoring tools? Are these numbers being reduced by refining the monitoring process? 

  • Security patches: how many organizational devices are up to date? 

  • Testing: how often are disaster recovery and incident response plans tested via hands-on enhanced tabletop exercises? What were the results of the last test?

  • Vulnerabilities: regularly conduct vulnerability assessments and rectify issues. 

Increase your preparedness level with HTB Enterprise

Hack The Box can help your teams prepare and mitigate security incidents with our comprehensive hands-on upskilling:

  • New Machines are released weekly featuring the latest CVEs and TTPs, for both offensive and defensive teams.

  • Our Professional Labs feature real-world scenarios to better prepare teams for real incidents.

  • Machines and Labs are highly hands-on and practical, improving muscle memory.

 

Security incidents 

It’s important to track the number of security incidents throughout your cyber performance program, as these should begin to reduce as skills improve. Remember to ask the following questions:

  • Proactive security measures: how many incidents have been prevented due to measures such as endpoint protection, intrusion detection systems, and threat intelligence?

  • Number of security incidents: how many have been detected and resolved in the past month/quarter/year?

  • Downtime: what’s the average downtime as a result of each incident and how is this number being reduced? 

  • Incident response plan: are you regularly testing and updating your response plan to ensure it’s still relevant and effective?

Cost per incident 

Cybersecurity managers need to regularly track the cost per incident and measure a reduction in cost as employees get more experienced in their roles. The quicker they can respond and rectify an incident, the lower the cost and overall business impact. 

Cost per incident covers:

  • Employee overtime.

  • Reduction in productivity. 

  • Suspension of regular activities. 

  • Loss of communication with customers, leading to lost sales, for example. 

  • System downtime. 

  • The cost of investigating the attack. 

Time to detection and mitigation

With a better understanding of the threat landscape and vulnerabilities in general, are incidents detected and mitigated faster?

We can track this with mean-time metrics:

  • Mean-time-to-detect (MTTD).

  • Mean-time-to-resolve (MTTR).

  • Mean-time-to-contain (MTTC).

How to track cybersecurity performance

Track performance programs
 

To optimize upskilling, we need to have metrics and goals in place that align with both our team and company KPIs. 

There’s no one-size-fits-all approach to upskilling metrics, but one thing that remains consistent is outcomes. If we determine exactly what our end goal is, we can assign meaningful metrics.

Before I want to know metrics, I need to know outcomes. Once I know the outcomes, I can gather the correct information. 

 

I want to understand and know what my 6-month or 12-month training plan is for each individual. Normally this will be based on a work role from NIST/NICE. As they progress, they then can obtain higher salaries, new skills, etc. So my outcomes are based on frameworks.

 

Therefore, if I know I have a junior SOC analyst and I will allow them to have 5 hours a week to train, I want to know how long they are on the platform, when are they on the platform, and what they are doing on the platform.

 

Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.

Here are some strategies to put in place to track your metrics to optimize upskilling for individuals on teams:

  • Benchmark existing skills: track the percentage of team members achieving a specific score range in a Capture The Flag (CTF) event. This can be broken down into different skill areas (e.g., network security, application security) to identify specific skill gaps.

  • Assign upskilling programs: track the number or percentage of team members enrolled in and completing upskilling programs aligned with industry standards like MITRE ATT&CK or NIST/NICE frameworks. This also provides an incentive for employees to earn more as they progress through industry frameworks. 

  • Set measurable goals: specific performance targets based on skill areas. For example, a goal could be to "increase the number of team members proficient in cloud security by 20% within the next six months."

  • Regularly assess and monitor progress: tracking improvements in scores or performance in periodic assessments, such as bi-annual CTF events, tabletop exercises (TTXs), or simulations.

Optimize cybersecurity upskilling with Hack The Box

  • Use our CTFs to benchmark your team’s capabilities and identify skills gaps.

  • Search and assign Labs based on MITRE ATT&CK framework techniques and tactics.

  • Track your team's activity, engagement, completion rate, and skill progression with our activity tracker.

 

Now that we have metrics in place to optimize and track employee performance, we can measure the impact our cyber performance plans are having both on teams and the organization as a whole. 

For organizations, it’s important to train and identify CVEs in a safe environment. This enables cybersecurity teams to put predictive concepts in place and take a proactive approach to security. 

Having the right cyber performance plan in place can facilitate this, but you can measure the impact of cyber performance with metrics like: 

  • % of time users spend upskilling.

  • % of upskilling program completed/certificates earned. 

  • % increased team engagement.

  • % of decreased response time and improved recovery post-incident.

How does this look in action?

Before embarking on your cyber performance program, put together some statistics on data breach costs, for example. Once the upskilling has time to take effect, you can compare these costs from before and after the program. Ideally, they should be lower due to teams containing attacks faster.

Key stakeholders care most about reducing risk and taking care of their employees. If your cyber performance program can achieve this, the changes to your cybersecurity team and posture will be huge.

As a cybersecurity leader, it’s essential to pay close attention to each individual when it comes to team development. In an industry where talent and knowledge are lacking, we must devote time and resources to the development of our teams.

To do this successfully, individual success metrics must also be decided and measured: 

I would compare results to the cyber performance training we mapped out for the individual. Are they on track to meet all training requirements? What are the new skills they learned? Are we getting feedback on user satisfaction? Is the intent of the plan aligned with their actual goals and team goals?

 

Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box.

Individual success benefits an organization as a whole, which requires regular tracking of defined metrics: 

As an organization deeply invested in the upskilling of our IT security employees, particularly those in penetration testing and red teaming, we employ a comprehensive approach to track individual success.

 

Firstly, we monitor the progress of certifications and training. Our employees are encouraged to pursue relevant certifications like OSCP, CRTO, and others. We make 20% of working time available for training and further education. We track the number of certifications obtained, courses completed, or hours dedicated to training.

 

We also conduct regular skill assessments. These assessments cover various aspects of IT security, from network penetration to social engineering, enabling us to quantitatively measure skill enhancements.

 

In terms of project performance metrics, we evaluate how effectively our employees identify vulnerabilities, the complexity of the security issues they uncover, the time efficiency in system breaches, and the viability of the security solutions they propose.

 

Moritz Samrock, Red team manager, Laokoon Security.

Regularly review individual success by tracking the following metrics:

  • Job productivity: are they resolving more tickets? Remediating more vulnerabilities? Set targets for what you’d expect after a certain level of upskilling.

  • Feedback: are they satisfied with the upskilling program? What do managers and colleagues have to say about their improvements?

  • New skill acquisition: track which skills they are upskilling most frequently in. 

  • Career development: are they on track to be promoted or move laterally in the company? Adopting more blue or red skills?

You can track an individual’s completion rate and skill progression with HTB Enterprise, mapped to MITRE ATT&CK frameworks.

8bit transforms the way their teams upskill with HTB

Using HTB as their workforce development platform transformed 8bit’s upskilling process.

Measurable metrics and progress indicators are one of the main reasons why the 8bit team managed to seamlessly onboard five junior team members, practicing on over 300 live targets in just 10 weeks.

 

One of the key pillars of cyber performance is to attract, engage, and retain cybersecurity employees, minimizing the fatigue and burnout that plague the industry. 

By providing upskilling opportunities, clear career paths, and engaging team activities, you can drastically reduce burnout, boosting productivity and retention. This helps improve employee’s overall well-being and can positively impact mental health, which is a key concern in the demanding world of cybersecurity. 

After speaking with 803 active cybersecurity professionals in the HTB user base, we found that more than 70% of managers view team events like CTFs as a viable way to boost employee engagement.

What’s more, 68% of security team members rated “opportunities to learn skills” as the most successful way of staying engaged at work.

cybersecurity survey
 

Employee engagement and retention often require a lot of observational metrics to track, by seeing a shift in behaviors, you’ll identify whether employees are satisfied with their roles and professional development: 

Another key metric is the contribution to knowledge sharing. This includes internal contributions to our knowledge base, as well as external engagement like articles, workshops, or talks.

 

Innovation is at the heart of what we do. We track the development of new tools, scripts, or methodologies for penetration testing and red teaming, recognizing the impact and originality of these innovations.

 

Peer and supervisor feedback is integral. This qualitative measure helps us assess soft skills, which are essential in our collaborative and dynamic work environment.

 

Participation in competitions and CTFs (Capture The Flag) is another metric. These events are excellent for applying and testing skills in real-world scenarios.

 

Lastly, we encourage and track contributions to research and publications in the field. This not only enhances personal growth but also contributes to the broader cybersecurity community.

 

Moritz Samrock, Red team manager, Laokoon Security.

So, how can you practically track employee engagement and retention?

  • Measure employee Net Promoter Score (eNPS): how many cybersecurity employees are likely to recommend your organization as a place to work? 

  • Conduct employee engagement surveys to monitor job satisfaction and engagement before and after upskilling. 

  • Track voluntary employee turnover rate by monitoring your voluntary employee turnover rate. Highly engaged employees are less likely to resign, monitor how this number changes are investing in employee performance. 

Prove that cybersecurity workforce development works

Security leaders are on the frontlines of change. And it falls on them to wave the flag for improving cyber performance and security posture. 

By actively tracking these metrics, you’re able to prove that investing in cybersecurity teams is no longer optional, it’s essential for successful cyber performance and an improved security posture. 

This proactive approach to cybersecurity is a requirement for today’s threat landscape, making these metrics vital to your organization.

Become an HTB Subject Matter Expert

Are you a cybersecurity professional who wants to contribute to articles like this one?

Join the HTB SME program and you’ll benefit from the following:

  • A huge audience: Our HTB community has over 2.7 million members, giving you a huge platform to share your knowledge with and feature in our editorial content.

  • Networking opportunities: Meet other HTB SMEs and expand your professional network, meeting people you may never have connected with if it wasn’t for your involvement with HTB.

  • Recognition: Any insights you provide will be credited to you, including your name, title, and LinkedIn profile. Some content will get additional love on social media too!

  • Share your experiences: We’re giving a voice to cybersecurity professionals, providing you with a platform to share your knowledge and experiences.

  • Help others: Your unique insights are incredibly valuable to other cybersecurity professionals, or even individuals just starting their careers.

💡Find out more on our blog: Become an HTB Subject Matter Expert

 

Dan Magnotta (Mags22), HTB Federal Business Development & Capture Manager, Hack The Box

Dan Magnotta is an accomplished professional in cybersecurity and intelligence operations with more than a decade of experience in the military and private sectors.

His career began with dedicated service to the U.S Department of Defense, where he played critical roles in the U.S. European Command and U.S. Special Operations Command Europe, contributing significantly to cutting-edge cyber strategies.

In addition to his civilian role, he serves as an LCDR in the U.S. Navy Reserve, showcasing his leadership and dedication as an Executive Officer for a Navy Reserve Unit. His expertise in cybersecurity, operational analysis, and strategic planning is extensive.

At Hack The Box, he tailors solutions to meet the unique requirements of government agencies and organizations worldwide, leveraging his deep understanding of both military and civilian cybersecurity needs.

Hack The Blog

The latest news and updates, direct from Hack The Box