Measures and Metrics: How to Track More Than Just the SOC

“If you can’t measure it you can’t improve it.”

Security teams often report metrics like threats blocked or alerts received, providing little value to leadership. These numbers don't accurately reflect security function success or introduce accountability. How can we do better?

A perfect state vision for cyber security is a well-understood risk profile with sufficient controls and automated reporting to verify their effectiveness. This talk will show you how to get started, answering questions like:

• How do I know if my controls are working?
• How do I measure control effectiveness quantitatively?
• What's the difference between a metric and a measure?
• What metrics should a CISO, GRC Analyst, CEO, or Board see?

By the end, we'll articulate success more specifically:

Before: ""The ransomware risk is low because we have backups, MFA, and antimalware.""

After: ""The ransomware risk is low because critical controls meet KPIs: 100% successful backups and DR testing in 12 months, 98% standard and 100% privileged users with MFA, 94% MFA bypass prevention.""

Before: ""The email filter is on and configured to block attacks.""

After: ""The email filter has 95% precision and 81.9% recall. Recall is below our 90% KRI, requiring review. Low recall indicates under-classification of malicious emails.""

These insights provide accountability and clarity, leading to:

• Real-time security posture updates
• Faster compliance reporting
• Alerting when control effectiveness falls below satisfactory levels

Key points explored, with practical examples aligned with NIST SP 800-53:

1. Identifying Meaningful Metrics:

• Using existing controls to identify meaningful metrics
• Examples beyond SOC: incident response times, vulnerability remediation rates, user behavior analytics
• Prioritising metrics that reveal control and process effectiveness

2. Practical Examples from Fictional Companies:

• Case studies across industries and maturity levels
• Applying metrics in different contexts, emphasising NIST SP 800-53 adaptability

3. Guidance from NIST SP 800-55:

• Introduction to the Measurement Guide for Information Security
• Identifying and selecting metrics aligned with organisational goals
• Four types of metrics: Implementation, Effectiveness, Efficiency, and Impact

4. Data Collection and Analysis:

• Methodologies for accurate data collection
• Automated tools for data aggregation and analysis
• Importance of context in data interpretation

5. Reporting and Communication:

• Strategies for presenting metrics to different stakeholders
• Role of visualisation tools
• Regular reporting cycles and adapting to evolving threats

Attendees will learn to develop and implement a robust metrics program beyond traditional SOC reporting, enabling them to collect, analyse, and communicate meaningful security metrics that reflect true security posture and drive improvements in risk management.