Cyber Security Institute

§ Current Worries

Top 3 Worries

  • Regulations
  • Old Firewall Configurations
  • Security Awareness

§ Listening

For the best information

  • The underground
  • Audible
  • Executive Excellence
  • Music (to keep me sane)

§ Watching

For early warnings

  • 150 Security Websites
  • AP Newsfeeds
  • Vendors

Sunday, August 20, 2006

How to Use Metrics

Knowing what to look for and how to analyze it can spell success for a security operation and the organization it serves.  The fact that established metrics and measures for the full range of security programs are few and far between tells a story about the historical disconnect between these functions and the core businesses they serve.  The risk environment has changed significantly over the past 30 years, with shocking wake-up calls to CEOs, boards and shareholders.  Attentive corporations have had to address the exposures uncovered in these times with more sophisticated and mainstream corporate security organizations.  With this mainstreaming comes the obligation to measure performance and demonstrate bottom-line contributions.  Metrics are a natural descendant of this process.  It is also essential that we recognize security’s contribution to the corporate system of internal controls.

Internal controls, established to mitigate a variety of business risks, provide the dashboard to inform management on the status of core activities and to apply the brakes that keep the enterprise safely on course.  The security organization plays a critical role in identifying, measuring, preventing and responding to a growing inventory of risks.  We must be able to measure the probability and potential consequences of an identified risk, or management has no gauge to assess and prioritize what actions to take.  Metrics are central to understanding the adequacy of security controls and where to focus our limited resources for the greatest contribution to the protection strategy.

This excerpt from the book Measures and Metrics in Corporate Security, Communicating Business Value gives a few examples of the ways CSOs can think about the data they collect as part of their security operations and identifies what is important to measure, and how to communicate with senior business executives about what the data indicates about their organization’s risk environment and how it’s being managed.

Security programs gather volumes of data every day.  The successful security executive defines his business plan and the performance of resources and services around clearly articulated measures.  Those measures should be aligned with core business strategy and priorities.

Figure 1 illustrates how a CSO has evaluated the importance of various security metrics, based on their relevance to business drivers such as managing costs and risks, focusing on return on investment, complying with the law and company policies, and protecting the lives and safety of employees.  Note the last column on the right, which is checked every time: internal influence.

Effective use of metrics that matter to business leadership, demonstrating the value of security operations, wins a security executive important capital.  Every CSO should have half a dozen dials to watch on a regular basis.  These indicators could be “survival metrics,” the hot buttons on a dashboard you are expected to address that monitor the wellness of your organization or an issue of particular concern to management.

You may find that you have more than one dashboard—-yours and the one your boss and a few key players expect you to watch and report on.  The CFO could be an excellent resource to advise you on the presentation of dashboard metrics since this officer typically reports performance metrics to management on a regular basis.  While these dashboards view an array of priorities, you need first to identify what risks are important.

One way to drill down on a particular risk and determine its priority level is through risk mapping.  Risk mapping is about plotting the dynamics of the risk incident landscape.  A presentation model of risk dynamics or risk profiling may be found in the risk map on Figure 2.  More consequential incidents are at the top of the map, and more frequent ones are to the right.  In Figure 2, eight types of internal misconduct cases were plotted for the month, and the five highlighted all had inadequate supervision and poor policy awareness as contributing causes of the infractions.

There is a valuable story to be told to management, and it is particularly useful in quarterly or annual presentations to display notable trends, their contributing causes and suggestions for mitigation tactics.  Measures mapping helps you do that by looking at areas of risk, the contributing causes to those risks and actions implemented to mitigate those risks, and then measuring the effectiveness of those actions.

It’s a CSO’s job to find the appropriate model for security measurement and reporting objectives that fits his organization.

Posted on 08/20