Tuesday, November 15, 2011

ALARP as a risk benchmarking target?

ALARP (As Low As Reasonably Practicable) is a fundamental concept in risk management. In essence, it represents the concept that we should mitigate threats (negative risks) down to the level where the expenditure of resources is balanced against the benefit.   As the saying goes, a picture paints a thousand words so this concept is probably best illustrated in Figure 1 below.

FIgure 1: ALARP - the point where risk and resources achieve optimal balance
Understanding the concept and measuring it in practice are however two entirely different things.  There are a number of challenges that can conspire against us in this mission, the main two being:

  • You need a consistent performance scorecard with at least 2 years of historical data. Unless there is some barometer that can give a quantitative or semi-quantitative level of risk, there is no way to objectively measure risk levels 
  • Much of what we spend on risk management is 'hidden' from view. It's relatively easy to quantify the cost of insurance, hedging, risk management training budgets etc but that is just the tip of the iceberg. A simple example of this would be an organization that relocates its office for security or safety reasons. It's easy enough to say that the organization spent $x on relocating and an extra $y/mth in rent but few organizations actually track this difference, much less allocate it to risk management.

If we want to understand when we are in the ALARP region (and it's more accurate to think of it as a 'region' rather than a point value), then the first step is to do the analysis that is required in order to build a performance scorecard. More on that later but here are some SIMPLE questions to consider:

  • Scope - What is the scope of our scorecard? The whole organization? Or do we want to separate out each division, each geographic location or each separate facility? Maybe just the safety and health program, or perhaps Health, Safety, Environment and Security?
  • Indicators - What are the specific indicators that we care about?  Are we more interested in lag or lead indicators? Do we have enough information already and if not, what would we need to track?
  • Measures - How do we measure success? At what point do we read the optimal resource/risk balance? Is there a specific number? If not, can we use client (internal or external) surveys to provide a semi-quantitative measure as a proxy for risk?
  • Performance - What long term performance are we seeking? How will our ALARP strategies and measures contribute to organizational performance?
  • Longitudinal data - What period of time do we want to consider? Will we measure monthly, quarterly, annually? What is the lifespan of our performance benchmarking tool? If we align it to our risk management framework, it will need to be revised when the framework is revised. Will we need to modify historical data to keep our long term trend information meaningful?
  • Excellence - How will we achieve excellence? How will we know when our benchmarking is excellent? Is it when the CEO or Board signs it off? Will we consider it excellent when it's working to plan? Or when external independent auditors sign it off? Equally, what do we need to do to achieve true excellence? How will we track hidden risk management costs? Eg: Will we create cost codes to track the hidden risk treatment costs such as the extra cost of rent as in the example above?

As you can probably see, SIMPLE doesn't mean simple, but lets start with the concept of ALARP as being important to our risk performance benchmarking. If we're really serious we also need to think about how to measure the AHARP concept (As High As Reasonably Practicable) for positive risks (aka. opportunity realization). More on AHLARP at this link but that's an idea for another day.

Friday, November 11, 2011

Three fundamentals of KPIs

Measuring performance seems complex (and it is) but let's not lose sight of the basics. The trick with key performance indicators (KPIs) comes down to three questions:
  • What is KEY? What are the long term objectives of the enterprise?
  • What is PERFORMANCE? What results do you aim for, what is success, what is failure, what are the acceptable ranges? 
  • What are INDICATORS? What metrics define desired performance, at reasonable cost? 
What do you really care about? You could apply this question to your personal preferences in life but what we are talking about here is what is important to your organization.  Organizations don't have a single identity so often you're trying to amalgamate a variety of disparate views, values and opinions. Certainly, you can be guided by policy statements, organizational objectives and strategy documents, and they are probably the first place to start looking for answers. In the end however, someone needs to be accountable to define what is actually 'important'.  Depending on the scope of your study, this could be the CEO, the budgetholder, Chairman of the Board or simply the line manager responsible for that specific area.

Here are a few questions that might guide your investigations when establishing what is 'key':

  • What is the scope of this KPI?  The whole organization? A small project? My workgroup?
  • What couldn't the organization survive without? You could take a 'red team' approach to this by getting a team together to ask "How would I attack or cause this organization to fail?"
  • What are we trying to achieve here? Ie. What are the objectives of this organization/group/project?

As the adage goes "what gets measured, gets managed".  It's essential to have some simple and clearly defined view of what performance means.  That means you either need to ask binary questions (Eg: Is our gross profit greater or less than 10% of turnover?) or measure it quantitatively (Eg: What is our nett profit?). 

In some cases, you might also choose to use some semi-quantitative questions. These aren't quite as good as pure quantitative data but can at least provide some sort of ordinal measure (Eg: Is the organization improving our score each year?).  A simple example of a semiquantitative score can be found in the figure below. This simple example looks at converting some simple 'word pictures' into an ordinal scale. One of the limitations of this scale is that it's primarily subjective. That doesn't mean that it's not useful however. If you can think of 10 or more areas to measure, and are rigorous with developing the criteria for each of the four grades, you'll end up with a fairly good idea as to where your weaknesses are and whether or not the organizations performance is increasing over time.
Figure 1: Example of a Semi-Quantitative Scale

Here are a few questions that might guide your investigations when establishing how to measure 'performance':

  • What is the timeframe that I'm interested in?
  • What would we consider as catastrophically bad?
  • What would we like to be able to report to the Board/boss/shareholders? 
  • Would data would we need?
  • What information/data/statistics do we already have?
  • How much would it cost to get ALL the data that we'd need and how would we go about acquiring it?


What are the metrics that can help us measure our performance? This question above all others is entirely dependent on context. An insurance company might have a wealth of data about it's clients risks but if it is developing KPIs for financial performance

Here are a few questions that might help with developing 'Indicators'':

  • Would data would we already have?
  • What information would would we like that we don't already?
  • How much would it cost to get all the data that we'd need and how would we go about acquiring it?
  • Are there industry standards or legislated reporting requirements that we can piggyback on?
  • What compliance framework do we already have?
  • What systems (accounting, safety, incident reporting, sales, etc) do we have in place? How could I modify or integrate those systems to get a better result?
  • Which are lead and which are lag indicators?

Keep at it. Ask lots of people for help, input and go with the simple questions. What's important to you? How would you measure it? What information would help you understand your business? Once you can get to these answers, (which is every bit as hard as it sounds) you have the exact set of KPI's your organization needs. More is waste, less is mismanagement.