The Reengineering of Risk Assessment

It's amazing how often we come across companies that are spending millions of dollars having their employees rate risks using ordinal or Likert scales within a commercially purchased tool or in a home-grown spreadsheet or Word document. This needs to stop and it needs to stop fast! Shareholder value is being degraded by these poor practices and we even see these methods being promulgated by a number of the most popular best practice frameworks. If you're interested in joining us in calling for the "reengineering of risk assessment", then you need to make sure that you are fully conversant with the following information. Print it out and tape it up in a highly visible location.

 

Called “Likert (“Lick-ert”) Scales”; A typical question using a Likert Scale might pose a statement and ask the respondent whether they strongly agree, agree, etc. The responses elicited may be coded e.g. 1-2-3-4-5, but this remains just a coding.  It makes no sense to add a response of agree (coded as 2) to a response of undecided (coded as 3) to get a ‘mean’ response of 2.5 (what would it mean?). So how can you analyze data from a Likert scale?

 

The data collected are ordinal: they have an inherent order or sequence, but one cannot assume that the respondent means that the difference between agreeing and strongly agreeing is the same as between agreeing and being undecided.

 

The wide majority of Risk and CSA assessments are based on inaccurate measurement scales and weak math foundations where counting rather than true measurement is used.

Data collected are ordinal; prevents further mathematical operations to analyze the data
No “True Zero”
Not an Accurate Way to Measure!
Summarize using a median or a mode (not a mean); the mode is probably the most suitable for easy interpretation.
Express variability in terms of the range or inter quartile range (not the standard deviation).
  

 

We can display the distribution of observations in a dotplot or a barchart (but not as a histogram, because the data is not continuous.