The traditional method for calculating standard RoI is that it equals the sum of the gain minus the cost, divided by the cost. The higher the resulting value, the greater the RoI. The difficulty in calculating a return on security investment, however, is that security tends not to increase profits (gain), but to decrease loss – meaning that the amount of loss avoided rather than the amount of gain achieved is the important element.
The “monetary value of risk can be estimated by a quantitative risk assessment,” says ENISA. This involves ‘single loss assessment’ (SLE), ‘annual rate of occurrence’ (ARO) which together provide the ‘annual loss expectancy’ (ALE). Following the standard RoI approach, RoSI can be calculated by the sum of the loss reduction minus the cost of the solution, divided by the cost of the solution. In short, a high result is better for RoI, and a low result is better for RoSI
This is where it gets difficult: how do you measure the ‘loss reduction’? Technically, it is the ALE (without the security solution) minus the modified ALE (after the security solution) – and the latter is difficult to quantify. To a large extent it is based on guesswork and surveys. “These approximations are often biased by our perception of the risk and the ROSI calculation can be easily manipulated (See ‘The data imperative’) to serve the user's interest or to justify a decision rather than enlighten it,” warns ENISA.
The data imperative was written by Bruce Schneier in 2008. “Depending on how you answer those two questions," he concluded; " and any answer is really just a guess -- you can justify spending anywhere from $10 to $100,000 annually to mitigate that risk.”
The ENISA paper is primarily addressed at how CERTs can produce RoSI analyses to see where they should concentrate their spending. “ROSI is a complex topic and this first attempt to introduce this topic has to be further developed to address remaining issues on CERTs and ROSI calculation,” concludes the paper. The CERT difficulty is complicated by protecting external assets and being reliant on third party loss statistics, but the paper provides valuable insights into a somewhat intractable problem for all businesses.
But one area in which both Schneier and ENISA agree is the dubious value of ‘loss’ defined by third parties. “This doesn't mean that ALE is useless,” says Schneier, “but it does mean you should... mistrust any analyses that come from people with an agenda.”
“It's often a better practice to extrapolate from the organization’s historical data on incidents than to rely on the study of a vendor,” suggests ENISA.