The concept of cyber risk rating (CRR) has a lot of promise. It represents an attempt to replicate the approach widely used in the credit risk analysis to the cybersecurity domain.
In theory, a CRR would provide a single, scaled, and regularly updated metric indicating the approximate level of cyber risk of the rated entity. A viable CRR would greatly help in addressing many challenging problems: rapid evaluation of cybersecurity posture, risk management of a large portfolio of third-parties, comparative cyber risk analysis, or continuous risk monitoring.
It shouldn’t come as surprised that the CRR market offering has been rapidly developing. Though conceptually the idea of CRR is very attractive, its actual implementation suffers from several limitations and methodological flaws. One of the aspects which attracted significant criticism is the fact that the CRR calculation is based almost entirely on data which is available in the public domain and can be externally collected (e.g. website security, email security, leaked credentials, TSL/SSL certificates).
In other words, the CRRs are not based on what is arguably the most relevant - the private data which resides in the internal systems of the rated entity. Some critics compared it to evaluating the fire safety of a building by looking at its photo.
This criticism is well-deserved. The methodology used in creation of CRR relies by and large on the publicly available data which can be collected without actively engaging the rated entity. Despite this obvious shortcoming, the CRR may nevertheless be used in a sensible fashion to provide useful insights for risk management.
First, one can treat a CRR exactly for what it is – a synthesis of publicly available data related to cyber risk of the rated entity. This is already a lot. If the rating provider does a good job in collecting good quality data available in the public domain, the result should provide a good representation of what can be learned about the rated entity “from the outside”. Such input could offer an economic alternative to proprietary analysis of external data and offer an excellent starting point for further analysis.
Second, the CRR can offer a “sanity check” when used together with other instruments of cyber risk evaluation. The case would be particularly valid if a significant divergence emerges between the level of cyber risk indicated by the CRR and the indication provided by the other instrument. In this respect the CRR could be particularly useful as quick “second opinion” on the self-assessment questionnaires.
Third, the CRR might be used as a basic filtering mechanism to eliminate the most “serious offenders”. Even though the CRR might not represent a full picture of the cyber risk a very low rating might provide sufficient proof of poor cybersecurity practice to for example exclude the company from establishing a business relationship with.
Fourth, tracking changes of the CRR over time could provide valuable insights. This could provide particularly interesting information of changes in CRR appear to be correlated with major corporate events. For instance, a significant and lasting improvement of the CRR following a change of the management could indicate improvement of cybersecurity practice. A decline after a major acquisition is made could indicate that the integration increased the level of cyber risk.
Fifth, the CRRs could provide insights by being used as an instrument facilitating the comparative analysis. The ratings could be used for quick identification of outliers, quick-and-dirty tool for prioritization of further analytical efforts, or an instrument to validate results produced using different approaches. In this respect the CRRs would serve more as an auxiliary instrument.
The CRRs are here to stay. We’re likely only in the initial phases of their development. The idea of having a scaled, regularly updated, and independently calculated representation of cyber risk level is extremely attractive, despite all the shortcomings.
There is no argument that CRRs are no substitute for an in-depth analysis based on access to internal systems and private data. It’s unlikely that the CRRs will, at least in the foreseeable future, provide a truly comprehensive and accurate representation of cyber risk. At the same time, it’s hard to expect that such resource-intensive approach will be justified in most cases from cost-benefit perspective.
It’s clear that CRR do not offer a silver bullet solution for estimating the level of cyber risk. They can still be very helpful in specific situations and when used in complementary fashion with other instruments. The key question is to understand the limitations of CRRs are find applications in which they can be effectively used in cost efficient fashion.
Adam Klus is a PhD student at the University of Eastern Finland, where he is researching the cyber dimension of economic warfare. Adam is interested in the strategic aspects of cybersecurity and cyber-enabled adversarial threats in the economic domain. He also holds CompTIA Sec+ and CompTIA CySA+ certificates. His previous professional experience includes several years in the investment industry.