What to do when ICS-CERT and NIST produce contradictory vulnerability analyses

Vulnerability disclosure organizations are considered to be the most important and reliable source of actionable information for vulnerability and risk assessment, including exposure data, exploit difficulty analysis and device vendor information. Due to the high cost and tremendous risks involved in implementing corrective measures (or not), vulnerability analysis inconsistencies are increasingly becoming a problem for ICS organizations’ CSOs.

Here at Radiflow, we argue that the current vulnerability scoring system is not tuned to ICS, as it incorrectly and inconsistently weighs up different impacts and misses some factors. Even NIST and ICS-CERT, the two main vulnerability disclosure organizations, are not always aligned.

The players

Two of the major vulnerability disclosure organizations are NIST and ICS-CERT.

• NIST holds and maintains the US national vulnerability database.
• ICS-CERT is the National Cybersecurity and Communications Integration Center (NCCIC) Industrial Control Center. The NCCIC ICS’s mission is to reduce risk in all critical infrastructure sectors by collaborating with players from law enforcement, intelligence, government, control systems owners, ICS operators, and device vendors. NCCIC also collaborates with international and private sector Computer Emergency Response Teams (CERTs) to share control systems-related security incidents and mitigation measures.

While both organizations continuously provide new vulnerability feeds and the analysis, their analyses are not always identical, and in some cases even contradictory. This makes it difficult for critical infrastructure operators to properly estimate the potential impact of vulnerabilities.

Between 2017 and 2018 Radiflow has detected about twenty such inconsistencies, which were included in the advisories released by ICS-CERT. The inconsistencies were not only in each vulnerability’s score but also on its detailed impact.

Anatomy of a vulnerability analysis discrepancy

For example, in ICSA-18-009-01, ICS-CERT released CVE-2017-16740 regarding Allen-Bradley MicroLogix 1400 Controllers, which stated that “Successful exploitation of this vulnerability could cause the [attacked] device to become unresponsive to Modbus TCP communications and affect the availability of the device.”

ICS-CERT gave this vulnerability an 8.6 (high severity) score, while NIST gave this CVE a score of 10 (critical).

More interesting and confusing are the scoring notes for the vulnerability, which detail the potential impact of the vulnerability (among other parts of the analysis). According to ICS-CERT, the vulnerability would have zero impact on confidentiality and integrity; NIST’s predicts a HIGH impact on the confidentiality and integrity. The detailed reports can be seen here:

• https://ics-cert.us-cert.gov/advisories/ICSA-18-009-01
• https://nvd.nist.gov/vuln/detail/CVE-2017-16740

What causes the inconsistencies between NIST and ICS-CERT?

To answer the question, Radiflow contacted ICS-CERT and NIST. ICS-CERT explained that their scoring details are formulated in coordination with the vendor and the researcher; once the advisory analysis is complete, they submit the information to NIST. If NIST disagrees with ICS-CERT’s CVSS scores, they develop and publish their own scores and analysis. NIST explained that they perform their scoring based on the vulnerability description, and in the cases we pointed out to them the description was aligned with their scoring.

Are you a “NIST” or an “ICS-CERT”? Take the Radiflow Survey to find out.

In light of the analysis inconsistencies between NIST and ICS-CERT, it’s clear that the fundamental requirement to adapt an accepted vulnerability score is not enough, since it still allows users the freedom to interpret and implement the analysis in different ways (e.g. decide whether the impact of a vulnerability is low or high.)

More examples for scenarios where ICS-CERT and NIST are not aligned can be found in the under-five-minute Radiflow Vulnerability Analysis Survey, which allows the participants to validate whether their perspective is more ‘ICS-CERT’ or more ‘NIST’. In addition, more cases where the current scoring system is misleading can be found in the survey.

In the survey, you’ll be asked to help in rating a few sample scenarios that represent the various currently-overlooked aspects that can impact holistic scoring.

The results will hopefully help to create a more comprehensive approach to scoring and most importantly, you’ll be able to compare your stance with those of your peers!

However, a U.S. presidential advisory committee convened in the wake of Edward Snowden’s leaks recommends against government stockpiling. The committee was charged with developing a set of recommendations for how to strike a balance between protecting national security interests, advancing the administration’s foreign policy agenda, and protecting citizens’ privacy and civil liberties.

The committee’s report includes 46 recommendations, including one on the topic of zero-day disclosure: “US policy should generally move to ensure that zero-days are quickly blocked, so that the underlying vulnerabilities are patched on US Government and other networks.” The report continues, “In rare instances, US policy may briefly authorize using a zero-day for high priority intelligence collection, following senior, interagency review involving all appropriate departments.”

It is clear that the panel’s recommendation favors disclosure. In response, the government stated that “there is a [zero-day review] process, there is rigor in that process, and the bias is very heavily tilted toward disclosure.”

That’s the view of Joe Nye, the veteran national security scholar, who makes the argument that “…if the United States unilaterally adopted a norm of responsible disclosure of zero-days to companies and the public after a limited period, it would destroy their value as weapons — simultaneously disarming ourselves, other countries, and criminals without ever having to negotiate a treaty or worry about verification. Other states might follow suit. In some aspect, cyber arms control could turn out to be easier than nuclear arms control.”

Still, the 2013 presidential advisory committee’s report referenced above counters RAND’s conclusion: “In almost all instances, for widely used code, it is in the national interest to eliminate software vulnerabilities rather than to use them for US intelligence collection. Eliminating the vulnerabilities — ‘patching’ them — strengthens the security of US Government, critical infrastructure, and other computer systems.”

It’s a fascinating debate, but don’t wait for a resolution before hardening your organization’s security posture to thwart zero-day based attacks. The chances are high that your organization may already have undetected malware leveraging zero-days vulnerabilities, since even next-generation antivirus solutions have a hard time detecting threats that are very different from what they have seen before. After all, a 6.9 year life span gives a zero-day lots of time to cause significant damage.