“Just because you’ve counted all the trees doesn’t mean you’ve seen the forest.” – Anonymous
Cyber Arms Race in Cash
The UK cyber security market is valued at $5 billion, according to the US-based website export.gov, and is widely regarded as the largest cyber security market in Europe.
At the same time, it is estimated that cybercrime costs the UK up to $30 billion annually, repeating an arms-race paradigm seen throughout the world.
To wit, whereas global information security spending is estimated at $124 billion, cybercrime losses are expected to reach $6 trillion by 2021.
Does the infosec community stand even a remote chance of turning this situation around? The response by many organizations: Pile security products higher and deeper.
But if your organization wants objective, vendor-agnostic evidence of cyber resilience, that proof lies in the *testing* pudding. Put simply, how can you tell whether your security investments are effective, if you don’t challenge them; if you don’t put them to the test?
Pros and Cons of Established Security Testing Methods
Security testing methods have evolved over the past two decades, and while all have their advantages, including maintaining regulatory compliance, they also have their limitations.
The following is a summary of the advantages and drawbacks of widely used security effectiveness testing methods and tools.
As put by Gartner, “Penetration testing helps answer the question ‘can they get in?’” and the follow-up assessment reports enable security teams to know if they are vulnerable to the latest hacking techniques.
However, in between pentesting engagements, usually carried out on a quarterly or monthly basis at best—security teams are left in the dark as to how well their security is performing. The effectiveness of corrective steps taken after a pentesting engagement are difficult to assess, requiring manual testing and some white hat knowhow.
The impact of changes made to an enterprise environment are usually not known until the next penetration test, including the impact of daily happenings such as the rollout of new technology, policy updates and configuration variations.
CBEST and Intelligence-led Penetration Testing
The Bank of England’s CBEST regulation steps up pentesting by requiring UK financial organizations to perform pentests that are based on intelligence that is relevant to them. According to the CBEST Implementation Guide, this includes information on “Targeting: potential attack surfaces” across the organization’s infrastructure, as well as “Threat Intelligence: relevant threat actors.”
Potential attack surfaces may include the consumer-facing applications, email gateways and secure-browsing web gateways deployed in an organization, as well as other potential attack vectors, such as the ability to move laterally from one network segment to another or exfiltrate records from a database.
Relevant threat actors, on the other hand, may include specific state-sponsored APT groups known for their attempts to circumvent international sanctions through highly-targeted fraudulent ATM and SWIFT transactions, or even extortion through ransomware attacks.
While this approach notches up security effectiveness testing, it still leaves organizations with infrequent assessments, that may be repeated only on a semi-annual or annual basis.
Vulnerability scans are useful for uncovering the presence of known vulnerabilities (CVEs) on machines and in software. But they fail to check the potential impact of their exploitation on the organization, nor do they check for controls implemented to prevent lateral movement, used by threat actors to advance in the network, and traverse network segments, towards lucrative targets. Essentially, they lack offensive-based threat modelling.
Testing a broader scope of attack vectors and security controls than traditional pen tests, red teaming has emerged to simulate sophisticated multi-step attacks. Red team exercises more closely resemble cyber threats that spread and exploit vulnerabilities across the kill chain and are effective at mimicking a multi-vector advanced persistent threat.
They do require a certain level of expertise and are usually sourced either from an external consulting firm or performed internally by in-house (usually blue team) professionals. Still, as with pentesting, their resource-intensive nature doesn’t allow them to be used as a method to test security posture or security effectiveness on a daily, weekly or even monthly basis.
Open Source Tools
Manual open source tools have emerged to facilitate the work of red teams and pen testers. These include Endgame Red Team Automation, MITRE Caldera, Red Canary Atomic Red Team and Uber Metta.
The fact that they are free and generate platform specific attacks is a big plus; however, they do require advanced technical skills to use, and may require modification to test multiple attack techniques, simulating the logical flow of a full kill chain attack. Also, they usually lack remediation suggestions.
Seeing the Forest, Not Just the Trees
At best, the tools above provide only a snapshot in time of an organization’s security posture. Their infrequent nature limits the value they can provide in determining an organization’s real time security posture, to answer the question, “Is my organization secure right now?” Nor can they answer the question, “How effective are my security controls?” as they reflect tests performed on an environment several weeks or months ago.
Gaining Back the Advantage of Time
Instead of relying on periodic exercises and engagements and always playing catch up on the very latest threats currently in the wild, security teams and executive management need to reconsider how they determine their organization’s real time security posture and security effectiveness.
With automated attack simulation tools now at their disposal, security teams can implement a repeatable, continuous testing process that tests security effectiveness across their attack surface and enables them to remediate gaps throughout their infrastructure—all while requiring minimal skill.
To complete the Gartner quote above, whereas “Penetration testing helps answer the question ‘can they get in?’; BAS tools answer the question ‘does my security work?’”