Rate Limiting: A Vital Tool for Modern Cyber Security
Rate Limiting: A Vital Tool for Modern Cyber Security – When evaluating web security solutions, people tend to be familiar with WAF (Web Application Firewall) and DDoS (Distributed Denial of Service) protection. Bot management also gets a lot of attention. However, there’s another security technology which tends to be less prominent, but is just as important.
Rate limiting is the restriction of traffic based on the frequency of requests arriving from a specific traffic source. To enforce rate-limit restrictions, a security solution must monitor the amount and timing of traffic coming from each requestor. When a requestor exceeds the allowable rate limit, all traffic coming from that source is blocked for a length of time.
For example, many web applications include a login form for users to enter their account credentials. Most users will complete the process with only one login attempt, while a few will experience problems (such as forgetting or mistyping their passwords) and need to try several times before logging in successfully.
However, when a single “user” fails the process dozens of times within a short period, this is probably not a legitimate user; instead, it’s most likely an attacker trying to gain access to a user’s account. A security solution with a robust rate limiting module will detect the excessive login attempts (usually after the first few failures) and will block that “user” from further access.
In this article, we’ll discuss the importance of this technology. As we will see, it is a vital part of defending against certain types of attacks. Therefore, when considering the adoption of a security solution, it’s very important to evaluate the solution’s rate limiting capabilities. Then at the end of this article, we’ll discuss the key rate-limiting features to look for in a solution, to ensure that it will provide robust protection.
The Usefulness of Rate Limiting
Rate limiting is a key technique for defending against multiple forms of web-based attack. Here are some of the more important use cases.
DDoS Attacks: A volumetric DDoS is perhaps the most dramatic form of cyberattack. Modern DDoS assaults can include massive volumes of incoming traffic (the current record is 2.5 Tbps). In these events, attackers attempt to overwhelm the targeted system and make it unavailable to legitimate users.
This is perhaps the most direct and obvious use for rate limiting. When a specific traffic source submits too many requests in a given time, then the web security solution should block all traffic coming from it.
Credential Stuffing: Periodically, there are reports in the media about a large company that was recently breached, and the incident usually includes a compromise of a database containing user information. The user credentials in that database are often abused very quickly afterward.
Bots are sent to prominent websites across the web, where they “stuff” the stolen credentials into login forms. For each credential set that works, the bots can take over the account from the legitimate user.
Frequently, the bots experience a high rate of success with this attack. (Unfortunately, many people use a single username/password combination across different sites.)
If left unchecked, a credential-stuffing bot might submit dozens, hundreds, or even thousands of credential sets into a single login form. Rate limiting is an effective way to detect and block it early during this process, thus preventing its ATO (account takeover) attack from succeeding.
Brute force attacks: These are similar to credential stuffing attacks, except that the bots don’t have lists of valid credentials to use. Instead, the bots try to guess login credentials by systematically creating and submitting different variations of characters until they discover credential sets that work.
A well-secured web application will have requirements (such as password length and composition) that reduce the success rate for brute-force attacks. Nevertheless, a large attack can consume a lot of network resources, and so it’s important to block them. Rate limiting works well for this.
Site scraping and data theft: Many web applications contain valuable data and content; this makes them attractive targets for illicit scrapers. Frequent victims include aggregators which sell access to content, ecommerce stores whose competitors try to collect their pricing data, insurance companies whose competitors want to undercut their rate quotes, and others.
By their nature, scraper bots attempt to access and copy as much data from the targeted application as they can. Rate limiting can detect this activity and prevent the scraping.
Inventory Denial (a.k.a. Inventory Hoarding). In this type of attack, threat actors send inventory-denial bots to targeted web applications. The bots begin transactions (e.g., reserving seats on airline flights, adding products to ecommerce shopping carts, and so on), without ever completing them. This reduces available inventory for actual customers. Rate limiting can block these bots.
The Importance of Rate Limiting
As the discussion above shows, robust rate-limiting features are vital in defending against many types of attacks. In fact, for many of these attacks, rate limiting is the only means of detection.
There are many attacks (e.g., protocol exploits, SQL injection attempts, and numerous others) which can be detected within the requests themselves. Note however that most of the attacks described above use requests that seem innocuous.
For example, a single failed login attempt appears to be exactly that—a failed login attempt. It is only by analyzing the larger context of the requestor’s activity that the malicious intent will become apparent.
How to Evaluate Rate-Limiting Capabilities
When considering a security solution, its rate-limiting features must be carefully evaluated. Here are some things to look for.
Requestor Identification and Tracking: Sophisticated attackers will attempt to evade rate limits by rotating IP addresses, spoofing their geolocation, and other tactics. The best security solutions can identify and track individual requestors, correctly counting their requests and enforcing rate limits despite these efforts.
Configurable Analysis: Most security solutions allow admins to specify the maximum number of allowable requests and the period of time within which that maximum is allowed. The best solutions also allow admins to sort and count requests based on different parameters, such as admin-specified headers, arguments, and cookies.
Automated Responses: Some solutions are designed for monitoring and alerting only. The best solutions will not only fire alerts, they can also auto-ban the offending requestor.
Layered Responses: Some solutions can auto-ban a requestor for a configured time, but after the time expires, the requestor can resume their attacks. This can create a cycle of ban/resume/ban/resume that goes on continually; in effect, these solutions cannot prevent attacks, they can merely slow them down. The best solutions will notice when a traffic source repeatedly violates a rate limit, and can impose much longer bans as a result.
Automated Recommendations: Most rate-limiting solutions will accept and enforce whatever limits are configured by the admin. However, it is vital that the limits are set correctly.
If the limits are set too low, this will create false positive alarms. (Some legitimate users will be excluded from the protected application.) Conversely, if the limits are set too high, the consequences can be even worse; it will create false negative alarms. (Instead of being excluded from the network, some attackers will be allowed access.)
The best solutions will help to avoid this. They can analyze traffic patterns over time and recommend correct limits based on historical behavioral data for each protected application. They will present different options for admins via graphical displays of historical data, explaining how the different limits would have affected incoming traffic and the implications of each choice.
Rate Limiting: A Vital Tool for Modern Cyber Security – This has been a brief overview of rate limiting, which has even more security implications than those described here. When evaluating a web security solution, it’s important to carefully consider its capabilities in this area.
Rate Limiting: A Vital Tool for Modern Cyber Security