How Bad Bots Hurt Your Website and What You Can Do To Protect It

Dr. Eduardo Rocha Senior Solutions Engineer & Security Analyst @ GlobalDots
5 Min read

In the last few years, the term “bot” has become widey used. Everyone is talking about bots. We hear it used in politics, on social media, and when discussing website traffic.

Bad bots are an all-too-common and growing problem. But what specific damage are they doing on your website? And how does it impact your business?

What is a Bad Bot?

Bad bots scrape data from sites without permission in order to reuse it (e.g., pricing, inventory levels) and gain a competitive edge. The truly nefarious ones undertake criminal activities, such as fraud and outright theft.

The Open Web Application Security Project (OWASP) provides a list of the different bad bot types in its Automated Threat Handbook.

Left unaddressed, bad bots cause very real business problems that could harm the success — or even the continuance — of your organization. Examining the problems doesn’t require deep knowledge of the technology behind attacks or the techniques used to prevent them. Instead it requires a solid understanding of your business.

Here are some quick bad bots facts:

  • Every business with an online presence is regularly bombarded by bad bots on its website, APIs, or mobile apps.
  • Unchecked bad bots cost businesses money every day. Different from the problem of data breaches, which are somewhat rare, automation abuse happens 24 × 7 × 365 because bad bots never sleep.
  • Bad bots are on your website for a purpose. Understanding what that purpose is helps you address the problem.

In 2017, 42.2% of all internet traffic wasn’t human, and there were significant year-over-year increases in both bad bot (+9.5%) and good bot (+8.8%) traffic.

bad bot growth

Bad Bot sophistication levels

GlobalDots created the following industry standard system that classifies the sophistication level of the following four bad bot types:

  • Simple — Connecting from a single, ISP-assigned IP address, this type connects to sites using automated scripts, not browsers, and doesn’t self-report (masquerade) as being a browser.
  • Moderate — Being more complex, this type uses “headless browser” software that simulates browser technology—including the ability to execute JavaScript.
  • Sophisticated — Producing mouse movements and clicks that fool even sophisticated detection methods, these bad bots mimic human behavior and are the most evasive. They use browser automation software, or malware installed within real browsers, to connect to sites.
  • Advanced Persistent Bots (APBS) — APBs combine moderate and sophisticated technologies and methods to evade detection while
    maintaining persistency on targeted sites. They tend to cycle through random IP addresses, enter through anonymous proxies and peer-to-peer networks, and are able to change their user agents.

You can read more about Bad Bots sophistication levels, their impact on various industries and a lot more in our Bad Bot Report 2018.

How Bad Bots hurt your website

Bots are tailored to target very specific elements of a website, but can affect more than just stolen content, spammed forms, or account logins. The Open Web Application Security Project (OWASP) published the Automated Threats Handbook for Web Applications, which profiles the Top 20 automated threats and categorizes each threat as one of four types:

  • Account Credentials – Includes account aggregation, account creation, credential cracking, and credential stuffing.
  • Payment Cardholder Data – Includes carding, card cracking, and cashing out.
  • Vulnerability Identification – Includes footprinting, vulnerability scanning, and fingerprinting.
  • Other – The catch-all category. Includes, ad fraud, CAPTCHA bypass, denial of service, expediting, scalping, scraping, skewing, sniping, spamming, and token cracking.
bad bots

How to protect your website from bad bots

Every website is targeted for different reasons, so there’s no one-size-fits-all solution to the bot problem. There are, hoverer, certain steps you can take to make sure you’re protected from bad bots.

On its surface, a visit from a human and a bot may appear nearly identical. Bots can appear as normal users, with an IP address, browser and header data, and other seemingly identifiable information. But dig a bit deeper by collecting and reviewing in-depth analytics and other request data and you’ll be able to find the holes in the bots’ disguises.

Now that you’ve separated human traffic from bot traffic, you can dig a bit deeper to see which bots are good and which are bad. Good bots include search engine crawlers (Google, Bingbot, Yahoo Slurp, Baidu, and more) and social media crawlers (Facebook, LinkedIn, Twitter, and Google+). Generally, you want to allow these good bots access to your site, since they help humans find and access your site. Bad bots include any bots that are engineered for malicious use. These bots attempt scraping, brute force attacks, competitive data mining causing brownouts, account hijacking, and more.

Knowing the difference between the bots visiting your site lets you take action on bad bots and allow access to good bots.

Here’s overview of the things you must do before you can start protecting yourself from malicious bots.

Understand your vulnerabilities

Data is collected through every interaction and transaction online. Every business with a web presence is collecting sensitive data that might be of value to bad actors.

Businesses must continually evaluate and evolve their security measures to stay ahead of hackers. It’s crucial to understand the nature of the threat and have a clear plan of action to patch and protect their vulnerabilities online.

Tell the difference between bot protection myths and facts

In order to make informed and actionable decisions about the security in your business, it’s important to have the right information.

For example, you may have heard that all bots are bad. That’s not the case — there are plenty of bots that serve perfectly legitimate, even helpful functions.

It’s also often assumed that all bot attacks involve hacking. In fact, many bot attacks are simply probing for vulnerabilities that a hacker can exploit later.

Detect, categorize and control

Detecting bot traffic is the first step. Once bot traffic has been identified, the next step is to categorize the type of traffic. If it’s known bot traffic – like that of search engine bots – it should be allowed to pass. But known malicious bots, or bots of unknown intent, shouldn’t be allowed to pass.

Finally, the malicious bot traffic must be controlled. The type of bot mitigation required will depend on the type of attack. For a denial of service attack, your security software should simply divert the traffic. If the bot is looking for vulnerabilities or trying to commit fraud like shopping cart stuffing, the software should both deny access and return a false “page not found” 404 to the bot, to stave off future attacks from the same source.


Bad Bots are an increasing threat to enterprises worldwide. They’re often difficult to detect, and the damage they do can cripple a business. If you have any questions about how we can help you protect your website and business from bad bots, contact us today to help you out with your performance and security needs.

Latest Articles

AWS FinOps Best Practices

Amazon Web Services (AWS) is the most established cloud vendor on the planet – its vast array of services increases every year, with a staggering 200 new offerings released in 2020 alone. These services offer businesses the ability to adjust resources based on demand – a cost-effective pay-as-you-go model and a global infrastructure for widespread […]

Nesh (Steven Puddephatt) Senior Solutions Engineer @ GlobalDots
30th November, 2023
Reduce Cloud Costs by Up to 80% with Automated K8s

In the dynamic landscape of cloud computing, Kubernetes (K8s) has emerged as a cornerstone for container orchestration. There is no doubt that K8s clusters are effective, yet companies continue to grapple with the complexities of managing K8s clusters, especially in production. GlobalDots, recognizing this challenge, has curated a pioneering solution: a platform that transforms the […]

30th November, 2023
FinOps Tools: Key Factors to Consider

For organizations already well-established in the cloud, vendors such as AWS and Azure often bite large chunks out of annual revenue. Monthly cloud bills accumulate quickly  – the 5 biggest cloud vendors rake in a combined total revenue of over $80 billion a year, with Microsoft and Amazon claiming the lion’s share of over $20 […]

Nesh (Steven Puddephatt) Senior Solutions Engineer @ GlobalDots
22nd November, 2023
AWS Innovations Decoded: GlobalDots’ Top 20 Picks

Join AWS experts from GlobalDots as they decode the top 20 cloud innovations you need to know in a 2 part Webinar. Gain insider insights on leveraging these transformative technologies to boost performance, tighten security, and reduce costs. Discover real-world applications to apply these advancements to your business. Reserve your spot now! 🚀 Stay Ahead: Learn […]

31st October, 2023

Unlock Your Cloud Potential

Schedule a call with our experts. Discover new technology and get recommendations to improve your performance.

Unlock Your Cloud Potential