How a European Electronics Retailer Ended the Bot Menace on its Portal
It’s May 2020 and a large European electronic goods retailer realizes its bot problem is going from bad to worse.
In a single week, its online store is hit with eight million bot visits to systematically scrape pricing and product information without authorization — not to mention 53,000 customer account takeover attempts, 136,000 denial of inventory attacks, and 234,000 attempts to conduct affiliate link fraud. Even without these malicious activities, bot traffic is tying up valuable resources in ways guaranteed to hurt the bottom line.
For a firm operating three hundred retail stores that attract ten million shoppers a year to its online site, this level of bot traffic is unsustainable. It’s risking not only lost sales and rising costs but also damage to a brand image built with huge effort over many years.
Bad Bots Affect Retailers In A Range of Sinister Ways
- Account takeover (ATO) fraud – Criminals breaking into a customer’s account to carry out a range of frauds, including identity theft, stealing loyalty points, or making fraudulent transactions. This often happens because of credential stuffing, which exploits the fact that many customers reuse the same username and password across multiple accounts. When those reused credentials are leaked or breached and then sold on the dark web, users’ accounts are vulnerable to ATO.
- Denial of inventory – Filling baskets with products without paying for them. This ties up inventory, artificially reducing product availability while damaging the retailer’s sales.
- Affiliate fraud – Earning rewards from a site by generating large amounts of junk traffic. Retailers end up paying for nothing.
- Wasting resources – Dealing with Bot traffic makes it difficult for marketing teams to get accurate KPI data to plan for growth.
- Carding attacks – Using retail websites to ‘test’ stolen credit and debit card data. Retailers are often left with the cost of reimbursement.
- Scalping – Using automated programs to grab desirable inventory before real customers can. The goods are then resold at inflated prices on the secondary market.
- Web scraping – Stealing price data for rivals. A variation on this is scraper bots that steal other site content, including reviews and product descriptions.
Oddly, the last two bot threats are not illegal in many countries. However, this doesn’t mean that the retailer should tolerate them. For example, scalping can hurt the reputation of a retailer with genuine customers.
[You may also like: Bot Manager vs. WAF: Why You Actually Need Both]
Bots can be Hard To See Until Trouble Strikes
Bad bots plague today’s retailers all year round but the extent of the problem often becomes even more magnified at peak times such as holiday seasons when traffic naturally spikes.
For example, during Thanksgiving 2021, a group of six prominent e-commerce websites protected by Radware Bot Manager were flooded with bot traffic. Traffic volume ranged from more than 4 million bots per day to well over 9 million bots per day during the week prior to the holiday. To a retailer without bot protection for its website or apps, these numbers represent potential attacks that can wreak havoc on the user experience. Some retailers may experience website slowdowns that frustrate shoppers; others may see a drain on the inventory of highly sought-after products that are snatched up by scalpers for resale rather than loyal customers; still others will field complaints on their support line about cashed-out gift cards and loyalty points. Not only does the customer experience suffer, but ultimately brands are damaged, and revenue is lost.
Figure 1: Aggregated bot hits on six e-commerce websites in November 2021
[You may also like: Why Organizations are Failing to Manage Rising Bot Attacks]
Trialing Radware Bot Manager
So how does the bot battle end for the large retailer mentioned at the beginning of this blog?
The retailer knew something was wrong, but what? How big was this problem? In search of a solution, the company made the decision to trial Radware’s Web Application Firewall (WAF) and application protection solution.
Radware analysts discovered that more than 50% of all visitors to the retail site were in fact bots, an unsustainable situation that if left unchecked could lead to the adverse consequences already discussed.
The retailer then initiated a proof of concept (POC) trial with Radware Bot Manager, using our NGINX connector to integrate with its website. After analyzing visitor traffic for a week, Bot Manager went into ‘Active Mode’ and began to block over 2 million bad bots every day thereafter. Suspected bots were shown a CAPTCHA to solve to enter the website. Overall, only 0.25% of these challenges were solved, which meant that almost all bots were blocked, and genuine visitors were not shown a CAPTCHA while visiting the website.
CAPTCHA challenges, of course, are only an initial step in Radware’s overall bot detection process that is powered by our patented intent-based deep behavior analysis technology. This technology offers unmatched accuracy in detecting sophisticated bots that emulate human behavior as they traverse a website or application.
The Radware Advantage
Soon after our successful engagement with this retailer, we learned that earlier they had also conducted a POC with a global provider of CDN and bot mitigation services, which fell short in two major areas when compared to the results from Radware’s POC:
- The competing solution detected approximately 20% fewer bad bots than Radware Bot Manager. Considering Radware detected and blocked 2 million bots daily, this would theoretically mean 400,000 bad bots carried out attacks every day.
- The CDN-bundled bot mitigation solution mentioned earlier required all website traffic to be rerouted through its servers for bot detection, which was an unacceptable proposition for this retailer (and any organization serious about data privacy and protection).
In the end the result was clear – Radware’s bot protection had proved itself more effective under real world conditions.
Click here to calculate how much bad bots are costing your business.
