← Guides

AI bots are hammering stores: hundreds of thousands of requests

Updated 2026-08-03

How much AI bots really cost you

Numbers from our AI Traffic Index #1 (period 13 Jul – 1 Aug 2026, a panel of three stores: two in Moldova, one in Georgia):

• one AI vendor alone: 606,017 requests to a single store — about 34,000 a day;
• only 8 of those requests reached product pages — the rest went to service pages and category filters;
• at the panel's most machine-readable store, bots read 73,715 product pages.

Over 600,000 requests from one vendor to one small store in about three weeks sounds alarming. Before you start cutting bots off, let's see what it actually means for your server.

Is that a lot for your server?

Convert it to requests per second. 34,000 hits a day from the top vendor averages under 0.4 req/s. That's nothing for any host — on average. The catch is bursts: crawlers don't spread load evenly, they come in waves — dozens of requests in a second, then silence.

What happens next depends on what your server returns:

• Static and cached. If product pages and images come from cache or a CDN, even hundreds of thousands of hits barely touch the origin.
• Dynamic. If every bot visit hits PHP and the database (cold cache, personalization, heavy filters), the same traffic starts to bite.

One number stands out: of those 606,017 requests, only 8 reached product pages — the rest hit service pages and category filters. If those pages are rendered fresh on every visit, you pay for every one of them.

When to actually worry

Watch the symptoms, not the raw hit count:

• origin response time climbs, especially on product pages;
• 5xx errors and timeouts show up during off-peak, low-human hours;
• CPU and database max out without any rise in human traffic;
• product images return 403/406 to bots.

That last one is sneaky. It isn't the most common problem in our rating of 63 MD/RO stores — that is missing or incomplete JSON-LD Product — but it hurts twice: a bot gets a 403 on an image, treats it as a failure, and comes back to retry. You lose product visibility in AI answers and gain extra load from retries at the same time. Fix the images and part of the "aggression" disappears on its own.

How to see your own load

1. Open your access logs and filter by known bot User-Agents: GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended, Bytespider, CCBot.
2. Measure the bot share of total requests and the distribution by hour — that's where the bursts show.
3. Verify by IP, not just by name. A User-Agent is trivial to fake; plenty of random scrapers hide behind GPTBot. Botmetria checks bots against the vendors' published IP ranges (or reverse DNS) and reports impostors separately.

No time to grep logs? Connect Botmetria analytics (the WooCommerce plugin, the Cloudflare Worker or a server collector) to see AI-bot hits by vendor and by page. The free Botmetria audit doesn't measure your load: its "Hygiene" category only checks how your home page answers GPTBot and how fast. The AI Traffic Index gives you a market baseline.

How to throttle the aggressive ones gently

The goal is to shed load without losing the bots that put you into AI answers. From safest to strictest:

1. Cache. The cheapest win. Serve pages and images via a CDN and full-page cache. Then even tens of thousands of bot hits a day cost the origin almost nothing.
2. Rate-limit. Cap requests per IP/subnet — say N per second with a graceful 429 slowdown rather than a hard block. Well-behaved bots respect 429 and back off.
3. Firewall by IP range. If one vendor is genuinely too aggressive, block it surgically by its verified ranges, not by User-Agent (or you'll catch fakes and miss the real ones).

One safety rule above all: never block Googlebot or Bingbot. Those are classic search engines; lose them and your regular SEO dies. Botmetria's firewall has that fuse built in — it filters AI crawlers while leaving search engines untouched.

What not to do

• Nuke everything. Block GPTBot and OAI-SearchBot and you vanish from ChatGPT answers — and 90% of AI referral visits in the index came from ChatGPT.
• Block by User-Agent blindly. Real bots suffer; fake ones just change their name.
• Confuse crawlers with people. A bot that runs JS is a human inside an assistant; one that doesn't is a crawler. Only throttle the latter, and gently.

Good news: AI orders in issue #1 of the index were 0, but the traffic is real. Your job isn't to evict the bots — it's to let them read your products cheaply. Then the load becomes manageable and your store stays visible to AI.

FAQ