← Guides

text/markdown for AI agents: a clean version of the product page

Updated 2026-08-03

What an AI agent sees instead of your product

When an AI agent opens a product page, it doesn't see your nice layout. It gets the whole HTML: the header, the mega-menu, the cookie banner, the "customers also bought" carousel, a forty-link footer and a dozen inline scripts. The 200 useful words about the product drown in hundreds of kilobytes of markup. The model spends its context window on noise and may never find the price or the availability.

text/markdown fixes this head-on: you hand the agent a clean text version of the same page — title, description, price, availability, shipping terms — with no menu tags and no scripts.

Why clean markdown beats HTML

Bots come in two kinds. The ones that execute JavaScript behave like humans in a browser. The ones that don't — crawlers like GPTBot, ClaudeBot or PerplexityBot — take whatever the server returns immediately, with no rendering. For them your whole interface is noise, not help.

Clean markdown gives you three things:

• Meaning density. Instead of 300 KB of HTML for one product, the agent gets a couple of kilobytes of text. It reads the substance, not the layout.
• Unambiguous structure. Headings, lists and a spec table parse far more reliably than a soup of <div>s with class names.
• Zero scripts and popups. Nothing to break on render, nothing to skip.

In the Botmetria audit, markdown is its own category worth 20% — the same weight as discovery, and more than logistics or hygiene. It's also the easiest to lift from zero, because almost nobody serves it yet.

How it works: content negotiation

The mechanism is called content negotiation. The agent sends a request header:

GET /product/nachos-crunch HTTP/1.1
Accept: text/markdown

The server sees Accept: text/markdown and, instead of HTML, returns markdown with a Content-Type: text/markdown header. Same URL, different representation. A normal browser asks for text/html and gets the usual page. This isn't cloaking: the content is identical, only the format differs.

The second way is simpler — .md mirrors. Next to /product/nachos-crunch lives /product/nachos-crunch.md with the same content in markdown. A mirror like that is easy to open in a browser, eyeball, and link to from your llms.txt.

Ideally you do both: negotiation for agents that know how to send Accept, and the .md mirror as a fallback.

We serve markdown ourselves — and hold 100/100

The rule is simple: don't recommend what you don't do yourself. botmetria.com serves text/markdown via content negotiation on its own pages. And in our own audit the markdown category — that same 20% — scores 100/100 for us. Call it a dogfood proof: the mechanism works, we live on it.

For context: across 63 stores in Moldova and Romania the median AI-readiness is just 38 out of 100 — the red band (≤40). markdown is one of the most underrated categories here: serving clean text is cheaper than rewriting all your logistics or fixing page speed.

How to turn it on in 20 minutes

On WooCommerce the easiest path is the Botmetria plugin: it builds the markdown version of every product card and sets up both negotiation and the .md mirror automatically, from the data already in the product.

On a custom site, add an endpoint or snippet that returns markdown for each card:

1. Assemble the text: # Title, a description paragraph, and lines for Price, Availability, Shipping/returns.
2. Serve it on Accept: text/markdown and at the URL with a .md suffix.
3. Set the response header Content-Type: text/markdown; charset=utf-8.

Here's what the agent should get:

# Nachos Crunch 200 g
Corn chips with sea salt.

**Price:** 39 MDL
**Availability:** in stock
**Shipping:** 1–2 days in Chișinău, 14-day returns

It's worth mirroring the same price and availability in JSON-LD Product: the agent reads markdown as text and the structured data as machine facts. Together they cover the question reliably.

How to check

• Open the .md mirror in a browser — it should be clean text, no header and no menu.
• Test negotiation from the terminal: curl -H "Accept: text/markdown" https://yoursite/product/.... Expect Content-Type: text/markdown in the response, not text/html.
• Run the free Botmetria audit — the markdown category shows your score and exactly what to fix. Compare yourself with the market in the rating.

Common mistakes

• Renaming HTML to .md. If it's the same markup inside, it's useless. Markdown has to be real text.
• Different content in HTML and markdown. If the markdown price lags behind the page, the agent will quote the wrong one. Keep a single source of data.
• Forgetting Content-Type. Without the right header the agent won't know it's looking at markdown and will treat the response as plain text or HTML.
• Blocking the bots you serve markdown to. Handing GPTBot markdown and cutting it in robots.txt at the same time is a contradiction. And never touch classic Googlebot/Bingbot.

FAQ