Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.keupera.com/llms.txt

Use this file to discover all available pages before exploring further.

AI assistants don’t just answer from training data — they crawl the live web. The bot tracker tells you which AI bots are reading your site, how often, and which pages they touch. The data feeds the Analytics → Bot Traffic dashboard.

Why a separate tracker?

The JavaScript analytics pixel only fires when a real browser executes JS. Most AI crawlers fetch raw HTML and never run scripts, so they’re invisible to client-side tracking. The bot tracker runs server-side (or via the WordPress plugin) and inspects the User-Agent of every request.

Bots detected

BotOperator
GPTBot, ChatGPT-User, OAI-SearchBotOpenAI
ClaudeBot, Claude-Web, Anthropic-aiAnthropic
Google-Extended, GooglebotGoogle
PerplexityBotPerplexity
Meta-ExternalAgent, FacebookBotMeta
BytespiderByteDance
Applebot-ExtendedApple
Cohere-aiCohere
YouBotYou.com
CCBotCommon Crawl
Diffbot, OmgiliOther AI training crawlers
The list is updated continuously as new agents appear.

Enable it

1

Open the website settings

In the Keupera dashboard open Settings → Website.
2

Toggle 'Enable Bot Tracking' on

Flip the switch. Keupera now accepts bot events for this website.

Install the collector

Pick whichever option matches your stack.
Detect known bot UAs in your edge layer (Cloudflare Workers, Vercel Middleware, Nginx, Express, etc.) and POST to /api/v1/track-bot with one or more events:
curl -X POST https://app.keupera.com/api/v1/track-bot \
  -H "Authorization: Bearer $KEUPERA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "events": [
      { "bot_name": "GPTBot", "path": "/pricing", "visit_date": "2026-05-08" }
    ]
  }'
Batch up to 50 events per call. The endpoint is idempotent on (bot_name, path, visit_date) — duplicates increment the visit counter rather than creating new rows.
const BOTS = /GPTBot|ClaudeBot|PerplexityBot|Google-Extended|Bytespider|CCBot|Anthropic-ai/i;

export default {
  async fetch(req, env, ctx) {
    const ua = req.headers.get("user-agent") || "";
    const match = ua.match(BOTS);
    if (match) {
      ctx.waitUntil(
        fetch("https://app.keupera.com/api/v1/track-bot", {
          method: "POST",
          headers: {
            "Authorization": `Bearer ${env.KEUPERA_API_KEY}`,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            events: [{
              bot_name: match[0],
              path: new URL(req.url).pathname,
              visit_date: new Date().toISOString().slice(0, 10),
            }],
          }),
        })
      );
    }
    return fetch(req);
  },
};

Read the data

Bot data is exposed both in the dashboard and via the API:
EndpointReturns
GET /api/v1/bot-traffic/summaryTotal visits per bot
GET /api/v1/bot-traffic/dailyDaily time-series per bot
GET /api/v1/bot-traffic/top-pagesMost-crawled paths
See the full schemas in the Analytics API reference.