AI assistants don’t just answer from training data — they crawl the live web. The bot tracker tells you which AI bots are reading your site, how often, and which pages they touch. The data feeds the Analytics → Bot Traffic dashboard.Documentation Index
Fetch the complete documentation index at: https://docs.keupera.com/llms.txt
Use this file to discover all available pages before exploring further.
Why a separate tracker?
The JavaScript analytics pixel only fires when a real browser executes JS. Most AI crawlers fetch raw HTML and never run scripts, so they’re invisible to client-side tracking. The bot tracker runs server-side (or via the WordPress plugin) and inspects theUser-Agent of every request.
Bots detected
| Bot | Operator |
|---|---|
| GPTBot, ChatGPT-User, OAI-SearchBot | OpenAI |
| ClaudeBot, Claude-Web, Anthropic-ai | Anthropic |
| Google-Extended, Googlebot | |
| PerplexityBot | Perplexity |
| Meta-ExternalAgent, FacebookBot | Meta |
| Bytespider | ByteDance |
| Applebot-Extended | Apple |
| Cohere-ai | Cohere |
| YouBot | You.com |
| CCBot | Common Crawl |
| Diffbot, Omgili | Other AI training crawlers |
Enable it
Open the website settings
In the Keupera dashboard open Settings → Website.
Install the collector
Pick whichever option matches your stack.WordPress (recommended)
WordPress (recommended)
Install the Keupera Connector plugin. It hooks No further setup — bot tracking starts automatically.
template_redirect, inspects every frontend request’s User-Agent, queues hits in a transient, and flushes them in batches of 10 (or every 5 minutes via WP-Cron, whichever comes first) to:Custom server / middleware
Custom server / middleware
Detect known bot UAs in your edge layer (Cloudflare Workers, Vercel Middleware, Nginx, Express, etc.) and Batch up to 50 events per call. The endpoint is idempotent on
POST to /api/v1/track-bot with one or more events:(bot_name, path, visit_date) — duplicates increment the visit counter rather than creating new rows.Cloudflare Worker (example)
Cloudflare Worker (example)
Read the data
Bot data is exposed both in the dashboard and via the API:| Endpoint | Returns |
|---|---|
GET /api/v1/bot-traffic/summary | Total visits per bot |
GET /api/v1/bot-traffic/daily | Daily time-series per bot |
GET /api/v1/bot-traffic/top-pages | Most-crawled paths |