The Invisible Traffic: How to Finally Track the AI Agents Crawling Your Site

For years, your website has been haunted by ghosts. Thousands of visitors—ChatGPT, Claude, Perplexity, and Gemini—are landing on your pages, scraping your data, and using it to train models or answer live user queries. The problem? They don’t execute JavaScript. That means your Google Analytics dashboard shows a big fat zero for some of the most consequential traffic your brand will ever receive.

In the old world, if a user didn’t click a blue link, the visit didn’t exist. In the new world of Generative Engine Optimization (GEO), the “visit” happens when an AI agent reads your site to cite you in a LLM response. Today, OtterlyAI launched Agent Analytics, a tool designed to turn these digital ghosts into measurable data.

The Why: Why “Invisible” Traffic is Killing Your Marketing Strategy

Traditional analytics like Google Analytics 4 are built for humans. They rely on a tracking code that fires when a browser renders a page. But AI agents—the bots that power the answers users see in Search Generative Experience (SGE) or Perplexity—bypass the browser entirely. They hit your server directly.

If you can’t see them, you can’t optimize for them. You might be wondering why your brand isn’t being cited by Claude, only to realize Claude’s bot hasn’t successfully crawled your pricing page in three months due to a server-side block you didn’t know existed. Forget SEO. The New Battleground is AI Visibility. OtterlyAI’s Agent Analytics moves the tracking from the browser to the server log, giving you a literal “source of truth” for who is actually reading your content.

Step-by-Step: How to Make AI Traffic Visible

OtterlyAI has made the integration relatively painless, focusing on where your traffic hits first: the edge or the server.

Select Your Integration Method: Choose between a Cloudflare Worker, a WordPress plugin, a Netlify Edge Function, or a direct Webhook.
Connect Your Server Logs: If you use Cloudflare (the easiest route), deploy the OtterlyAI worker. This will sit at the “edge,” silently noting every incoming request from identified AI bots without slowing down your site. The End of the AI Shakedown: Cloudflare Just Handed the Keys Back to Creators allows you to take control of these crawlers.
Categorize the Bots: Once data begins to flow, use the dashboard to filter between “Indexers” (looking to list you), “Trainers” (using your data for future models), and “On-Demand Fetchers” (someone is asking about you right now).
Audit Your High-Value Pages: Check which agents are hitting your conversion-heavy pages. If ChatGPT is hitting your blog but ignoring your “Best Features” page, you have a surfacing problem.
Correlate Crawls to Citations: Compare your crawl frequency against OtterlyAI’s “Brand Report” to see if a spike in bot activity leads to a higher Share of Voice (SoV) in AI search results.

💡 Pro-Tip: Don’t just look at volume; look at “Crawl-to-Citation” lag. By tracking when a bot hits a new product announcement and when that info first appears in a ChatGPT response, you can map the “index speed” of different LLMs, helping you time your PR launches more effectively.

The Buyer’s Perspective: Is It Worth the Switch?

Until now, marketing teams have been flying blind, using “referral traffic” as a proxy for AI success. But referral traffic from AI is notoriously low because the goal of an AI agent is to give the user the answer without them needing to click through.

OtterlyAI’s value proposition isn’t just about showing you a list of bots; it’s about proving ROI to stakeholders. When a CMO asks, “Why are we spending $10k a month on SEO if clicks are down?” you can now point to the data showing a 400% increase in AI agent fetches, leading to your brand being the top-cited source in Google AI Overviews. The Death of the Click: How Real Estate Agents are Bracing for the “Answer Engine” Era highlights how this shift is impacting various industries.

Compared to raw log analyzers (which are a nightmare for non-technical marketers) or standard SEO tools (which are still playing catch-up with LLMs), OtterlyAI is positioning itself as the “Google Analytics of the AI Era.” It’s a niche but increasingly vital category.

FAQ

Does this tool track what specific users are asking ChatGPT?
No. It tracks the agent (the bot) that ChatGPT sends to your site to find the answer. You see the page they visited, not the private prompt the user typed.

Will installing this slow down my website?
No. By using Edge Functions (like Cloudflare Workers), the tracking happens at the network level, independent of your site’s loading speed.

Is this GDPR compliant?
Yes. Since the tool tracks bots and server-level requests rather than individual human user behavior or PII (Personally Identifiable Information), it is inherently more privacy-friendly than traditional tracking.

Ethical Note/Limitation: While Agent Analytics shows you who is visiting, it cannot force an AI agent to cite you accurately or prevent an LLM from “hallucinating” facts about your brand.