Why I Built an Open Source AEO Tool
It's basically an LLM brand tracker, a proxy for web visibility.
SEO is one part technical and one part content production; AEO (answer engine optimization) is much broader. LLMs serve up content from across the web, showing what content is considered high fidelity. To rank in ChatGPT’s answers, you have to have high web visibility, ie., good brand.
Consider this web visibility view from Ahrefs. For example purposes, I’m comparing brands of a few of the AI tools. They're tracking mentions across the entire web – news sites, forums, social media, you name it. We see Cursor’s explosion in the market a few months ago, but ChatGPT remains more prominent today. This broadly maps to how these tools live in most people’s minds, just adds data to back it up.
But here's the thing - web visibility is the means to the end. To improve it, you need to know exactly where to show up and what to talk about. That's where AEO tools should come in.
The AEO tooling landscape
I evaluated seven different AI brand tracking tools (Ahrefs being one of them). The main issue I encountered with Ahrefs is that it searches the term, not the website. So, for a company with a common name like “Railway”, this wasn’t particularly helpful.
There are many bespoke tools out there, but I selected a few from this list from Graphite. Many of them could tell me "you were mentioned 47 times this week in ChatGPT responses." However, I was fairly shocked to learn most tools don’t actually expose the websites used to cite your brand, when this is the most actionable piece!
Note: there were a few that did include sources, but they were either 1) really expensive or 2) made me enter my own prompt topics as opposed to suggesting a few.
Without the full citation source context, the data is colossally unactionable. Understanding the data isn’t enough to make it impactful - the tool must inform your marketing strategy.
💡 The key to making AEO insights actually influence your marketing strategy:
sources + topics = your entire content, devrel, and growth strategy.👉 Topics where competitors are present and you aren’t = what you write about next
👉 Sources cited where you aren’t present = where you write and distribute
That’s it, that’s the strategy. Which is why topics should be programmatically generated from prompts (not given), and sources are the most actionable part of an AEO tool.
Enter an OSS LLM brand tracker
Being fairly meh about most of the tooling that exists today, I decided to see what I could do myself. I vibe coded in Replit, realized a lot was hard-coded, exported the code, did some more prompt engineering, and voila. An open-source tool to track how your brand shows up in LLMs (just ChatGPT to start): github.com/sarahkb125/llm-brand-tracker.
The tool does a few critical things:
analyzes your website to understand your positioning
identifies competitors (with your input, because context matters)
generates diverse prompts that mirror real user queries
collects ChatGPT's responses to those prompts
tracks which sources are cited in each response
It is truly open source, relying only on an OpenAI API key to get moving. Everything else happens within its own environment.
This was my approach to tooling, but now onto how marketing teams should action on data from any AEO tool.
How to use an AEO tool
The magic happens in three phases: defining what to track, understanding the results, and developing a strategy to bridge the gaps.
Phase 1: Defining Your Topics
Don't just track your brand name. Think about the problems you solve and how people describe them.
the tool will crawl your domain
add aspirational industry terms for prompts (“AEO” may not be popular as a term yet, but where the space is going)
make sure the appropriate competitors are mentioned (both your product and search competitors)
Iterating on all the above will pay dividends to make sure the results you’re building a strategy around are complete and context-driven.
Phase 2: Understanding the results
As an output, you will likely get:
topics you iterated on through prompting
the % of time your brand is mentioned in prompts
sources ordered by relevance
Understanding these results means, at a high level, answering the following questions:
are you present alongside competitors, or are there spaces where your competitors overtake the results?
do you have a niche where only you are primarily present? (this is really hard to achieve, to be clear)
are the sources ones you expected? heard of? community has mentioned to you? (they should not be a total surprise)
This phase is primarily evaluating if the results are skewed or not. If skewed, you may need to go back and generate more prompts per topic, iterate on topics, or reframe who your competitors are.
Phase 3: Using the information to create strategy
This is where strategy emerges from data. When you see the results, pay attention to:
Topics where you're absent but competitors aren't
Which domains appear most frequently
Whether the sources are blogs, documentation, or third-party reviews
Segment the sources into three actionable categories. Third-party blogs respond to community growth - if developers love you, they'll write about you. Forums are pure devrel territory - your team needs to be present and helpful. Media requires traditional PR muscle. Everything else? Probably outside your influence.
Voila, strategy commence.
Final thoughts
The space is very early, so we have no idea where it will go. It’s possible Ahrefs will iterate and win. It’s possible OpenAI/Anthropic/etc will generate their own insights product and all these startups go to die. For now, marketing teams should understand the huge opportunity of getting ahead in ranking their brand in LLM results.
As for the open-source tool, it’s directionally correct but not production-ready. Where I’d like it to go:
be deployable anywhere (it’s not containerized yet nor does it have auth)
find a wider variety of prompts that cover more surface area automatically
have a smoother UX, be less buggy, and more configurable
If you don’t want to pay over $30k/year and want influence on your own results like me, take a look and even contribute if you are so inspired!
Thanks for reading. I love talking growth, marketing, and tech broadly. Find me on LinkedIn or Twitter.
this is awesome, the Gauge team is doing a lot in actionability right now :)