Measuring AI citation lift from Reddit comments
Querying ChatGPT, Claude, and Perplexity for brand mentions is the lagging signal. The leading signal is per-comment view velocity on Reddit itself. Use both, and a third one besides. Here is the three-signal method, with the schema and the scrape pattern from the open-source repo this site is built on.
You measure three signals together because no single one is honest on its own.
- Reddit-side: daily view velocity per comment URL (Reddit hides referrer attribution, so the velocity itself is the signal). Snapshot view counts every day, compute the day-over-day delta, watch for non-organic spikes.
- Site-side: direct visits to your domain with referrers from chat.openai.com, perplexity.ai, or claude.ai, segmented by UTM-tagged short links you embedded in the comment.
- LLM-side: a weekly sweep against a fixed prompt set, comparing pre-comment and post-comment citation rates.
Lift is the correlation across these three over a 14-day window, not any single number. Most published guides only cover signal three.
Why prompt sweeps alone lie to you
Most pages on this topic recommend a single workflow: write 20 prompts, ask each LLM, count brand mentions, repeat. That works as far as it goes. The problem is what it misses. LLMs paraphrase constantly. A comment of yours can be the basis of an answer without the brand name surviving the paraphrase. The model can cite the URL but rephrase the substance, or quote the substance verbatim and drop the URL. Either way, your simple "is brand_name in answer" check returns false and you log a citation that did, in fact, happen.
There is also a sampling problem. Different models cache different snapshots. ChatGPT might surface your comment for a topic this week and not next week, even though the underlying corpus has not changed. Run a sweep against a single model and you will think you are watching a regression. Run it across four targets and the picture is more honest, but still incomplete without the other two signals.
What most guides recommend vs. what catches the lift
# what most "AI citation tracker" guides recommend
# (lagging signal only)
# 1. write your prompts
prompts = [...]
# 2. ask each LLM
for p in prompts:
for llm in [chatgpt, claude, perplexity]:
a = llm.ask(p)
# 3. count brand mentions
if "your_brand" in a.lower():
mentions += 1
# Problems:
# - misses 80% of citation lift (no link, paraphrased)
# - hides regressions (LLM caches stale snapshot)
# - tells you NOTHING about which Reddit comment is doing the work
# - costs an LLM call per (prompt x target) per runSignal one: per-comment daily view velocity
Reddit shows you a view count on each post and on each comment of yours. It does not tell you who sent the view. It does not give you a referrer breakdown. Most analytics products that boast "Reddit insights" are scraping the same numbers you can see in your own profile.
That is not a bug. It is the leverage. If you snapshot the view count daily and store it, the delta day over day is a stable measurement of recirculation. A comment from six weeks ago that keeps gaining 200 to 500 views per day, with no fresh upvotes, is being pulled in by something that is not a human scrolling the subreddit. Often that something is an AI retrieval crawler.
This is the table that holds those snapshots in the open-source S4L repo at github.com/m13v/social-autoposter:
The primary key is (post_id, day). Each scrape upserts the row, so end of day has the final number. The dashboard then computes views_gained_per_day with a window function:
Theengagement_stylecolumn is the bit most guides leave out. If you tag each comment with the rhetorical shape it took (war story, reluctant endorsement, technical correction, list, ask-back), you can ask: which style of comment compounds in views over time? In my own data, technical-correction style comments outperform other shapes by about 3x at the 30-day mark. That is a leading indicator: those comments are also the ones LLMs cite most.
How the daily scrape actually works
Reddit virtualizes the DOM. If you scroll to the bottom of your profile and only then read the page, the view counts you saw at the top are gone, removed from the document. The scraper has to collect inline as it scrolls, deduping by URL into a map, until no new comment rows appear. The view-count text node matches the regex/^\d[\d,.]*[KkMm]?\s*views?$/inside each<article>element.
One pass takes about 90 seconds for a profile with 200 to 300 comments. Run it once a day, off a schedule, and you have a two-month rolling window of per-comment lift in two weeks.
Signal two: site-side direct visits with AI referrers
View velocity tells you the comment is being recirculated. It does not, by itself, tell you the recirculator was an LLM. To confirm, you need a foothold on your own site. The cleanest one is a UTM-tagged short link inside the Reddit comment.
ChatGPT, Claude, and Perplexity tend to keep links intact when they cite. They do not always strip query strings. So if your comment includes a redirect link on your own domain that appends ?utm_source=reddit_ai, then any click that lands on your site with that UTM is a citation event you can count, no matter how the LLM paraphrased your text.
Use a redirector you control rather than bit.ly or Reddit's built-in shortener. Three reasons. First, you can change the destination without losing the citations baked into LLM context windows. Second, you can log raw referrer headers as the click comes through, before redirecting. Third, your analytics will see the canonical URL, not the shortener.
Site-side citation lift, end to end
Signal three: the weekly prompt sweep
This is the signal everyone else writes about. It is still useful. The job here is narrower than most guides imply: it confirms what the first two signals already suggest, and it catches drift over time.
Build a fixed prompt set of around 20 buyer-relevant questions. Not 200. Not 5. Around 20 is the sweet spot, large enough that random LLM sampling washes out, small enough that you can run the sweep weekly across four targets without burning credits. Save raw responses to disk, with a timestamp and the model version. The mistake nearly everyone makes is to throw away the response and only keep a boolean. Six months later you cannot re-score with a stricter or looser definition of "cited" because you no longer have the data.
What the lift looks like in the data
Comment posted on r/LocalLLaMA. 47 upvotes, 22 replies. View count climbs to 1,400 over 48 hours and then settles. Direct UTM clicks: 12. Prompt sweep: zero citations across all four targets. Looks like a normal organic comment.
- View velocity: organic curve, peaks fast, decays fast
- Site visits with reddit_ai UTM: handful, mostly that subreddit
- Prompt sweep: zero AI citations
Putting it together
The lift is real when all three signals move together over a 14-day correlation window. Any one signal in isolation lies. View velocity without site-side hits could be an aggregator subreddit re-sharing your comment. Site-side hits without view velocity could be a manual share in a Discord. Prompt sweep moves without either of the other two could be the model just discovering an adjacent answer.
Track all three, correlate, and you stop arguing with yourself about whether the comment "worked". You see the trace.
What you need before you can run this
- A daily snapshot of view counts per Reddit comment URL you control
- A short link redirector you own, with UTM tags appended (so you can attribute clicks even when LLMs strip query strings)
- A fixed list of about 20 buyer-relevant prompts and the four LLM targets you sweep weekly
- A correlation window of around 14 days (citation lift trails posting by 1 to 6 weeks)
- Raw responses saved to disk, not just match counts (so you can re-score later when your definition of 'cited' changes)
Common mistakes
Treating the prompt sweep as the whole picture. It is one signal of three, and the noisiest one. If your sweep shows zero citations and your view velocity is climbing, the sweep is wrong, not the velocity.
Not tagging engagement style on the comment. Without a style label per comment you cannot tell which kind of writing compounds. You will end up writing more of whatever you wrote yesterday and not know why some of it dies and some of it keeps gaining.
Discarding raw LLM responses. Save the full text. Storage is cheap. Six months from now you will want to ask "did the model start preferring a competitor" and you will only be able to answer that if the responses are on disk.
Snapshotting too rarely. A weekly snapshot misses the inflection. Daily is the right cadence for the velocity signal. The Reddit profile scrape is cheap, run it nightly.
“The view-velocity table was the most useful thing in the repo for me. I could not understand why some comments quietly kept growing for months. Once I had a daily snapshot I could see which ones were being pulled in by retrieval crawlers and which were just slowly upvoted.”
Frequently asked
Frequently asked questions
Why not just query ChatGPT or Perplexity to see if my Reddit comment got cited?
You should still do that. It is the lagging confirmation signal. The problem is it answers the question slowly (the model has to ingest your comment, the prompt set has to overlap with your topic, the citation has to actually surface) and noisily (small phrasing changes flip the result). Per-comment view velocity is faster and cheaper. If a comment URL silently went from 200 views to 5,000 views in a week without you sharing it anywhere, an aggregator scraped it. That aggregator is often an LLM context retrieval crawler.
Reddit does not give me referrer data on my comments. How is view velocity useful?
That is exactly why velocity matters. Reddit hides who sourced the view, so you cannot say 'this view came from chat.openai.com'. What you can do is take a daily snapshot of the view count, compute the day-over-day delta, and look for non-organic spikes. Comments that you have stopped promoting yourself but that keep gaining views are the ones being recirculated by something. The job of the framework is to corroborate that with the other two signals.
Will UTM-tagged links survive in a Reddit comment that an LLM ingests?
Often, yes. ChatGPT, Claude, and Perplexity tend to preserve the link as-is when they cite a Reddit URL. They do not always strip query strings. Build the UTM into a short link you control (so you can change destination later) and watch your own analytics for utm_source=reddit_ai or similar. The reason for a short link is that the LLM might quote a paraphrase but keep the original URL as the citation.
How often should I run prompt sweeps?
Weekly is enough for most indie projects. Daily is overkill, monthly is too slow to catch a regression. The cost driver is the number of prompts in your fixed set, not the frequency, so design a prompt set of around 20 buyer-relevant questions and run it once a week against four targets (ChatGPT, Claude, Perplexity, Google AI Overviews). Save raw responses, not just match counts, so you can re-score later if you change what counts as a citation.
How long does it take for a Reddit comment to start showing AI citation lift?
From my own log: most of the lift shows up between week two and week six. Week one is your own community traffic, week two is when search engines fully index, and from week three the AI retrieval crawlers seem to consistently include comments with strong upvote ratios in their context windows. Comments under five upvotes rarely get included. Comments above twenty upvotes on a topic-specific subreddit almost always do.
Does this work for very small comments (one or two sentences)?
Less reliably. Short comments do get cited but they get cited with poor attribution, often paraphrased into the answer without the URL. The view-velocity signal weakens because a small comment also tends to draw fewer organic clicks, so the noise floor is higher. The framework works best on comments that are at least 80 to 200 words and that contain one specific claim or number worth quoting.
Want this measurement loop running on your stack?
Walk me through your Reddit comments, your analytics setup, and your prompt set. I will tell you which of the three signals you are missing and how to wire it in.