Reddit comment discovery overhead is a 5-gate cascade, not a search bill

If you arrived from an X reply, the question you are probably asking is: why does finding good Reddit threads to comment on feel so expensive? Most guides answer with a single number: 10 to 15 hours a week, or 79 dollars a month. Both are wrong. The actual cost is spread across five filter stages, each one discarding more candidates than it keeps. The open-source S4L pipeline writes every stage down with named thresholds, which is the only way to see the shape of the overhead. Here it is.

The shape of the cascade

A useful frame: discovery is not a queue of things to do, it is a queue of things to discard. Every stage's job is to throw work away cheaply so the next stage spends its budget on a smaller, better pool. The trap most playbooks fall into is treating “hours spent searching” as the whole bill, when search is the cheapest step in the cascade. Most of the dollars land on candidates that get fetched, snapshotted twice, and then dropped at a gate two or three steps later.

Gate 1: fetch (the only stage most pages count)

The pipeline hits up to 50 subreddits per cycle by batching them five at a time into multi-sub requests (Reddit's old.reddit JSON API accepts r/a+b+c+d+e), capped at 10 batches per cycle to stay inside the public rate limit. A clean cycle uses 10 to 15 HTTP requests with a 4-second base delay; on a 429, exponential backoff stretches the delay to 30 seconds, and 3 consecutive failures bail the cycle. The number of fresh searches the LLM is allowed to author for the cycle is bounded by MAX_DISCOVER_SEARCHES = 3 (overridable via SAPS_REDDIT_MAX_SEARCHES).

This is the line item most “Reddit marketing tool” reviews quote when they say discovery is expensive. It is not, in dollars. Reddit publishes a public JSON endpoint that costs zero per call, and the throughput limit is set by their rate-limiter, not by the pipeline. The fetch stage's real cost is correlation: every thread it surfaces is a thread the later stages will spend on.

Gate 2: ripen (the gate nobody else writes down)

A freshly-found thread looks alive at fetch time and might be dead 5 minutes later. Posting a careful reply on a thread whose author has already drifted off and whose readers have stopped clicking is the most common form of wasted draft cost. The ripen stage exists to catch exactly that: snapshot the thread at T0, sleep, snapshot at T1, and compute a momentum composite.

The 4x weight on comments versus upvotes is deliberate: one new comment on a thread is a stronger signal that someone is still reading than four new upvotes. The +5 intent boost lets a flat-but-on-theme thread (one upvote, no new comments, but the title literally asks “what do you use for X”) clear the floor it would otherwise miss. The floor sits at 1 instead of 5 because once the intent boost shipped, dropping the raw floor was safe: pure-noise threads still need a real upvote or new comment to pass.

Gate 3: draft (the LLM relevance check)

Even on a thread that cleared momentum and shows on-theme title intent, there is one more discard step. The draft prompt asks the LLM to decide whether the thread's actual audience has a plausible bridge to the project being represented. Plenty of threads pass the algorithmic gates and still produce token-overlap false positives: an r/programming thread about “best tool for X” that turns out to be about C++ build systems, when the project being represented is a tweet-ghostwriting service. The LLM can read that, and when it does, the candidate is marked status='failed' with reason draft_gate_omit.

The reason this is treated as permanent and not transient: the same dead thread would otherwise keep clearing momentum (engagement on the thread is real, just not for our audience) and the pipeline would re-pay the fetch and gate cost every cycle on a decision that always lands the same way. The one-strike rule went live on 2026-05-07 and dropped the recurring spend on permanently-omitted threads to zero.

Gate 4: post (where the queue earns its keep)

The post stage drives a real browser through CDP to submit the reply. Failures here split into two buckets, and the bucket determines whether the discovery cost is recovered or wholesale lost.

Permanent failures mark status='failed' so they never re-enter the cascade. Transient failures stay status='pending', bump attempt_count, and get salvaged by the next cycle's Phase 0 as long as the row is still inside FRESHNESS_HOURS=24 and below MAX_ATTEMPTS=3. If the salvaged row's draft is younger than DRAFT_TTL_MIN=60, the LLM redraft is skipped entirely. That is where the persistent queue earns its place in the architecture: without it, every transient browser crash would discard the discover+ripen+draft sunk cost.

Gate 5: salvage (the cleanup that bounds the bill)

Phase 0 of the next cycle does two things. It hard-expires any pending row older than 24 hours (no audience left to reach), and it re-assigns still-fresh orphan rows to the current batch_id so they get one more retry. The 30-minute retry backoff stops the pipeline from re-attempting a freshly-failed candidate inside the same cycle window; the failure reason needs time to stabilize before we know whether it was transient.

The queue schema makes the bookkeeping cheap. One row per thread URL (UNIQUE constraint), every state transition recorded with last_failure_reason, and three indexes carefully chosen for the access patterns Phase 0 actually performs (salvage SELECT, per-cycle COUNT, log_post lookup). Full schema is in scripts/migrate_reddit_candidates.sql.

What the cascade tells you about cost per posted comment

Run this math against your own discipline. If you scroll /new for 30 minutes and pick 3 threads to comment on, you have run the same five gates by eye. The fetch was the scroll. The ripen was the moment you skipped a thread with no replies in an hour. The draft gate was the moment you decided the audience would not bridge. The post gate was the moment your reply went up. The salvage gate was the moment you closed the tab and never came back. The pipeline writes these moments down so the cost is measurable; the manual version feels free until you tally the hour.

A useful rule of thumb from running the OSS pipeline in production: the ratio of fetched threads to posted comments lands somewhere between 30:1 and 80:1 on a tight ICP. Most of the discard happens at ripen and the LLM draft gate. The fetch stage's output is wide on purpose, because a narrow fetch reaches the gates with too little to work with and they starve. A wide fetch with strict gates costs the same as a narrow fetch with loose gates, and the first one ships better comments.

If you are doing this without the pipeline

The minimum useful version is shorter than the OSS repo. Snapshot a thread's upvotes and comment count when you find it. Wait at least 5 minutes. Re-fetch. If neither number moved and the title is not literally asking for a tool, skip. If the title is asking for a tool, draft anyway. Keep a list of threads you have already decided against. Re-check that list before drafting anything new.

That single discipline (the do-not-revisit list) recovers most of the dollar-equivalent that the persistent queue gives you. Most people do discovery without one and end up reconsidering the same dead thread three times in a week.