Why your keyword alerts keep failing you

Somewhere in your Slack there is a channel called #mentions. You set it up eighteen months ago. It fires forty times a day. You have it muted.

This is the standard lifecycle of a keyword alert, and it's worth being precise about why it happens, because the failure isn't the tool being too noisy. The failure is that keywords were never the right unit of meaning in the first place.

A keyword is not an intent

Take a company that sells project management software. The obvious alert is "project management tool." Here is a sample of what that phrase actually surfaces on any given Tuesday:

• A student asking for free options for a group assignment
• A thread from 2019 that got necro-bumped
• Seventeen vendors replying to each other's promos
• A person who wants a self-hosted, offline, no-account tool and says so explicitly
• One actual buyer, mid-thread, asking what people switched to after their team outgrew Trello

Four categories of junk and one lead, and the keyword matched all five identically. That last person is worth interrupting your day for. The other four cost you the attention you needed to catch them.

The uncomfortable math: if your alert fires 40 times a day and one of them matters, you need to be 97.5% diligent about reading noise to catch 100% of signal. Nobody is. So the channel gets muted, and the one buyer scrolls past unanswered.

"More filters" is a trap

The usual fix is boolean gymnastics. Add negative keywords. Exclude subreddits. Require the word "recommend." I spent a while doing this myself before building ThreadSonar, and the pattern was always the same: every filter that removed junk also removed the weirdly-phrased real thing.

Real buyers don't write like your boolean query. They write "our PM setup is holding the team hostage and I'm done." No tool name, no "recommendation," no category keyword. A human reading that sentence knows instantly it's a buyer. A keyword system has no idea it exists.

That's the actual gap: keyword systems match text, but qualification is a judgment about a person. Who is talking? Are they venting or shopping? Are they in your market or a student? Did they already reject your whole category two sentences earlier?

What judgment looks like

When we built our pipeline we ended up with three stages, and the shape matters more than our specific implementation:

Cheap hygiene first. Language, length, age, duplicates. Boring, free, removes a third of everything.

Semantic distance second. Not "does it contain the keyword" but "is this near what an ideal match sounds like." This is where the weirdly-phrased buyer survives and the necro-bump dies.

An actual judge last. A model reads the surviving post with your context: what you sell, who buys it, who your competitors are, and, critically, your disqualifiers. It scores the post, says why it matched, and tags who's talking: an in-market buyer, your own customer, someone asking for a feature, a competitor's unhappy user.

That last tag turned out to matter more than we expected. A current customer venting about you and a stranger shopping for your category can contain identical keywords. One is a churn save, the other is pipeline. Any system that can't tell them apart isn't listening, it's grepping.

The test to run on your own stack

Open your mentions channel. Take the last 20 alerts. For each one, answer two questions: would I actually reply to this, and if yes, is the thread still alive by the time I saw it?

When we ran this with early users, the median was 1 reply-worthy alert in 20, and roughly half the reply-worthy ones were already dead threads. That's the real cost of the firehose. It isn't the noise. It's that the setup trains you not to look, so speed dies too, and in live conversations speed is most of the value.

If your numbers look like that, the fix isn't a better boolean. It's a system that reads before it alerts. Ours delivers the post, a 0 to 100 score, the reason it matched, and who's talking, and refunds anything you mark not-relevant. But whatever you use, hold it to the same bar you'd hold a human SDR: if it interrupts you, it should be right most of the time, and it should be able to tell you why.