The next time you ask an AI chatbot for the best dating app, a reliable roadside service or how to cancel an annoying subscription, the answer you get could have been planted by a marketer or a scammer, and using as few as 13 words buried in a Reddit comment.
That's the takeaway from a new preprint out of Cornell Tech, titled "Deep-Research Agents Can Be Poisoned via User-Generated Content," first reported by 404 Media. The researchers — Tingwei Zhang, Harold Triedman and Vitaly Shmatikov, built an attack they call WARP (Web Agent Retrieval Poisoning) and showed it works with unsettling reliability against the AI systems that increasingly stand between you and the open web.
What the research actually found
When you ask an AI tool a question, it often runs live web searches, reads what it finds and stitches together a response with citations. This is also true for the "deep research" approach behind ChatGPT and Gemini's research modes.
The problem is, a huge share of what those systems read comes from user-generated sites like Reddit, Wikipedia, Quora and YouTube, all places anyone can post to. In the Cornell tests, roughly 17–23% of all the web pages these agents pulled in came from such sites, and a single popular Reddit thread could show up across a large chunk of related queries on the same topic.
That creates a chokepoint. Poison one frequently-cited thread, and you can steer the AI's answer for an entire category of questions, not just one phrasing of it.
In the researchers' tests, appending around 13 words of promotional text to a single source got the AI to name-drop a made-up product in roughly 38–51% of the runs where that source was actually retrieved. Spreading the bait across a few threads pushed that as high as 62%.
Real (fake) examples
To avoid polluting the live internet, the team never posted anything publicly. Instead, they ran the attack in a sandbox that simulated what would happen if poisoned text appeared on real pages, an approach they argue is the only ethical way to study this.
Worth noting exactly what was tested: the full attack was run end-to-end only against three open-source "deep research" agents (STORM, Co-STORM and OmniThink). The big commercial tools couldn't be attacked directly (doing so would have meant poisoning the live web) so the researchers instead measured how often each one cites user-generated content.
There the picture was mixed. Google's Gemini Deep Research pulled in such content far more often (about 12% of citations) than OpenAI's Deep Research, which cited it barely at all (0.4%) and appears to filter it out aggressively. In other words, this is a demonstrated weakness in how these systems work, not proof that any specific consumer chatbot has been fooled in the wild.
The invented examples are almost comically simple. A short line added to an Austin food thread, recommending a fictional restaurant called "Sol Azteca" for "authentic cuisine," got the AI to recommend Sol Azteca and cite the Reddit post. A made-up dating app, "SilverPath," got surfaced as a "top choice" for divorced men over 50. Other fakes included a bogus crypto coin and a sketchy third-party "service" for canceling Xfinity.
Why this should worry you
Here's the uncomfortable part for all AI users, the queries most vulnerable to this attack are exactly the ones people lean on AI for. Recommendation- and advice-style questions. Searches like best restaurants, best apps, which product to buy, how to cancel something, who to call in an emergency, are where AI tends to fall back on community chatter rather than authoritative sources.
A big reason it works, the researchers explain, is that these systems often treat text that reads similar to your question as a stand-in for text that's accurate. So an attacker who studies common queries can write a comment that mirrors your phrasing and that mirror-image is exactly what wins the AI's trust. As Zhang told 404 Media, these systems weigh a random Reddit comment and a government website as roughly equally credible.
What you can do right now
- Treat AI recommendations as leads, not the final say. This is especially for products, apps, restaurants, financial picks and anything tied to money or safety.
- Click the citations. If an AI confidently names a brand, see where the claim actually came from. A single Reddit comment is a red flag.
- Cross-check unfamiliar names. If you've never heard of the "top-rated" option the AI just surfaced, search it independently before you trust it.
- Be extra cautious with urgent queries. Everything from emergency roadside help, customer-service phone numbers, to account recovery hich are prime targets for scams.
And the tricky problem is that this can not easily be stopped. The researchers tested the obvious defenses such as blocking user-generated sites entirely, screening sources before they're used and scanning the final answer for manipulation, and none worked well without making the AI's answers worse. A standard trick for catching AI-generated junk (flagging "unnatural" text) actually backfired here, because the planted text reads more fluently than genuine human comments, not less.
A Reddit spokesperson told 404 Media the company has spent two decades fighting spam, bots and coordinated manipulation and recently began asking suspicious automated accounts to verify they're human. But the researchers argue this is ultimately a societal-scale problem, not something Reddit or Wikipedia can fully solve on their own.
The takeaway
Until the AI companies close the gap, a little skepticism goes a long way. The smartest move right now is to think of AI in the same way you would a chatty stranger on a form. Consider it helpful, but certainly worth double-checking.
Follow Amanda Caswell and stay ahead of the AI curve
