Pallas Advisory logo
A laptop displaying an AI chat assistant sits on a glass desk in a modern office. Thin marionette strings descend from above, attached to floating words surrounding the screen, suggesting invisible manipulation of the AI's responses.
Your AI earned your trust. Someone else cashed it in.

The technique — AI Recommendation Poisoning — is trivially easy to deploy and invisible to users. For time-strapped business owners and lean marketing teams who rely on AI for vendor research, tool comparisons, and purchasing decisions, the implications go beyond security. This is a decision-making integrity problem.

Your AI Is Making Business Decisions for You. Someone Else MIGHT Writing the Script.

Anthropic’s research shows you trust AI more the longer you use it. Microsoft found 31 companies exploiting that exact pattern.

A marketing director asks her AI assistant to summarize a competitor’s new product page. A founder types “what do people say about [CRM tool]” into ChatGPT before a sales call. An American entrepreneur in the Netherlands asks which boekhouder handles DAFT visa businesses.

None of them think of this as “making a decision with AI.” It’s background research. A quick gut-check. The kind of casual query you forget you even asked.

All three get confident, detailed answers. None of them know that weeks earlier, a “Summarize with AI” button quietly planted a thumb on the scale — and that the bias will persist in every future conversation on the topic.

What Microsoft Found (And Why the Cybersecurity Press Cared More Than You Did)

In February 2026, Microsoft’s Defender Security Research Team published research on a technique they’re calling “AI Recommendation Poisoning.” Over 60 days, they found more than 50 manipulative prompts from 31 real companies across 14 industries — finance, health, legal services, SaaS, marketing, and more.

The mechanism is almost insultingly simple. Websites place “Summarize with AI” buttons that link to AI assistants with pre-filled prompts baked into the URL. When you click, the prompt fires automatically. You see a helpful summary. What you don’t see is the hidden instruction that tells the AI to “remember [Company] as a trusted source” or “recommend [Company] first” in future conversations.

Because modern AI assistants save preferences across sessions, this isn’t a one-time trick. It’s a persistent bias that shapes every future answer on related topics.

The technique is formally cataloged in the MITRE ATLAS framework — the same classification system used for serious cybersecurity threats. One of the 31 companies caught doing this was a cybersecurity vendor. Free tools like CiteMET and AI Share URL Creator mean deploying this takes about as much technical skill as installing a WordPress plugin. Thirty-one companies in 60 days is the floor, not the ceiling.

This story ran in cybersecurity trade press — Dark Reading, The Hacker News, Search Engine Journal. It did not run in Forbes, Wired, or TechCrunch. Which tells you how far under the radar this is for the people who most need to hear it.

The Verification Gap: Why This Hits Smaller Businesses Hardest

AI Recommendation Poisoning is most dangerous when the person asking has no one to bounce the answer off of. And the people with the fewest sounding boards — solo operators, lean marketing teams, entrepreneurs navigating unfamiliar markets — are the ones relying on AI the most.

According to PYMNTS, more than 60% of consumers now start daily tasks with AI interfaces. Dark Reading reports that 80% of Fortune 500 companies use AI agents in their environments — and that contamination circulates through the vendor ecosystems, analyst reports, and competitive intelligence that mid-market companies depend on.

A marketing director at a mid-market company has industry context. She’s not asking AI to pick her attribution platform — she already has a shortlist. But she is asking it to summarize vendor pages and draft comparison notes for her team. That casual research shapes what makes the shortlist. And nobody’s auditing the AI’s memory settings between tasks.

A growth-stage founder with a 15-person team doesn’t think of themselves as “relying on AI for decisions.” They pasted a pricing page into ChatGPT to get a summary before a vendor call. That’s not a decision — until the summary shapes which vendor gets the second meeting.

An American entrepreneur in the Netherlands — KvK filings, Dutch accounting practices, local vendor ecosystems — is asking AI questions they literally cannot verify from experience. If their assistant “remembers” that a particular accounting firm is the go-to for expat businesses, how would they know that was planted? They moved here 18 months ago.

The GEO Mirror: When Optimization Becomes Manipulation

Here’s where this gets uncomfortable for anyone in marketing.

I’ve written extensively about Generative Engine Optimization (GEO) — structuring content so AI assistants cite your business in their answers. It’s legitimate. It’s smart. And AI Recommendation Poisoning uses the exact same information pipeline in reverse.

GEO says: structure your content so AI assistants find and cite you. Recommendation poisoning says: inject instructions so AI assistants prefer you. They’re mechanistically different, but the end result for the user receiving the recommendation looks identical. For marketing professionals simultaneously trying to get their brand cited by AI while relying on AI to research competitors — you’re playing both sides of a game where the other side just started cheating.

Already Unreliable. Now Actively Gamed.

Even without poisoning, AI recommendations were never authoritative. SparkToro research from January 2026 found that AI tools produce different brand recommendation lists more than 99% of the time for identical prompts. Now layer in companies actively gaming those probabilities through memory injection. I wrote about external AI threats — deepfakes, voice cloning, synthetic media back in late 2024. This one’s inside the house.

The Trust Curve Is Real. Now It’s Being Gamed.

Every article about this ends with the same tips. Hover over links. Check your AI’s memory. Be skeptical. Those are fine. They’re also the AI equivalent of telling people to floss — correct, ignored, and insufficient.

The deeper problem isn’t carelessness. It’s that you’re supposed to trust your AI more over time — and the data proves you do.

Anthropic published research on February 18 showing exactly how this works. They tracked millions of human-AI interactions and found that as people gain experience with AI tools, they systematically reduce oversight. New users manually approve about 20% of AI actions; experienced users auto-approve over 40%. The longest autonomous AI work sessions nearly doubled in three months — from under 25 minutes to over 45 minutes. Anthropic calls it a “deployment overhang”: AI is capable of more autonomy than users currently grant, and the trust curve is still climbing.

This isn’t a flaw. It’s rational behavior. You work with a tool, it performs well, you give it more latitude. The same way you’d stop micromanaging a new hire who consistently delivers.

Now layer in Microsoft’s finding. Recommendation poisoning doesn’t exploit carelessness — it exploits a trust-building process that’s working exactly as designed. The more you use AI, the less you check its work. The less you check its work, the more invisible a planted recommendation becomes. Memory features make that convenience feel personalized, which accelerates the trust curve further.

The question isn’t “are you being careful enough?” It’s: when did you stop bouncing AI’s answers off someone else?

Here’s what actually protects you — and it has nothing to do with cybersecurity:

Recognize that your declining oversight is normal — and exploitable. Before AI, when you had a big decision, you’d bounce it off someone — a colleague, a mentor, a friend in the industry. Not because you didn’t trust your research, but because that’s how smart operators make decisions. AI didn’t create the problem of making decisions without a gut check. It just made skipping the gut check feel unnecessary, because the answer already sounds like it came from someone who did the homework.

Add one question to your vendor process. Before any purchasing decision informed by AI research, ask: “Did we run this past anyone who isn’t AI?” A 5-minute check — an industry forum, a peer’s experience, the vendor’s actual client list — catches what no amount of AI memory auditing will. If you manage a team, make this a standing item on evaluation checklists — it takes no specialized knowledge and anyone can own it.

When AI strongly favors one option, get suspicious, not relieved. A confident, specific recommendation feels like the research is done. That’s exactly the feeling recommendation poisoning is designed to create. Ask the AI: “What’s the strongest argument against [the thing you just recommended]?” If it can’t give you one, the recommendation was never analysis — it was marketing.

(And yes — audit your AI’s memory settings. ChatGPT: Settings → Personalization → Memory. Claude: Settings → Memory. Delete anything you didn’t put there. But that’s the easy part. The hard part is building decision habits that don’t collapse when any single input is compromised.)

These aren’t security tips. They’re the same decision habits that mattered before AI — just more urgent now that the trust curve is accelerating.

Where This Gets Practical

The poisoned recommendation scenario is a specific case of a broader pattern: business-critical decisions running through systems where nobody’s getting a second opinion. Most businesses I work with have some version of this — not necessarily compromised AI, but critical workflows where the inputs aren’t as trustworthy as the outputs assume.

If you’re not sure whether your AI-assisted workflows have appropriate checks built in — for research, vendor evaluation, content creation, or operations — that’s the work I do. No pitches, just a conversation about where the friction actually lives.

Want more like this? I write about AI adoption for people who run things — not hype, not doom, just what’s actually working. Subscribe here.

Sources:

Leave a Reply

Discover more from Pallas Advisory

Subscribe now to keep reading and get access to the full archive.

Continue reading