Data Poisoning

Data poisoning is the deliberate corruption of the data an AI system learns from or draws on — training sets, fine-tuning data, or the documents and web pages a model retrieves at answer time — so that the system later behaves the way the attacker wants. Where prompt injection manipulates a model in the moment, poisoning plants the manipulation upstream and waits.

How it works

The attack exploits the fact that modern AI is built on data nobody fully audits. An attacker can seed public sources that are likely to be scraped into training sets, contribute tainted examples to open datasets, or tamper with an organization's own data pipeline — a mislabeled malware corpus, a doctored knowledge-base article. Retrieval-augmented systems widen the surface: a model that consults live documents or search results can be steered by poisoning what it retrieves, which is where the technique converges with SEO poisoning and with hidden instructions planted for AI crawlers. Effects range from degraded accuracy to targeted backdoors — a spam filter taught that the attacker's domains are safe, a support bot that confidently recites a planted "procedure," a security classifier blind to one specific payload. Because the poisoned behavior ships inside a trusted system, it inherits that system's credibility, much like a supply chain attack inherits a vendor's.

How to defend against it

Treat data pipelines as part of the attack surface. Track the provenance of training and retrieval sources, validate and version datasets, restrict who can write to knowledge bases that AI systems consult, and monitor model behavior for drift after data updates. For procured AI, ask vendors how they source and vet training data. And close the human loop: employees should treat AI answers as input rather than verdict, verify consequential claims against primary sources, and report odd model behavior the way they would report a phish — the habits a human risk management program builds for email transfer directly to the agent era.