The dominant assumption inside most large-scale AI training pipelines is that data quantity compensates for data quality. Scrape enough of the web, filter for language, deduplicate, and ship. This assumption is expensive, and in 2026 it is increasingly untenable. Data poisoning attacks do not require an adversary to breach a data center. They require only that a malicious actor can place content where a crawler will find it. Given that web-scale crawls for foundation models routinely pull from billions of URLs, the attack surface is enormous and largely unmonitored.
This article maps the current threat landscape around data poisoning, examines the artist-led countermeasures Nightshade and Glaze as case studies in adversarial provenance, and argues that data provenance infrastructure is not a copyright compliance tool. It is a model safety requirement.
What Data Poisoning Actually Is
Data poisoning is an attack on a machine learning model at the training stage rather than at inference. An adversary injects carefully crafted examples into a training corpus to corrupt the model's learned behavior. The corruption can be targeted or indiscriminate. Targeted poisoning nudges the model toward specific misbehavior on specific inputs. Indiscriminate poisoning degrades overall model quality.
The formal threat model distinguishes two phases. In clean-label attacks, the poisoned samples carry correct ground-truth labels but contain imperceptible perturbations that shift the loss landscape in the adversary's favor. In dirty-label attacks, labels themselves are manipulated. Both variants have been demonstrated against image classifiers, large language models, and multimodal systems.
The attack surface expands dramatically for foundation models because pre-training corpora are collected before any specific downstream task is defined. An adversary does not need to know how the model will be fine-tuned or deployed. Poisoning the pre-training corpus can plant latent behaviors that activate only under specific trigger conditions, a property researchers call a backdoor or trojan.
Work published on arXiv by Carlini et al. (arXiv:2302.10149) demonstrated that an adversary who controls as few as 0.01% of training examples in a web-scale corpus can influence model behavior in measurable ways. That fraction sounds small. At the scale of Common Crawl or LAION-5B, it represents millions of documents or images.
How Scraped Training Sets Create Attack Surface
Web-scale training corpora are assembled by crawling public URLs, filtering for quality signals like perplexity or CLIP similarity scores, and deduplicating. The pipeline is automated and the provenance of individual examples is rarely preserved beyond a source URL.
This architecture creates at least three structural vulnerabilities.
Domain expiration attacks. Researchers demonstrated in 2023 that a substantial fraction of URLs in LAION-400M pointed to domains that had since expired or changed hands. An adversary who registers an expired domain can serve poisoned images or text to any future crawl that revisits those URLs. Because crawlers often re-scrape corpora to refresh data, this is not a one-time window.
Coordinated injection at scale. An adversary who controls a content farm, a wiki, or a large social platform can seed thousands of subtly poisoned examples across many apparent sources, evading filters that look for concentrated anomalies from a single origin.
Supply chain attacks on derived datasets. Many training pipelines do not scrape the web directly. They start from publicly released datasets like Common Crawl, C4, or LAION. If a poisoning attack succeeds against one of these upstream sources, every model trained on a derivative inherits the compromise. The lineage is rarely documented in sufficient detail to trace the exposure.
The absence of cryptographic provenance at the data-record level means there is no reliable way to verify that a training example today matches what was indexed at collection time, or to audit whether a given sample was included in a specific training run.
Nightshade, Glaze, and Artist-Side Countermeasures
In late 2023, researchers at the University of Chicago released two tools that reframed the poisoning conversation. Glaze and Nightshade were designed not by adversaries trying to corrupt AI systems but by artists trying to protect their work from unauthorized scraping.
Glaze applies pixel-level perturbations to images that preserve human-perceptible appearance while shifting the CLIP embedding of the image toward a different artistic style. A model that trains on Glazed images learns incorrect style associations. The artist's work appears to be in a style it does not occupy.
Nightshade goes further. It generates adversarial perturbations specifically designed to corrupt the prompt-image correspondence that text-to-image models learn during training. An image of a dog, Nightshade-perturbed, teaches the model to associate the concept "dog" with visual features from an entirely different category. If enough Nightshade images enter a training set, the model's generalization on the targeted concept degrades.
The University of Chicago team published their methodology (arXiv:2310.13828 for Nightshade) and reported that as few as 300 poisoned images caused measurable degradation in Stable Diffusion-style models for targeted concepts. Poisoning effects also transferred to adjacent concepts through the model's learned semantic graph, a property they called concept bleeding.
These tools are significant for two reasons beyond their immediate use case. First, they prove that a distributed, uncoordinated population of data contributors can collectively degrade a model without centralized coordination. Second, they expose what happens when provenance is absent: a training pipeline has no mechanism to distinguish a Nightshade-perturbed image from a clean one because it has no record of what the original looked like or who created it.
The lesson is architectural. Provenance is not just a rights-management feature. It is a data integrity feature.
A Taxonomy of Poisoning Attacks Against Foundation Models
Understanding the threat landscape requires distinguishing attack types by mechanism and by goal.
Backdoor / Trojan attacks embed trigger-activated behavior. The model behaves normally on clean inputs and produces adversary-controlled outputs when a specific trigger pattern appears at inference. Work by Chen et al. (arXiv:1712.05526) established the foundational threat model. Subsequent work has demonstrated backdoor attacks against large language models, multimodal models, and code-generation systems.
Availability attacks aim to degrade overall model performance rather than introduce targeted behavior. They are blunter instruments and easier to detect through standard evaluation benchmarks, though they can be effective against niche capability domains.
Gradient-based targeted attacks use knowledge of the model architecture or loss function to craft maximally effective poison examples. These are the strongest attacks but require white-box or gray-box access assumptions that may not hold for closed training pipelines.
Clean-label poisoning is the most operationally realistic threat for web-scale systems. The adversary does not need to mislabel data. Perturbations are imperceptible to automated quality filters and to human reviewers. The attack from Witches' Brew (arXiv:2009.02276) demonstrated clean-label poisoning achieving high success rates on image classifiers with no label manipulation.
Prompt injection via training data is an emerging variant specific to instruction-tuned language models. If fine-tuning data includes crafted examples that teach the model to treat certain string patterns as override commands, those patterns can later hijack model behavior at inference. The boundary between data poisoning and prompt injection collapses when fine-tuning data is unaudited.
Why Provenance Is a Safety Problem, Not Just a Copyright Problem
The policy conversation around AI training data has concentrated on copyright: who owns the content, whether training constitutes infringement, and what licensing frameworks should govern model commercialization. These are real legal questions. They are not the most urgent technical questions.
The more urgent technical question is integrity. Can a model developer verify that the training corpus delivered for a given run is the corpus that was audited, filtered, and approved? With current infrastructure, the answer is almost universally no.
Provenance, in the technical sense, means a cryptographically verifiable chain of custody from the origin of a data record through every transformation to its inclusion in a training job. This is distinct from metadata like a source URL or a collection timestamp, which are mutable and unauthenticated.
The W3C PROV-DM data model (https://www.w3.org/TR/prov-dm/) provides a formal framework for expressing provenance as a directed graph of entities, activities, and agents. Applied to training data, a PROV-compliant record would capture who created a data record, what transformations it passed through, and which identities attested to its integrity at each stage.
Without that chain, a poisoning attack at any upstream node is undetectable at the training node. The training pipeline receives a poisoned corpus and has no cryptographic evidence that the corpus has been tampered with. Quality filters catch statistical anomalies, not adversarially crafted perturbations designed specifically to evade those filters.
The NIST AI Risk Management Framework (NIST AI 100-1) explicitly lists data poisoning as a risk category under model integrity and calls for supply chain controls on training data. The framework does not mandate specific cryptographic mechanisms, but its risk decomposition maps directly onto the provenance gap described here.
From a policy perspective, treating provenance as a copyright instrument underinvests in its safety function. A framework that records data origin solely to resolve licensing disputes ignores the integrity verification problem entirely. Both functions require the same infrastructure: a signed, auditable record at the data-record level. Building that infrastructure once and using it for both purposes is the correct engineering approach.
Defenses: Detection, Filtering, and Certified Robustness
No single defense eliminates data poisoning risk. The current state of the art relies on layered controls.
Dataset filtering and anomaly detection. Influence function analysis (Koh and Liang, arXiv:1703.04730) estimates the training loss contribution of individual examples. High-influence outliers are candidates for review. This approach is computationally expensive at foundation model scale but tractable when applied to filtered subsets or to fine-tuning corpora.
Data provenance and integrity checks. Cryptographic hashing of training records and signing by attested data sources creates a tamper-evident log. If a record has been modified after attestation, the signature fails. This does not prevent poisoning at the origin, but it closes the supply chain gap between collection and training.
Certified robustness via randomized smoothing. Cohen et al. (arXiv:1902.02918) introduced randomized smoothing as a mechanism for certifying that a classifier's prediction is robust within a defined perturbation radius. Applied to training data, analogous approaches can bound the influence of any single data point on the final model.
Federated and differential privacy training. Training with differential privacy (DP-SGD, Abadi et al., arXiv:1607.00133) clips per-example gradients and adds calibrated noise. This limits the influence of any individual training example, which is exactly the property that limits poisoning effectiveness. The cost is accuracy degradation, but recent advances in private fine-tuning have reduced that gap substantially.
Human-in-the-loop auditing for high-risk domains. For training data used in safety-critical applications, automated filtering is not sufficient. Audit workflows that route high-influence examples to qualified reviewers are operationally expensive but necessary when deployment stakes are high.
Consent receipts and data trust frameworks. If training data originates from contributors who have given structured, verifiable consent under a data trust or fiduciary model, the provenance chain starts at a known, authenticated identity. Poisoning at the origin requires compromising an authenticated contributor, which is a significantly harder attack than injecting content into an anonymous web crawl.
PDAOS and the Case for Consent-Native Training Data
The Personal Data Asset Origination System (PDAOS), developed through Own Your Data Inc, approaches training data provenance from the supply side rather than the detection side. Rather than scraping unattributed content and filtering for quality after the fact, PDAOS anchors data records to cryptographic identities at origin. Each data asset carries a provenance chain: who created it, under what consent conditions, and through what transformations.
For AI training specifically, this architecture means that a model developer can request a corpus of training examples from PDAOS contributors and receive records that are cryptographically signed at origin. Poisoning via domain expiration attacks is impossible because the data does not originate from a URL. Coordinated injection attacks require compromising authenticated contributor identities. Supply chain attacks on derived datasets are traceable because the lineage graph is preserved.
This is not primarily a copyright argument. It is a model integrity argument. A consent-native corpus is a corpus whose provenance is verifiable end-to-end. That property eliminates the largest class of realistic poisoning attacks while simultaneously producing the attribution record that licensing and compensation frameworks require.
Dr. Patrick Fisher's analysis in The Invisible Data (Volume 6 of The Invisible Series) frames data provenance as a prerequisite for trust rather than a compliance add-on. That framing is technically accurate. A model whose training corpus has no verifiable provenance is a model whose safety properties cannot be fully audited. The absence of that audit trail is not a documentation gap. It is a structural vulnerability.
Engineers building or auditing foundation model pipelines in 2026 should treat data provenance infrastructure with the same priority as adversarial robustness testing, red-teaming, and output monitoring. Those practices address the inference end of the pipeline. Provenance addresses the training end. Securing one without the other leaves a well-documented attack vector wide open.
For teams exploring consent-native data infrastructure, the PDAOS specification and MyDataKey identity layer are documented at mydatakey.org. The W3C Verifiable Credentials specification (https://www.w3.org/TR/vc-data-model/) provides the standards-layer foundation for cryptographic attestation of data records and contributor identities.
The threat is not hypothetical. The infrastructure gap is real. Closing it requires treating provenance as a first-class engineering requirement, not a legal afterthought.