Pretraining Data Can Be Poisoned through Computational Propaganda
Researchers demonstrate that computational propaganda can poison large language model pretraining data, embedding harmful behaviors that are difficult to detect.
- Computational propaganda can effectively poison web-scale pretraining corpora.
- Attacks via public discussion interfaces bypass limitations of previous Wikipedia-based studies.
- Poisoned data introduces harmful behaviors that are hard to detect and mitigate.
Previous studies on data poisoning largely focused on static sources like Wikipedia, which do not reflect the complexity of real-world pretraining corpora. This new paper expands the scope by examining attacks via public discussion interfaces, a mechanism used for computational propaganda at scale.
The authors show that adversaries can inject malicious content into these web-scale pipelines. The study specifically analyzes how this poisoned data interacts with modern data curation and filtering processes used during model training.
Results indicate that these attacks are feasible and effective. They introduce harmful behaviors into the resulting models that persist even after standard safety training and are notably hard to mitigate.
Highlights the need for robust data sanitization pipelines and source verification.
Underscores a critical risk of model alignment failure due to corrupted training inputs.
Identifies a significant technical vulnerability in the foundation of current AI systems.
Shows how online misinformation campaigns can directly influence AI behavior.
- Computational Propaganda
- The use of algorithms, automation, and human curation to manipulate public opinion online.
- Data Poisoning
- A type of attack where malicious data is inserted into a training set to corrupt the model.
SecurityxAI can’t deny Grok makes CSAM anymore. So it’s suing users.
SecurityCould China and Russia really destroy Starlink? Only with a boomerang.
SecurityWhy teens deserve access to safe AI
Least privilege for AI agents: Identity, access, and tool binding - Microsoft
Iran Used Ad Tracking To Hunt American Soldiers: Report - Reason Magazine
QFireNet: A Quantum-Enhanced U-Net for Wildfire Segmentation from Sentinel-2 Imagery
Researchers introduce QFireNet, a quantum-enhanced U-Net model for wildfire segmentation from Sentinel-2 imagery, published on arXiv.
Organize your curiosity: Generative AI tools prove adept at structuring volumes of information - Editor and Publisher
Generative AI tools are effective in structuring large volumes of information, helping to organize and make sense of complex data. This development has significant implications for various industries and applications.
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
Researchers introduce C3R, a method to prevent incorrect domain evidence from polluting multi-domain retrieval tasks. It uses conformal risk control to ensure reliability without needing labels at query time.
TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories
Researchers introduced TEDDY, a 1.84‑million‑parameter transformer trained on 73 million ICD‑10 codes from 1.6 million children, to forecast future diagnoses and visit timing.
Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Models
Researchers propose SIRUS, a framework for suppressing target concepts in text-to-video models. This allows for more controlled video generation without retraining the model.
KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation
Researchers introduced KeyFrame-Compass, the first comprehensive benchmark designed to evaluate how faithfully video generation models can reproduce specific keyframes while maintaining overall video quality.