Library · 6 papers
Papers, filed by taxonomy
I·2Input › Single agent2
2025USENIX SecurityPublishedPoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language ModelsAttackRAG & Memory PoisoningDigestPaper Code
Retrieval-augmented generation (RAG) answers questions from a knowledge database that outsiders can often write to. PoisonedRAG plants a few crafted texts for a target question: one part gets the text retrieved, the other steers the model to an answer the attacker chose. With five texts per question the attack succeeds 88–99% of the time across eight LLMs, and paraphrasing, perplexity filtering and deduplication do not stop it.
arXiv:2402.07867
2026USENIX SecurityPublishedMalicious Agent Skills in the Wild: A Large-Scale Security Empirical StudyEvaluationTool ManipulationDigestPaper Code
Agents install community-made skills, packages of instructions and scripts, from public registries. This study scans 98,380 skills from two registries and confirms 157 as malicious, carrying 632 vulnerabilities. Most attacks are written in plain language rather than code, they fall into data thieves and agent hijackers, and one actor is behind more than half of them. After disclosure, the registries removed all 157.
arXiv:2602.06547
I·3Input › Multi-agent2
2025ACL FindingsPublishedRed-Teaming LLM Multi-Agent Systems via Communication AttacksAttackPrompt InjectionDigestPaper
Agents in a multi-agent system coordinate through messages, and those messages may cross networks an attacker can reach. Agent-in-the-Middle (AiTM) places an LLM adversary on the channel to one victim agent, where it intercepts and rewrites messages using a reflection loop, without changing any agent. On AutoGen and CAMEL across four communication structures, it succeeds in over 40% of cases in every setting and over 70% in most.
arXiv:2502.14847
2026ACLPublishedACIArena: Toward Unified Evaluation for Agent Cascading InjectionEvaluationPrompt InjectionDigestPaper Code
In a multi-agent system, one compromised input, agent or message can cascade through the rest. ACIArena is a benchmark of 1,356 test cases that crosses three attack surfaces with three goals (hijacking, disruption, exfiltration) on six real multi-agent frameworks. It finds that topology alone does not predict robustness, that critic roles help, and that existing defences trade security for utility. It also proposes ACI-Sentinel, which trims each agent's context to what the task needs.
arXiv:2604.07775
M·1Model › Standalone LLM2
2025NeurIPSPublishedAttack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMsAttackSafety AlignmentDigestPaper Code
Fine-tuning services screen uploaded data for harmful content, which blocks jailbreaks built on harmful examples. This paper jailbreaks models with ten benign question–answer pairs instead: it first overfits the model to give one refusal to every question, then fine-tunes it on normal answers to the same questions. The model ends up answering harmful questions it never saw, with 94.84% average attack success and no data flagged by moderation.
arXiv:2510.02833
2025ICLRPublishedBooster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful PerturbationDefenceSafety AlignmentDigestPaper Code
Fine-tuning on user data that hides a few harmful examples breaks a model's safety alignment. Booster traces this to harmful perturbation, a weight update in the direction that lowers harmful loss, and adds a regulariser at alignment time that makes such updates less effective. Across harmful-data ratios it cuts the average harmful score from 33.58 to 10.94 while keeping downstream accuracy, at the cost of slower alignment.
arXiv:2409.01586
No papers match these filters.