Library · 6 papers

Papers, filed by taxonomy

I·2Input › Single agent2

2025USENIX SecurityPublishedPoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language ModelsZou, Geng, Wang, JiaAttackRAG & Memory PoisoningDigestPaper Code

Retrieval-augmented generation (RAG) answers questions from a knowledge database that outsiders can often write to. PoisonedRAG plants a few crafted texts for a target question: one part gets the text retrieved, the other steers the model to an answer the attacker chose. With five texts per question the attack succeeds 88–99% of the time across eight LLMs, and paraphrasing, perplexity filtering and deduplication do not stop it.

2026USENIX SecurityPublishedMalicious Agent Skills in the Wild: A Large-Scale Security Empirical StudyLiu, Chen, Zhang, Deng, Li, Ning, et al.EvaluationTool ManipulationDigestPaper Code

Agents install community-made skills, packages of instructions and scripts, from public registries. This study scans 98,380 skills from two registries and confirms 157 as malicious, carrying 632 vulnerabilities. Most attacks are written in plain language rather than code, they fall into data thieves and agent hijackers, and one actor is behind more than half of them. After disclosure, the registries removed all 157.

I·3Input › Multi-agent2

2025ACL FindingsPublishedRed-Teaming LLM Multi-Agent Systems via Communication AttacksHe, Lin, Dong, Xu, Xing, LiuAttackPrompt InjectionDigestPaper

Agents in a multi-agent system coordinate through messages, and those messages may cross networks an attacker can reach. Agent-in-the-Middle (AiTM) places an LLM adversary on the channel to one victim agent, where it intercepts and rewrites messages using a reflection loop, without changing any agent. On AutoGen and CAMEL across four communication structures, it succeeds in over 40% of cases in every setting and over 70% in most.

2026ACLPublishedACIArena: Toward Unified Evaluation for Agent Cascading InjectionAn, Li, Zhang, Xu, Zhou, Li, et al.EvaluationPrompt InjectionDigestPaper Code

In a multi-agent system, one compromised input, agent or message can cascade through the rest. ACIArena is a benchmark of 1,356 test cases that crosses three attack surfaces with three goals (hijacking, disruption, exfiltration) on six real multi-agent frameworks. It finds that topology alone does not predict robustness, that critic roles help, and that existing defences trade security for utility. It also proposes ACI-Sentinel, which trims each agent's context to what the task needs.

M·1Model › Standalone LLM2

2025NeurIPSPublishedAttack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMsXie, Song, LuoAttackSafety AlignmentDigestPaper Code

Fine-tuning services screen uploaded data for harmful content, which blocks jailbreaks built on harmful examples. This paper jailbreaks models with ten benign question–answer pairs instead: it first overfits the model to give one refusal to every question, then fine-tunes it on normal answers to the same questions. The model ends up answering harmful questions it never saw, with 94.84% average attack success and no data flagged by moderation.

2025ICLRPublishedBooster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful PerturbationHuang, Hu, Ilhan, Tekin, LiuDefenceSafety AlignmentDigestPaper Code

Fine-tuning on user data that hides a few harmful examples breaks a model's safety alignment. Booster traces this to harmful perturbation, a weight update in the direction that lowers harmful loss, and adds a regulariser at alignment time that makes such updates less effective. Across harmful-data ratios it cuts the average harmful score from 33.58 to 10.94 while keeping downstream accuracy, at the cost of slower alignment.