FAQ
What's the difference between LLM red teaming and prompt injection testing?
LLM red teaming is the broad discipline — end-to-end adversarial probing of an AI system covering the model, retrieval layer, agentic orchestration, output handling, and downstream actions. Prompt injection is one technique within that. A red-team engagement uses prompt injection alongside jailbreaking, model extraction, data leakage probing, tool-misuse attacks, multi-turn context manipulation, and agent-coordination attacks.
Short answer
LLM red teaming is the broad discipline — end-to-end adversarial probing of an AI system covering the model, retrieval layer, agentic orchestration, output handling, and downstream actions. Prompt injection is one technique within that. A red-team engagement uses prompt injection alongside jailbreaking, model extraction, data leakage probing, tool-misuse attacks, multi-turn context manipulation, and agent-coordination attacks.
The framing matters
Buyers sometimes scope "prompt injection testing" when what they actually want is broader. Conversely, sometimes they scope "LLM red teaming" when they have a narrow specific concern — for example, "do our guardrails catch known injection patterns" — that doesn't need a full red-team engagement.
A useful scoping decision tree:
- **"We want to know if our agent can be made to take harmful actions"** → red team engagement. Scope includes tool selection, action authorization, multi-step planning attacks. - **"We want to verify our system prompt and guardrails hold up against jailbreaks"** → focused prompt injection assessment. Scope is narrower, deliverable is a coverage report. - **"We want a formal compliance artifact aligned to OWASP LLM Top 10"** → compliance-flavored assessment that maps findings to the framework but doesn't go beyond it. - **"We want everything because we're launching a new agentic product"** → full red team plus model supply-chain assessment plus integration-layer review.
What we typically scope
For agentic AI systems (where the model can take actions through tools), the answer is almost always full red team. The single biggest class of finding we see is "prompt injection in retrieved context → unintended tool invocation → real-world side effect." A narrow prompt-injection-only scope misses this because the injection vector is in a third-party data source, not the user's input.
For non-agentic chat applications, a focused prompt injection assessment is often the right scope. The model can't take actions; the worst case is compromised output, which is mitigated by output filtering.
Related FAQs
- Do you test agentic AI systems? (P1.5)
Related services and research
- AI & ML System Security - Prompt-Injection Defense Architecture — Field Guidance
---