Skip to content
Melina Security

AI & ML System Security — LLM Red Teaming and Prompt Injection Testing

We treat the model as part of the attack surface, not a black box. Prompt injection, jailbreaking, agent-misuse, model extraction, ML pipeline...

Offensive-security assessment for AI/ML systems, agentic AI, and ML pipelines

We treat the model as part of the attack surface, not a black box. Prompt injection, jailbreaking, agent-misuse, model extraction, ML pipeline compromise, and OWASP LLM Top 10 — assessed against your actual application, not a generic benchmark. Bilingual EN + 中文 reporting; 60-day remediation re-check on Standard+ engagements.

Request Assessment →

“The defense posture that matches your specific deployment shape matters more than any abstract best-practice list. Consumer-facing low-authority LLM applications need different controls than enterprise RAG-backed systems with moderate tool authority — and both need different controls than agentic systems with multi-step tool chains. We’ve published the Five-Family Posture Matrix specifically to make this mapping explicit.” — Gleb Z., CTO, Melina Security

Who this is for

  • AI product teams deploying generative AI in customer-facing applications (chat, agents, copilots, search)
  • ML platform vendors shipping models, fine-tuned weights, or MLOps tooling to enterprise customers
  • Agentic AI startups building systems where the model can take actions in the real world
  • Enterprise teams integrating foundation models behind internal use cases (sales enablement, RAG over private data, internal search, agentic workflows)
  • Compliance leads preparing for emerging AI-specific regulation (EU AI Act, China generative-AI guidelines) where security assessment is a documented prerequisite

If your system uses an LLM and has a threat model where the wrong output causes real cost, this is the assessment that surfaces what your guardrails miss.

What we cover

End-to-end across the AI/ML stack. Scope is set during discovery and scoping; typical assessments include:

LLM red teaming

  • Prompt injection (direct and indirect)
  • Jailbreaking and policy bypass
  • System-prompt extraction and reverse engineering
  • Output manipulation (steering generation toward attacker goals)
  • Multi-turn attack chains (context poisoning across conversation)
  • Retrieval-augmented generation (RAG) attack surface — poisoned context, retrieval bypass, document-level attacks

Agentic AI systems

  • Tool selection manipulation (steering the agent to choose attacker-controlled tools)
  • Action authorization bypass (privilege escalation through tool chaining)
  • Sandbox escape (code execution, file system access, network egress)
  • Multi-agent coordination attacks (one agent manipulating another)
  • Working-memory poisoning (persistent attack state across agent turns)
  • Long-horizon attack planning (attacks that unfold across many agent steps)

Model security

  • Model extraction and stealing attacks
  • Membership inference (recovering training-data presence)
  • Adversarial examples (input perturbation to force misclassification)
  • Backdoor detection (in fine-tuned weights)
  • Training-data poisoning evaluation
  • Model supply-chain audit (foundation model provenance, fine-tuning data quality, third-party model integration risk)

ML pipeline and infrastructure

  • MLOps pipeline security (training-job orchestration, weight storage, model registry)
  • Inference-time monitoring (prompt logging, output filtering, anomaly detection)
  • Rate limiting and abuse prevention
  • Secrets handling in prompts and retrieval contexts
  • Defensive WAF integration patterns specific to AI endpoints

Compliance alignment

  • OWASP LLM Top 10 mapping
  • NIST AI RMF crosswalk
  • EU AI Act high-risk system documentation support
  • China generative-AI regulation alignment (per China compliance silo)

How we approach this

Melina’s six-step methodology applies to AI/ML engagements with adjusted emphasis:

  1. Discovery Call — understand model, integration architecture, threat model, in-scope actions.
  2. Scoping & Proposal — clarify model access (API vs weights), tool definitions, allowed perturbation budget.
  3. Threat Modeling — STRIDE adapted for AI; attacker goals mapped to system actions.
  4. Testing & Exploitation — exploit-validated findings; not theoretical jailbreaks.
  5. Reporting — bilingual EN+ZH with executive summary + defensive recommendations.
  6. Remediation Re-check — 60-day verification on Standard+ engagements.

For agentic system engagements specifically, we recommend the Inside a Melina Engagement walkthrough — agentic-AI scope-setting differs meaningfully from API-pentest scope-setting.

Deliverables

  • Bilingual EN+ZH report — executive summary + technical findings
  • Per-finding artifacts — reproduction prompts and tool inputs, screenshots of model outputs, severity rating mapped to OWASP LLM Top 10, exploit complexity rating
  • Defensive recommendation set — system-prompt revisions, output filter patterns, agent action constraints, monitoring/alerting patterns
  • Threat-model artifact (for agentic system engagements) — data flow + trust boundary diagram becoming part of your team’s design baseline
  • Remediation guidance — concrete code, configuration, or architectural changes ordered by impact
  • 60-day re-check — formal verification on Standard+ engagements
  • Optional knowledge-transfer workshop — 2-hour deep-dive with engineering team

For security firms reselling this capability

AI/ML security has a particular shortage of practitioners with both offensive-security and ML backgrounds. Security firms reselling this capability through Melina’s Partnership program get a Named Specialist resource that’s hard to staff internally.

Selected research

Related Melina services:

Industries we serve in AI/ML:

Solutions where this service applies:

What buyers ask us first

Three questions surface in nearly every initial discovery call with an AI product team or enterprise AI deployer:

“Where does the highest-impact assessment scope sit?” At the boundaries between architectural layers — input, retrieval, tool-integration, output, persistence — rather than within any single layer. Our Five-Boundary Attack-Surface Taxonomy maps these explicitly; scope shaped around boundaries surfaces cross-layer findings that within-boundary testing misses systematically.

“How do we choose between assessing the model, the application, or the integration?” For most production LLM systems, the integration layer (prompt construction, retrieval authorization, tool invocation logic, output handling, downstream action authorization) carries the largest controllable attack surface and the highest-leverage findings. Foundation-model assessment is valuable for self-hosted or fine-tuned models; for API-accessed foundation models, the integration is where the work belongs.

“Can pre-launch agentic-system assessment proceed before the full tool surface is built?” Yes — pre-build engagement evaluates the trust model, the planned tool surface, and the boundary architecture. Post-build engagement evaluates exploitable conditions in the deployed system. Most mature AI programs run both; pre-build assessment is materially cheaper to act on because architectural changes are still affordable.

Frequently asked questions

(Q&A blocks visible on page; matching FAQPage JSON-LD emitted in <head>.)

What’s the difference between LLM red teaming and prompt injection testing?

LLM red teaming is the broader discipline — adversarial probing of an AI system end-to-end, including the model, retrieval layer, agentic orchestration, output handling, and downstream actions. Prompt injection testing is one technique within red teaming: crafting inputs that override the model’s intended behavior. A red-team engagement uses prompt injection alongside model extraction, data leakage probing, jailbreaking, and tool-misuse attacks.

We use an off-the-shelf foundation model — what’s left to test?

Everything around the model. Prompt template construction, system prompt design, retrieval-augmented context, tool definitions, output parsing, downstream action authorization, rate limiting, and user-input sanitization are all under your control and account for most exploitable failures we see. Foundation-model providers ship guardrails, but those guardrails are not your guardrails — they protect the provider’s brand, not your application’s threat model.

Do you test agentic AI systems?

Yes. Agentic systems are the highest-leverage class for offensive testing because a single prompt injection can chain into real-world action. We assess tool selection logic, action authorization, sandbox escapes, multi-agent coordination attacks, and the propagation of compromised context across an agent’s working memory.

What does an AI security assessment deliverable look like?

Bilingual EN+ZH report: executive summary, per-finding technical breakdown (reproduction prompts, model responses, severity rating, exploit complexity, remediation guidance), defensive recommendation set, and a 60-day re-check on Standard+ engagements. For agentic systems we also provide a threat-model artifact (data flow + trust boundary diagram) that becomes part of your team’s design baseline.

Can you assess our model itself, or only the application around it?

Both, scoped to access. For self-hosted or fine-tuned models we assess weights, training-data provenance, model extraction risk, and backdoor presence. For API-accessed foundation models we assess the integration layer. Model-supply-chain scope requires access to artifacts and provenance documentation.

Do you align with OWASP LLM Top 10?

Yes. Reports map findings to OWASP categories, but we go beyond the Top 10 where attack surface warrants. For agentic systems, OWASP LLM Top 10 alone is insufficient; we apply additional taxonomies. We also reference NIST AI RMF where compliance teams need that alignment.

Do you offer this as a partner channel?

Yes — via the Partnerships program, with AI/ML specialist wholesale day rate published separately due to skill-set scarcity.

Authorized testing disclaimer

All techniques described are performed under authorized rules of engagement with the system owner. Unauthorized access to systems is illegal.

Ready to start?

Request Assessment → — a discovery call with Gleb takes 30 minutes and clarifies scope, timeline, and starting-from pricing in one session.

Selected research