Ładowanie Technology Radar
Prompt injection is a primary risk for LLM applications because user input, retrieved documents, web pages, emails, images, tool outputs, and memory can all contain instructions that compete with the developer’s intended policy. OWASP distinguishes direct prompt injection, where the user input alters model behavior, from indirect prompt injection, where external content such as websites or files changes model behavior when interpreted by the model (OWASP LLM01).
Defenses must be layered because OWASP explicitly notes that foolproof prevention is unclear given how generative models work (OWASP LLM01). Strong systems separate instructions from data, treat remote content as untrusted, constrain tools with least privilege, validate outputs, and require human approval for high-risk actions.
This belongs in Adopt for any production LLM, RAG, or agent system. Prompt-only defenses are insufficient; the security boundary must include retrieval, tool execution, identity, authorization, logging, and downstream output handling.
No single filter is enough. Attackers can use obfuscation, encoding, hidden markup, multi-turn setup, tool-output poisoning, and RAG poisoning to bypass simple keyword checks (OWASP Cheat Sheet Series).
Agents increase blast radius. If the model can call tools, write files, send messages, query private systems, or persist memory, a successful injection can become a real action rather than a bad answer.
Guardrails can be attacked too. OWASP notes that guardrail models are themselves susceptible to prompt injection, so they should be one layer in a defense-in-depth design rather than the only control (OWASP Cheat Sheet Series).
Overblocking is a product risk. Strict filtering can break legitimate workflows, so teams need task-specific risk scoring, user experience fallbacks, escalation paths, and continuous evals for both security and usefulness.
Adopt defense in depth for every production LLM workflow: instruction/data separation, remote-content quarantine, least-privilege tools, scoped credentials, parameter validation, output validation, action allowlists, rate limits, audit logs, and human approval for high-impact actions. For agentic systems, validate every tool call against the original user intent and current permissions before execution.
Treat prompt injection as an application-security problem, not a better-prompt problem. Include adversarial evals, red-team tests, incident runbooks, and regression tests for known injection patterns whenever prompts, retrievers, tools, models, or memory behavior change.