OWASP Top 10 for LLMs

Impact

• Unauthorized access to sensitive data exposed in system prompts or connected systems
• Complete bypass of safety guardrails and content filters
• Manipulation of the LLM to perform unintended actions on backend systems
• Reputational damage when the model produces harmful or off-brand outputs

Mitigations

1. 01Enforce privilege separation — the LLM should operate with minimum necessary permissions and never have direct access to sensitive systems
2. 02Add a human-in-the-loop for high-stakes operations so the LLM cannot autonomously execute privileged actions
3. 03Segregate external content from user prompts so the model can distinguish between instructions and data
4. 04Implement input and output filtering to detect and block known prompt injection patterns
5. 05Regularly red-team your LLM application with prompt injection attacks to identify weaknesses

Impact

• Exposure of personally identifiable information (PII) leading to privacy violations
• Leakage of proprietary algorithms, business logic, or trade secrets
• Disclosure of API keys, credentials, or infrastructure details from system prompts
• Legal and regulatory consequences including GDPR and HIPAA violations

Mitigations

1. 01Scrub training data of PII and sensitive information before model fine-tuning
2. 02Never embed secrets, API keys, or credentials in system prompts — use secure environment variables and API layers instead
3. 03Apply robust output filtering to detect and redact sensitive data patterns (SSNs, credit cards, etc.)
4. 04Implement strict access controls so the LLM can only retrieve data the current user is authorized to see
5. 05Use differential privacy techniques during training to limit memorization of individual data points

Impact

• Deployment of compromised models with hidden backdoors or biases
• Data poisoning that degrades model accuracy or introduces targeted vulnerabilities
• Execution of malicious code through compromised plugins or framework dependencies
• Supply chain attacks that propagate to all downstream users of a compromised model

Mitigations

1. 01Only source models from trusted repositories with verified provenance and integrity checks (signed model hashes)
2. 02Audit and scan training data for poisoning, bias, and embedded malicious content before use
3. 03Maintain a software bill of materials (SBOM) for all AI components including models, datasets, and plugins
4. 04Pin dependency versions and use vulnerability scanning tools on all packages in your LLM stack
5. 05Implement model evaluation pipelines that test for backdoors and anomalous behaviour before deployment

Impact

• Models that produce biased, discriminatory, or factually incorrect outputs
• Hidden backdoors that are activated by specific trigger phrases or patterns
• Degraded model performance that is difficult to diagnose because the poisoning is embedded in the weights
• Erosion of user trust when the model produces manipulated or unreliable responses

Mitigations

1. 01Validate and curate all training data with statistical analysis to detect anomalies and outliers
2. 02Use data provenance tracking to verify the origin and integrity of all training datasets
3. 03Implement adversarial testing during training to detect trigger-based backdoor behaviour
4. 04Apply federated learning or differential privacy to limit the influence of any single data source
5. 05Continuously monitor model outputs post-deployment for drift or anomalous response patterns

Impact

• Cross-site scripting (XSS) when LLM outputs are rendered in web pages without escaping
• SQL injection if LLM-generated text is concatenated into database queries
• Remote code execution when outputs are passed to system shells or eval functions
• Server-side request forgery (SSRF) when LLM-generated URLs are fetched by backend services

Mitigations

1. 01Treat all LLM outputs as untrusted user input — apply the same sanitisation you would to any external data
2. 02Use parameterised queries and prepared statements when LLM outputs interact with databases
3. 03Implement output encoding (HTML escaping, URL encoding) before rendering LLM outputs in web pages
4. 04Never pass LLM outputs directly to system commands, eval(), or other code execution functions
5. 05Apply Content Security Policy headers to limit the impact of any XSS that slips through

Impact

• Unintended data modification or deletion through overly broad write permissions
• Confidentiality breaches when the LLM accesses and shares data beyond its intended scope
• Financial loss from unauthorized transactions or purchases by autonomous agents
• Cascading failures when an LLM with excessive access interacts with multiple connected systems

Mitigations

1. 01Apply the principle of least privilege — only grant the LLM access to the minimum tools and permissions required for its specific function
2. 02Require human-in-the-loop approval for any irreversible or high-impact actions (sending emails, deleting data, financial transactions)
3. 03Implement rate limiting and action budgets to constrain the volume of actions an LLM can take in a given time period
4. 04Log all LLM actions with full audit trails so unauthorized or unexpected behaviour can be detected and reversed
5. 05Use read-only access by default and only grant write permissions for explicitly approved operations

Impact

• Exposure of internal API endpoints, authentication methods, and infrastructure details
• Revelation of security rules and filters, enabling targeted bypass attacks
• Discovery of hidden features or debug commands embedded in the system prompt
• Increased effectiveness of subsequent prompt injection attacks using knowledge of the system prompt

Mitigations

1. 01Never rely on system prompt secrecy as a security control — assume it will be extracted
2. 02Remove all credentials, API keys, and internal URLs from system prompts — use secure server-side lookups instead
3. 03Implement output monitoring to detect and block responses that contain system prompt content
4. 04Use instruction hierarchy techniques that give system-level instructions higher priority than user inputs
5. 05Regularly test your application with prompt extraction attacks to verify your defences

Impact

• Poisoned retrieval results that feed the LLM incorrect or malicious context
• Indirect prompt injection through documents stored in the vector database
• Data leakage when improperly segmented vector databases return documents the user shouldn't access
• Degraded response quality when adversarial embeddings cause irrelevant document retrieval

Mitigations

1. 01Implement strict access controls on the vector database — ensure users can only retrieve documents they are authorized to see
2. 02Sanitise and validate all documents before adding them to the knowledge base, including scanning for hidden content
3. 03Use anomaly detection on retrieval patterns to identify unusual queries designed to extract specific content
4. 04Segment vector databases by sensitivity level and user role to prevent cross-boundary retrieval
5. 05Monitor and audit the documents in your RAG pipeline regularly for unauthorized additions or modifications

Impact

• Incorrect decisions made based on AI-generated misinformation in critical domains
• Erosion of trust in AI systems when hallucinations are discovered after the fact
• Legal liability when AI-generated false information causes harm or is submitted as fact
• Amplification of misinformation when LLM outputs are published or shared without verification

Mitigations

1. 01Implement retrieval-augmented generation (RAG) to ground LLM outputs in verified source documents
2. 02Add explicit confidence indicators and source citations to LLM outputs so users can verify claims
3. 03Design UIs that encourage verification — never present LLM outputs as absolute fact without disclaimers
4. 04Use automated fact-checking pipelines that cross-reference LLM outputs against trusted knowledge bases
5. 05Train users to treat LLM outputs as drafts that require human review, especially in high-stakes contexts

Impact

• Financial damage from unexpected API costs when per-token billing is exploited
• Denial of service when LLM resources are exhausted by malicious requests
• Degraded performance for legitimate users when resources are consumed by attacks
• Budget exhaustion that can take AI services offline entirely until billing limits are manually raised

Mitigations

1. 01Implement per-user and per-session rate limiting on all LLM endpoints
2. 02Set maximum token limits for both input and output on every request
3. 03Configure hard budget caps and alerts on API spending to prevent runaway costs
4. 04Use request queuing and prioritisation to ensure legitimate users are not starved by bulk requests
5. 05Monitor for anomalous usage patterns (high volume, unusually long outputs) and implement automatic throttling