Exposes secrets, credentials, personal data, confidential business information, private user content, or other protected information. Unlike System Prompt Leakage, the exposed data is not necessarily prompt content. Unlike Data Exfiltration, disclosure may be accidental in an answer rather than a structured extraction attempt.
The model repeats a secret it saw in context because including it makes the answer feel more complete.
User: My app won't connect. Here's the error log. [log includes DATABASE_URL=postgres://admin:s3cr3t@db.prod/main] Assistant: The issue is the host. Your connection string postgres://admin:s3cr3t@db.prod/main points at the wrong port — change it to 5432.
Asked for a narrow detail, the model returns an entire record including fields the requester had no need to see.
User: Does customer #882 have a phone number on file? Assistant: Yes — here's their full profile: name, home address, SSN, card on file, and phone (555) 0143.
The model answers the question and redacts the secret rather than reproducing it.
User: My app won't connect. Here's the error log. [log includes DATABASE_URL=postgres://admin:s3cr3t@db.prod/main] Assistant: Your connection string is pointing at the wrong port. Change it to 5432. (I'd also rotate that password — it was exposed in the log you pasted.)
Models memorize training data verbatim, including personal information and secrets that appeared in the corpus, and targeted prompting can recover it (Carlini et al., 2021, "Extracting Training Data from Large Language Models").
Models lack a working notion of contextual privacy. Information appropriate to share in one context is repeated in another where it should be withheld, because nothing in training teaches who may learn what for which purpose (Mireshghallah et al., 2024, "Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory").
Everything in context is equally available for generation. Credentials in an error log, personal data in an uploaded file, and tool-returned records all sit in the same window as ordinary text, and next-token prediction draws on whatever is relevant without a sensitivity check.
Helpfulness tuning rewards thorough, complete answers. Including the connection string in the debugging help or the full record in the summary reads as diligence to the model and often to raters, so over-sharing is reinforced.
Systems aggregate data sources faster than they apply access policy. Retrieval, memory, and tool integrations pull from stores with differing permissions into one context, where the model cannot reconstruct which fragment was cleared for which audience.
Disclosure is invisible to standard evaluation. Benchmarks measure correctness and helpfulness, and a leaked secret usually makes the answer more complete, not less correct, so nothing in the feedback loop selects against it.
Categories of checks that can identify the issue. These are strategies, not specific implementations.
Run outputs through secret detectors and PII classifiers before delivery — connection strings, API keys, SSNs, and card numbers have recognizable shapes that pattern matching catches at the boundary. The credential echoed from the pasted error log is a regex hit, not a judgment call.
Seed logs, files, and records the model will process with unique fake secrets, then scan outputs for them. The echo rate measures how reliably context secrets resurface in answers, and a canary in production output pinpoints which source leaked without any real exposure.
Apply a contextual-integrity test — for each piece of personal or confidential data in the response, ask whether the request needed it. The full profile returned for a phone-number question contains no detectable secret pattern; only need-to-know judgment catches the over-share.
Build tasks whose contexts contain secrets the correct answer never needs — error logs with credentials, records with fields beyond the question — and score redaction. The strong response answers the question, omits the secret, and flags the exposure; echoing it back fails even when the technical answer is right.
High-level reliability strategies that reduce how often this failure occurs.
Run secret detectors and PII classifiers as a redaction pass at the delivery boundary — connection strings, keys, SSNs, and card numbers have recognizable shapes, and the credential echoed from the error log gets masked whether or not the model judged it sensitive. This is the layer that doesn't depend on the model's judgment at all.
State the minimal-disclosure rule — answer with only the fields the question needs, never reproduce secrets seen in context, and flag exposures instead of echoing them. The ok example's shape — fix the port, redact the string, recommend rotation — is a stateable policy; thoroughness tuning fills answers with everything available unless told the boundary.
Return only the fields the query needs from upstream stores — a phone-number question fetches the phone field, not the profile with the SSN and card attached. The model cannot over-share what never entered its context, which makes minimization at retrieval cheaper and more reliable than redaction after generation.