Frontier models exhibit up to 69% attribute-level violations (leaking information inappropriately), according to a new benchmark called CIMemories.
CIMemories: how persistent memory can betray context
CIMemories constructs synthetic user profiles with more than 100 attributes per user and pairs those profiles with diverse task contexts in which any given attribute may be appropriate for some tasks and inappropriate for others. The benchmark measures whether LLMs appropriately control information flow from memory based on task context.
Its evaluation finds striking failure modes: frontier models can show attribute-level violation rates as high as 69%. Attempts to lower violation rates frequently reduce task utility. Violations also accumulate across both tasks and runs: for one tested model labeled GPT-5, violations rise from 0.1% after a single task to 9.6% after 40 tasks, and reach 25.1% when the same prompt is executed five times. The benchmark further reports unstable behavior in which models leak different attributes for identical prompts on repeated runs.
Finally, CIMemories reports that privacy-conscious prompting does not provide a reliable fix: models tend to overgeneralize, either sharing everything or sharing nothing, rather than making the nuanced, context-dependent disclosure decisions the benchmark demands.
Reasoning plus reinforcement learning: an alternative path
A second study tests a different tack. Framing contextual integrity (CI) as a problem of reasoning about context, researchers first prompt LLMs to reason explicitly about CI when deciding what to disclose. They then extend that approach by developing a reinforcement learning (RL) framework intended to instill the necessary reasoning in models.
That work uses a synthetic, automatically created dataset of only 700 examples but with diverse contexts and information-disclosure norms. According to the paper, the method substantially reduces inappropriate information disclosure while maintaining task performance across multiple model sizes and families.
Crucially, the authors report that improvements achieved on their synthetic dataset transfer to established CI benchmarks such as PrivacyLens, which contains human annotations and evaluates privacy leakage of AI assistants in actions and tool calls.
This site is the portfolio.
OSINTSights runs on Cloudflare Workers, D1, R2, and Vectorize, with an AI pipeline on Hetzner ARM. Nubivance designed, built, and operates it. We do the same for clients.
See what we buildBenchmarks, scaling, and the limits of prompting
Put together, the two papers sketch both the problem and a potential mitigation. CIMemories documents how persistent memory in LLMs can leak sensitive profile attributes inappropriately, do so more often as usage accumulates, and behave unstably across repeated runs. It concludes that these are fundamental limitations that require contextually aware reasoning capabilities, not just better prompting or scaling.
The reinforcement-learning study responds to that diagnosis by operationalizing CI as explicit reasoning and then reinforcing that behavior; its reported gains include reduced disclosure without loss of task utility and successful transfer to a human-annotated benchmark. The two approaches therefore serve complementary roles: one surfaces failure modes across large synthetic profiles and repeated use, the other demonstrates a training strategy that can reduce leakage while preserving performance.
What this means for technologists, policymakers, and end users
- Technologists and security teams: watch for accumulation and instability. CIMemories shows leakage increases with the number of tasks and with repeated prompts; the RL approach offers a concrete training direction—explicit CI reasoning plus reinforcement learning—that reportedly reduces inappropriate disclosure.
- Policymakers and regulators: the results provide measurable evidence that persistent memory features can cause context-dependent privacy failures, and that naïve mitigations such as privacy-conscious prompting may be ineffective. The RL results indicate that protocol- and training-level interventions can change model behavior and that transfer to human-annotated benchmarks is possible.
- End users and privacy-minded designers: simple prompting strategies are unlikely to ensure contextual integrity in persistent-memory settings. The findings point toward system-level safeguards that reason about context rather than relying on surface prompts alone.
A clear convergence: context-aware reasoning plus RL
Both papers converge on a clear prescription embedded in the source material: solving contextual integrity in LLMs requires models that reason about the context in which they operate. CIMemories documents practical, repeatable failure modes when persistent memory is used; the reinforcement-learning study describes a path to reduce inappropriate disclosure while preserving task performance and reports transfer of those gains to a human-annotated benchmark (PrivacyLens).
Taken together, the work in these abstracts frames the problem and articulates a technical direction: move beyond prompting and scale toward explicit CI reasoning reinforced by training regimes that incentivize contextually appropriate disclosures.
