Researchers cracked the 'encrypted' thinking box in ChatGPT, Claude, and Gemini — and pulled 182 real passwords and API keys out of public logs with it
A new paper shows the hidden reasoning traces behind major AI models use one shared encryption key, not a private one per user. Anyone who's pasted a real login into an agent — and anyone who's ever shared a session log for support or a demo — should assume that box wasn't private.
A paper published this week shows that the hidden "thinking" traces behind ChatGPT, Claude, and Gemini aren't as private as their encrypted, collapsed-by-default appearance suggests. Researchers decoded over 315,000 of these reasoning blocks scraped from public logs and pulled out 182 real credentials that users had typed into their prompts, assuming that part of the conversation was sealed off.
What did the researchers actually find?
Every major reasoning API — OpenAI's, Anthropic's, and Google's — hides the model's step-by-step "thinking" from the user by encrypting it into an opaque block, then has the client hand that block back on the next request. The point of the encryption is to protect the provider's IP, not to protect the user. Researchers from ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems, MATS Research, and Snyk found the mechanism has a structural flaw: each provider uses one global encryption key across every user, session, and model in its lineup, rather than a key tied to the individual session. That makes an encrypted block portable — a trace produced in one conversation can be fed into a different, weaker model from the same provider and forced to decode and repeat it back in plain text.
To prove it, they scraped 6,708 publicly posted AI agent session logs — the kind developers paste into GitHub issues, support tickets, and blog posts when debugging an agent — and decoded 315,320 hidden reasoning blocks. Inside them: 62 API keys, 33 passwords, and 367 other pieces of personal information that the people who posted those logs believed were hidden. The paper's authors say all three providers were notified before publication and updated their APIs, but anything already sitting in a public log stays decodable with the old technique.
Why does a developer-security paper matter to a construction back office?
Because the underlying habit it exposes — pasting a real login into an AI agent and trusting the "encrypted" part to stay private — is exactly the workflow this publication has been tracking take hold in construction over the past few weeks. GCs and subs are increasingly running browser-driving agents against bid portals, insurance sites, and accounting platforms with real credentials in the prompt, on the assumption that anything the model marks as hidden reasoning is sealed. This paper says that assumption doesn't hold once that session log leaves your control.
| Where it shows up | The exposure | What to check |
|---|---|---|
| Agent logged into a bid, insurance, or accounting portal | Credential sits inside that session's reasoning trace | Use a scoped or temporary login, never the controller's or the estimator's daily-driver password |
| Support ticket sent to an AI vendor after an agent misbehaves | Full session, including "hidden" reasoning, often gets pasted in for debugging | Strip credentials from the conversation before sharing, don't assume the collapsed box redacts itself |
| Vendor demo or case study showing your firm's AI agent in action | Screen recordings and exported transcripts can include the reasoning panel | Treat any exported or recorded agent session as public the moment it leaves your systems |
Has this actually been fixed?
Partially, and only for new sessions. The three providers changed how their APIs handle reasoning traces after the researchers' private disclosure, which should stop new sessions from being replayable the same way. It does nothing for logs, tickets, or demo recordings already posted publicly before the fix — those remain decodable by anyone who reads the paper's method. If your firm has ever shared an agent transcript externally for support or marketing, that transcript is the thing to go back and check, not tomorrow's session.
What should you actually do this week?
- Stop typing real, unscoped credentials into agent prompts. If an agent needs to log into a portal, give it a scoped or temporary login built for that purpose, not the one your controller uses everywhere else.
- Audit anything you've already shared. Support tickets, vendor case studies, and demo recordings that include an agent's session — including the collapsed reasoning section — should be treated as exposed, not private.
- Ask any vendor selling you an agent-driven portal tool whether they've applied the provider-side fix and whether any of your firm's session data was in a log they shared for debugging.
Forward this to whoever's been handing agent logins to your estimators or PMs — not just the person who reads AI research papers.
Construction AI Brief publishes new coverage on AI's construction stakes multiple times a week. Subscribe at constructionaibrief.com.
- Is the 'thinking' or reasoning box in ChatGPT, Claude, and Gemini actually private?
- Not as private as it looks. A paper published August 10, 2026 by researchers from ELLIS Institute Tübingen, the Max Planck Institute, MATS Research, and Snyk found that OpenAI, Anthropic, and Google each encrypt hidden reasoning traces with a single key shared across all users, sessions, and models on that provider — not a per-user or per-session key. That means a trace isn't cryptographically tied to the person who generated it.
- What did the researchers actually recover?
- Scraping 6,708 publicly posted AI agent session logs, they decoded 315,320 hidden reasoning blocks and pulled out 182 real credentials — 62 API keys and 33 passwords — plus 367 pieces of personal information, all from traces the platforms had marked as encrypted and hidden from view.
- Does this affect construction firms using AI agents to log into portals?
- Yes, indirectly but concretely. Any firm running an agent against a bid portal, insurance site, or accounting platform with a pasted-in login has that credential sitting somewhere in the model's reasoning trace for that session. The exposure isn't that a stranger can hack your session directly — it's that if that session log ever gets shared, screen-recorded, or posted (a support ticket, a vendor demo, a conference case study), the 'hidden' portion isn't actually redacted.
- Have OpenAI, Anthropic, and Google fixed this?
- The researchers disclosed the flaw to all three providers before publishing, and each updated their APIs afterward, per multiple reports covering the disclosure. Reasoning traces already sitting in previously published logs, support threads, or GitHub issues remain decodable with the old method — a fix applied today doesn't retroactively re-encrypt what's already public.
- What should a construction back office do about this right now?
- Stop pasting real, unscoped credentials into any AI agent prompt — use a scoped or temporary login for portal-automation tasks instead. And treat any AI agent transcript your team shares externally, including the collapsed 'thinking' section, as public information, not a private debug log.