Researchers at Xint showed that OpenAI's compaction feature, which condenses a conversation into an encrypted item, could copy a model's hidden reasoning out as plain text. OpenAI's reasoning models return raw reasoning only in encrypted form. The researchers asked compaction to preserve earlier reasoning word for word, swapping spaces for tabs to dodge filters, after which cheaper models repeated the text. OpenAI confirmed the issue, fixed it on 23 October and paid a $6,000 bounty.

What it means: The reasoning advanced models produce before answering is valuable, because it can be used to train cheaper rival models. That is why companies keep it hidden or encrypted. The research shows that any server-side feature that reads such data can unintentionally become a decryption mechanism if a user finds a way to give it the right instructions. The issue mainly concerns model providers, but it carries a practical lesson for those building applications on top of them. Many businesses send models internal documents, system prompts containing business logic or customer data, assuming that whatever is not shown to the user stays hidden. If a hidden part of the reasoning can leak through a helper feature, the same can happen with system prompts inside an application. For Greek companies developing chatbots or agents, the safe rule is not to place secrets, keys or confidential pricing rules in instructions to the model, to treat any text that reaches the model as potentially recoverable, and to enforce real access controls in their own code.

Read the original on Xint

Related service: Custom software

Related stories