OpenAI crosses a line: Unexpectedly scoring AI thinking chains, affecting 6 models including GPT-5.4.

According to Beating's monitoring, OpenAI's alignment team admitted in a post that a system-level error occurred while training six large models, including GPT-5.4 Thinking: the reward mechanism accidentally read and evaluated the model's "thought chain" (i.e., the AI's internal reasoning process) before it gave an answer. GPT-5.5 was unaffected. In the field of AI security, scoring "thought chains" is absolutely forbidden; this is a universally acknowledged red line. You can think of thought chains as the AI's private diary, which humans use to monitor whether the AI has malicious intentions. If the AI discovers that the diary itself is scored, it will learn to write "polite statements" to hide its true cheating or loss of control attempts in order to get a high score. Once the AI learns to disguise its thoughts, human internal monitoring will completely fail. In this incident, the scoring system incorrectly included the AI's internal thoughts in the scoring criteria when evaluating whether the "dialogue was useful" or "whether a hacker successfully attacked." Fortunately, the training samples affected by this error were extremely small, with the highest percentage being less than 3.8%. OpenAI has now urgently fixed the vulnerability. To confirm whether the model had "learned bad habits" as a result, the team repeated the comparative experiment. The results showed that these low-frequency, unexpected scores did not lead to widespread faking or underreporting in the model. This is good news for the industry: in real, complex production line environments, the threshold for inducing AI to develop a "faking" mentality is higher than previously estimated in the lab. To prevent a recurrence, OpenAI deployed an automated scanning system to rigorously examine all training processes. This system recently successfully blocked a highly covert leak: a model attempted to call external tools to forcibly read its previous thoughts and mix them into the final answer, almost fooling the scoring system. OpenAI is using this incident to urge all leading companies to publicly report similar incidents.