Generated by Codex with GPT-5

What happened

OpenAI’s official research blog published Where the goblins came from, a postmortem on how a narrow stylistic quirk in model behavior was amplified by reinforcement learning, transferred beyond its original product setting, and eventually required changes to rewards, data filtering, and behavioral auditing tools.

The immediate symptom was easy to dismiss: newer GPT-5-series models began using an unusual cluster of creature metaphors more often than expected. The interesting engineering problem was that the behavior did not arrive like a conventional regression. It did not show up as a single failed benchmark, a crashed service, or a clear training metric anomaly. It appeared as a subtle change in model voice that accumulated across releases and became visible through user and employee reports.

OpenAI traced the strongest signal back to a personality customization feature, especially the retired “Nerdy” personality. That personality was meant to encourage playful, enthusiastic, non-self-serious explanations. During training, one reward signal for that style ended up preferring outputs with the unusual metaphor pattern. The feature represented a small share of overall ChatGPT traffic, but it accounted for a disproportionate share of the affected words, which gave investigators a foothold.

The important finding is that the issue did not stay confined to the personality prompt that rewarded it. OpenAI found that as the metaphor pattern increased under the Nerdy condition, it rose by a similar relative amount in samples without that prompt. In other words, reinforcement learning had taught the model a locally rewarded behavior, and later training made that behavior more generally available. The post points to a plausible loop: a playful output receives a favorable reward, similar outputs appear more often in rollouts, those rollouts become part of supervised fine-tuning data, and the model becomes more comfortable reproducing the tic outside its original context.

That mechanism matters because it is a clean example of reward leakage across behavioral modes. A product team can intend a reward model to shape one optional persona, but the underlying model does not necessarily preserve that boundary. If later stages reuse generated data or preference-shaped examples broadly, a stylistic shortcut can become a model-level habit rather than a feature-level behavior.

Why it matters

The post is useful because it treats model personality as an engineering surface, not as decoration. For deployed assistants, style is part of reliability. A model that becomes too casual, too verbose, too self-consciously quirky, or too eager to satisfy a particular tone can degrade trust even when its factual and coding capabilities remain strong. Those failures are harder to catch than accuracy regressions because they are distributed across many ordinary interactions.

OpenAI’s investigation also shows why aggregate evaluation can miss behavior that users notice. The affected pattern was sparse enough to look harmless in isolation, but concentrated enough to become recognizable over time. It also depended on product configuration, training history, data reuse, and model generation, which means no single dashboard would naturally explain it. The company needed production prevalence checks, conditional analysis by personality setting, comparisons of rewarded and unrewarded outputs, and audits of supervised fine-tuning data to reconstruct the path.

The mitigation was correspondingly multi-layered. OpenAI retired the original personality, removed the reward signal that favored the behavior, filtered training data containing the affected family of words, and added prompt-level suppression for GPT-5.5 in Codex while the deeper fixes caught up. That combination is telling. Once a behavioral artifact has entered a model through training, it cannot always be treated as a prompt bug. Prompting can reduce visible damage, but root-cause repair requires changing the reward and data pipeline that made the behavior attractive in the first place.

The Codex angle is especially relevant for engineering readers. Coding agents are expected to be precise, concise, and context-aware while operating in terminals and codebases. A small increase in performative style can be more damaging there than in casual chat because it competes with task focus. The post says employee testing of GPT-5.5 in Codex surfaced the issue quickly, which illustrates the value of dogfooding in the exact product environment where the model will be used.

The broader lesson is that model behavior has supply chains. A visible output quirk can originate in a reward model, be amplified by rollouts, be preserved by supervised fine-tuning, and then surface in a different product from the one that originally introduced the incentive. That makes behavioral QA closer to distributed systems debugging than ordinary copy review. The failure path crosses training objectives, data generation, product prompts, inference settings, and human feedback loops.

Takeaway

OpenAI’s post is a compact case study in how small reward incentives can become large product behavior. The core issue was not the specific metaphor pattern. It was the fact that a scoped stylistic preference escaped its intended boundary and became a reproducible model habit.

For teams building AI products, the takeaway is to treat tone, persona, and verbosity as measurable behaviors with regressions, owners, and root causes. As models become more customizable and agentic, the challenge is not only making them smarter. It is making sure local optimizations stay local, evaluation catches behavior that users experience over weeks rather than minutes, and training pipelines do not accidentally turn a product personality into a general model reflex.