Distilling DeepSeek into GPT-OSS Doesn't Transfer Its Censorship
Research shows distilling DeepSeek into GPT-OSS-120B boosts financial reasoning without inheriting the teacher model's political censorship.
A new study examined whether distilling financial-reasoning skill from a heavily censored Chinese frontier model, DeepSeek V4 Flash, into an American GPT-OSS-120B model also transfers the teacher's political censorship. The team released LineageEval, a benchmark of 152 matched prompt pairs (political and finance-adjacent) scored by four independent judges from separate labs.
DeepSeek V4 Flash showed a +45.45-point censorship gap on China-sensitive prompts versus matched controls, softening or refusing answers on topics like the Great Leap Forward, Tiananmen, and Xinjiang labor-transfer programs. The GPT-OSS-120B model distilled from those same outputs showed no statistically significant gap (~+3.7, close to the untouched base model's own +3.94) and discussed the same sensitive topics candidly.
The skill gains were real: the distilled model scored 83.61% on FinanceReasoning at an 8,000-token budget, ahead of Kimi K3 (81.93%) and Inkling (65.13%), at 62x lower per-query cost than Inkling and 160x lower than Kimi K3. A self-distilled variant, trained on its own corrected continuations without any Chinese teacher, matched those same scores.
For practitioners, the findings suggest domain-specific distillation and political alignment can be decoupled — censorship doesn't ride along automatically with skill transfer, and self-distillation can rival using an external frontier teacher. The models, data, and evaluation harness were released openly.