Reference-Based Distillation Detection in LLMs

Short summary

This arXiv paper introduces a reference-based method for detecting whether an LLM was distilled from a stronger teacher model. By comparing how strongly a student aligns with candidate teacher outputs relative to an earlier checkpoint from the same lineage, the method recovers the true teacher with near-perfect accuracy in single-teacher scenarios. The authors also identify a glyph-level signal specific to o1/o3 models and apply their framework to contemporary models, finding new evidence of potential distillation relationships involving QwQ, DeepSeek-R1, and GPT-OSS.

• •Reference-based membership inference can identify teacher models used to distill later checkpoints with near-perfect accuracy
• •A distinctive glyph-level signal specific to OpenAI o1/o3 models aids detection
• •Applied to real models, the method surfaces potential distillation relationships involving QwQ, DeepSeek-R1, and GPT-OSS

Generated with AI, which can make mistakes.