Artificial intelligence may be entering a new phase in which the most expensive task is no longer creating every model from the beginning, but finding efficient ways for one model to teach another.
The Question Raised by Kimi
A recent controversy surrounding Moonshot AI’s Kimi model has brought an important technical concept into public discussion: knowledge distillation.
A United States official alleged that Moonshot AI had used outputs from an Anthropic model during the development of Kimi. The complete evidence and technical details have not been publicly established, so the allegation should not be treated as a proven account of how Kimi was trained.
Nevertheless, the controversy raises a much broader and more interesting question:
Can a company avoid some of the enormous cost of developing a frontier AI model by allowing an existing model to teach a new one?
The answer is yes, at least to a significant degree. This method is called knowledge distillation. It is neither new nor inherently improper. Researchers have studied it for many years, and AI companies routinely use forms of distillation to create smaller, faster, and less expensive models from larger ones.
What is new is the scale of the opportunity. When the teacher is a frontier language model trained with billions of dollars of computing infrastructure, the student may inherit a substantial portion of that capability without repeating the teacher’s entire educational journey.
The Teacher and the Student
The simplest way to understand distillation is through the relationship between a teacher and a student.
Imagine a professor who has spent thirty years studying mathematics, physics, history, literature, and computer science. The professor has read thousands of books, solved countless problems, made mistakes, corrected them, and gradually developed an organized understanding of the subjects.
A young student does not need to repeat the professor’s entire life.
The student does not have to rediscover algebra, reinvent calculus, reproduce every scientific experiment, or read every book the professor has read. Instead, the student attends lectures, asks questions, studies worked examples, and receives carefully organized explanations.
The professor has already performed much of the difficult intellectual compression. Decades of learning are transformed into lessons that the student can absorb in a much shorter time.
In AI distillation:
- The powerful model is the teacher.
- The model being trained is the student.
- The teacher’s answers become part of the student’s educational material.
The student does not receive a copy of the teacher’s internal neural network. Instead, it observes how the teacher responds to questions and gradually learns to reproduce similar behavior.
This is not entirely different from human education. A student cannot inspect the neurons inside a professor’s brain. The student learns by observing what the professor says, writes, demonstrates, and corrects.
How Distillation Works
A simplified distillation process may contain several stages.
1. Create a large collection of prompts
Developers prepare questions, instructions, problems, conversations, coding assignments, mathematical exercises, and reasoning tasks. The prompts may be generated by humans, assembled from existing datasets, or created automatically.
2. Ask the teacher model
The prompts are submitted to a powerful model. Its responses are collected. Depending on the purpose of the project, developers may request final answers, explanations, structured outputs, code, critiques, or step-by-step demonstrations.
3. Filter and evaluate the answers
Teacher models can still make mistakes. Their answers may therefore be checked, ranked, corrected, or filtered. Poor examples are removed, while high-quality examples become training material.
4. Train the student model
The student is trained to predict responses resembling the teacher’s successful answers. Through many examples, it gradually absorbs patterns of language, problem-solving strategies, formatting conventions, and task-specific behavior.
5. Test and improve the student
The student is evaluated on problems it has not previously seen. Developers may identify weaknesses, generate additional lessons, and repeat the process.
From the outside, this can look like continuous prompting. However, ordinary prompting and distillation are not the same thing.
Prompting uses a model to answer a question at that moment. Distillation collects many answers and uses them to alter the parameters of another model. The knowledge is no longer confined to a conversation. It becomes embedded in the student’s own weights.
Columbus’s Egg
The logic of distillation can also be understood through the famous story of Columbus’s egg.
According to the traditional anecdote, people told Christopher Columbus that his achievement was not especially remarkable. Once the route had been discovered, they argued, anyone could have made the voyage.
Columbus placed an egg on the table and challenged them to make it stand upright. After everyone failed, he lightly flattened one end of the egg and stood it on the table.
The others protested that the solution was obvious and that anyone could have done it.
That was precisely the point.
Before someone shows the solution, the problem appears impossible. Afterward, the solution appears obvious.
The story is probably apocryphal, and similar versions existed before it became associated with Columbus. Its historical accuracy matters less than the principle it illustrates.
The first explorer faces uncertainty. He does not know whether the route exists, whether the ship will survive, or whether the journey will end in success. Those who follow possess something enormously valuable: proof that the destination can be reached.
The first frontier AI laboratories faced a similar uncertainty. They invested extraordinary amounts of money in data centers, semiconductor chips, electricity, research, and engineering without knowing exactly what level of capability would emerge.
Once a powerful model exists, however, the world has seen the egg standing upright.
Competitors now know that a general-purpose language model can write software, solve mathematical problems, explain scientific ideas, summarize documents, and operate as the reasoning component of an AI agent. They can study its behavior, read published research, recruit experienced engineers, experiment with new architectures, and potentially use stronger models as teachers.
The pioneer pays the cost of proving that the path exists. The followers begin their journey with a map.
The Economics of Distillation
Training a frontier model from raw data can require massive investments in computing clusters, electricity, data preparation, engineering, and repeated experimentation. Much of the cost comes not merely from the final successful training run, but from all the failed experiments and uncertain decisions preceding it.
Distillation can reduce part of this burden because the teacher has already organized a great deal of information into useful behavior.
The student no longer has to extract every lesson directly from an ocean of unstructured text. The teacher can provide cleaner examples, targeted explanations, corrected code, and solutions designed specifically for learning.
We can think of the difference this way:
Training from the beginning:
Raw data
↓
Massive computation
↓
Repeated experiments
↓
Failures and corrections
↓
Frontier model
Training with a teacher:
Selected prompts
↓
Teacher-generated lessons
↓
Filtered training examples
↓
Student training
↓
A smaller or specialized modelThis does not mean that a student model can reproduce an entire frontier model cheaply or perfectly. It still requires substantial data, computing power, engineering, and independent training. However, it may obtain useful capabilities for a fraction of the cost required to discover all of them independently.
That changes the economics of the AI race.
A company that spends billions creating a frontier teacher may unintentionally illuminate the road for competitors. The followers can observe which capabilities are possible, which product designs attract users, which research directions work, and which mistakes should be avoided.
Distillation makes this process even more direct. The pioneer’s model may become not only an example to imitate, but also a machine capable of producing educational material for its future competitors.
What the Student Cannot Inherit
The teacher-and-student analogy is powerful, but it should not be pushed too far.
A student who memorizes a professor’s answers may perform well on familiar examinations but struggle when confronted with genuinely new problems. Similarly, a distilled model may imitate the visible behavior of a stronger model without acquiring all of its underlying abilities.
The teacher’s answers represent only a limited sample of what the teacher knows. They do not reveal the teacher’s complete internal structure, all its training data, or every capability hidden in its parameters.
The student may also inherit the teacher’s weaknesses:
- incorrect answers,
- reasoning shortcuts,
- cultural or statistical biases,
- unsafe behavior,
- confidently expressed uncertainty.
Distillation can therefore create a photocopy of both knowledge and error.
There is also a danger of intellectual inbreeding. If future models learn mainly from previous models rather than from human experience, scientific observation, real-world interaction, and original data, mistakes may circulate from one generation to the next.
The strongest student is not the one who merely repeats the teacher. It is the one that learns from the teacher while continuing to study the world independently.
Learning or Unauthorized Copying?
Knowledge distillation itself is a legitimate and widely used machine-learning technique. A company may distill its own larger model into a smaller model. A model provider may explicitly permit customers to use generated outputs for fine-tuning. Researchers may also distill models under licenses that authorize such use.
The controversy begins when one company systematically queries another company’s proprietary model for the purpose of building a competing product, especially when this violates contractual restrictions or involves deceptive access methods.
The distinction can be summarized as follows:
- Authorized distillation: The teacher belongs to the same organization, is openly licensed, or is used with the provider’s permission.
- Unauthorized distillation: The teacher’s outputs are collected contrary to access rules, contractual terms, or technical restrictions.
The technical process may be similar in both cases. The difference lies in authorization, ownership, contracts, and the methods used to obtain the outputs.
This creates a difficult philosophical and legal question. Humans routinely learn from books, teachers, competitors, and existing inventions. A programmer can study the behavior of a software product and develop a competing program without copying its source code. Yet automated extraction at industrial scale may resemble replication more than ordinary human learning.
AI law will have to decide where education ends and appropriation begins.
A New Engine of AI Progress
Distillation may become one of the hidden engines accelerating artificial intelligence.
The first generation of frontier models learned largely from human-created material: books, websites, scientific papers, software code, images, and conversations.
The next generation may learn from both humanity and earlier AI systems.
Human knowledge
↓
Frontier teacher model
↓
Distilled student models
↓
Specialized medical, scientific,
industrial, and personal AI systemsThis resembles the development of human civilization. Every child is born without knowledge of mathematics, medicine, engineering, or history. Yet each generation does not begin civilization again from zero. Schools, books, universities, and teachers transmit accumulated knowledge to the next generation.
Education is humanity’s system of knowledge distillation.
A medical student does not repeat every experiment in the history of medicine. An engineering student does not rediscover electricity. A software developer does not reinvent the transistor before writing a computer program. Each learner receives a compressed inheritance from earlier generations.
Artificial intelligence may now be developing its own version of this inheritance.
The most expensive frontier models could become professors to thousands of smaller students. Some students will be optimized for mobile devices. Others will work inside hospitals, factories, laboratories, vehicles, robots, or government data centers. They may be less capable in general, yet highly effective in their chosen domains.
This possibility makes it difficult for any single company or country to preserve a permanent technological lead. Research ideas spread. Engineers move. Models generate training data. Open-weight systems allow experimentation. Once an achievement has been demonstrated, competitors no longer need to prove that it is possible.
They have seen Columbus’s egg.
The first company may spend billions discovering how to make it stand. The next company studies the result, learns from the teacher, and attempts to build a better student at lower cost.
Yet imitation alone will not determine the ultimate winner. A student that only reproduces yesterday’s answers will always remain behind yesterday’s teacher. Lasting progress still requires original research, better data, new architectures, improved hardware, real-world experience, and the courage to attempt problems whose solutions have not yet been demonstrated.
Distillation helps travelers follow an existing map. Innovation is still required to draw the next one.
