What is distillation that China's DeepSeek allegedly used to piggyback off the advances of U.S rivals ChatGPT and Gemini?

This week, top White House advisers raised concerns that China’s DeepSeek may have used a technique called “distillation” to learn from and improve upon US rivals like OpenAI’s ChatGPT, reported Reuters.

Borrowing insights

Essentially, distillation involves one AI system learning from another. It allows newer models to “borrow” insights from more advanced systems, as per the report. This lets them reap the benefits of massive investments in time and computing power that were already poured into the original AI but without the hefty price tag, reported Reuters.

Violation of data

While it’s a common practice in the AI field, some experts said it is a violation of terms set by US tech companies like OpenAI, which prohibit such practices, reported Reuters.


Experts question whether DeepSeek may have used this technique to build a cheaper alternative to US-based AI models like ChatGPT and Gemini, according to the report.

ChatGPT maker OpenAI claimed it was aware of groups in China actively working to replicate US AI models via distillation, reported Reuters. The company is reviewing if DeepSeek has distilled its models inappropriately.

Databricks vice president Naveen Rao said “Competition is a real thing, and when it’s extractable information, you’re going to extract it and try to get a win,” as quoted by Reuters.

No solution yet

Technologists claimed blocking distillation would be challenging as open-source models like Meta’s Llama are freely accessible and can be used in private data centres, as per the report. Since small data samples are enough to make a big difference, it’s harder for tech companies and governments to track potential violations.

AI companies have tried to stop distillation of data but it’s not a full-proof plan. CEO of Groq Jonathan Ross said that his company has blocked all Chinese IP addresses from accessing its cloud to block Chinese firms from allegedly piggybacking off the AI models it hosts, reported Reuters.

Ross said “That’s not sufficient, because people can find ways to get around it,” quoted Reuters. “We have ideas that would allow us to prevent that, and it’s going to be a cat and mouse game … I don’t know what the solution is. If anyone comes up with it, let us know, and we’ll implement it.”

FAQs

What is AI distillation?
Distillation is when one AI system learns from another, taking insights from a more advanced model to improve itself. It’s a way for newer systems to benefit from the hard work already done by others, without the huge cost of development.

Why are companies concerned about distillation?
While distillation is common, US companies like OpenAI say it violates their terms, especially if it's used to replicate their models without permission. There's concern that it could allow companies to create cheaper alternatives to US-made AI systems.