Flood Sung, a researcher on the Dark Side of the Moon, recently published a long article of 10,000 words, which disclosed the research and development ideas of the k1.5 model for the first time and deeply reflected on the technical implications brought by OpenAI's o1 model. This disclosure not only reveals the latest progress of the Dark Side of the Moon in the field of artificial intelligence, but also provides the industry with valuable technical reference.
Flood Sung mentioned in the article that the importance of Long-CoT (Long Chain Thinking) was actually verified by Tim Zhou Xinyu, co-founder of the Dark Side of the Moon more than a year ago. By using small models to train multi-digit operations and converting fine-grained computing processes into long-chain thinking data for SFT (supervised fine-tuning), the team achieved significant results. This discovery provides an important theoretical basis for subsequent model optimization.
However, due to cost limitations, the Dark Side of the Moon has previously focused on the optimization of Long Context (Long Text Input). Flood Sung explained that Long Context mainly processes inputs, and through Prefill pre-filling and Mooncake technology, the team can better control costs and speed. In contrast, Long-CoT focuses more on the output, and while it is significant, it requires higher costs and longer processing times, which limits its application to some extent.
However, the release of the OpenAI o1 model has caused the Dark Side team to rethink the priorities of the technical direction. Flood Sung emphasized: "Performance is the most important thing, cost and speed will be continuously optimized with technological progress. The key is to achieve breakthrough performance first." Based on this understanding, the Dark Side of the Moon has begun to comprehensively promote Long-CoT research. Committed to bringing models to achieve free thinking ability closer to humans. This strategic adjustment marks a further breakthrough for the team in the field of artificial intelligence.
The release of this technical decryption article not only marks that the dark side of the moon has begun to systematically benchmark OpenAI's o1 model, but also conducts substantial research in related fields. Flood Sung's long article provides the industry with in-depth technical insights and provides new ideas for future research directions.
For readers who want to have an in-depth understanding of the cracking process of the O1 model, Flood Sung's 10,000-word long article can be accessed through the following link: Decrypting the 10,000-word long article of the O1 cracking process .