💡 LIMO: Less Data, More Reasoning in Generative AI

The LIMO (Less Is More for Reasoning) research paper challenges the conventional wisdom that complex reasoning in large language models requires massive training datasets. The authors introduce the LIMO hypothesis, suggesting that sophisticated reasoning can emerge from minimal, high-quality examples when foundation models possess sufficient pre-trained knowledge. The LIMO model achieves state-of-the-art results in mathematical reasoning using only a fraction of the data used by previous approaches. This is attributed to a focus on question and reasoning chain quality, allowing models to effectively utilize their existing knowledge. The paper explores the critical factors for reasoning elicitation, including pre-trained knowledge and inference-time computation scaling, offering insights into efficient development of complex reasoning capabilities in AI. Analysis suggests the models' architecture and the quality of data are significant factors for AI learning.