In the rapidly evolving world of artificial intelligence (AI), company founders often make bold predictions about the technology's potential to revolutionize various fields, especially the sciences. However, Thomas Wolf, co-founder and chief science officer of Hugging Face, offers a more tempered viewpoint. In a recent essay shared on X, Wolf expressed concerns that without significant breakthroughs in AI research, the technology may devolve into mere “yes-men on servers.”
Wolf elaborated on the limitations of current AI development paradigms, arguing that they are unlikely to produce systems capable of creative problem-solving akin to that which earns Nobel Prizes. He stated, “The main mistake people usually make is thinking people like Newton or Einstein were just scaled-up good students. A genius comes to life when you linearly extrapolate a top-10% student.” This highlights the necessity for AI to not only possess knowledge but to also engage in innovative thinking.
“To create an Einstein in a data center, we don’t just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask,” Wolf emphasized. This perspective sharply contrasts with the views of OpenAI CEO Sam Altman, who previously claimed that “superintelligent” AI could significantly expedite scientific discovery.
Wolf's critique extends to the fundamental issue of AI's current inability to generate new knowledge by connecting previously unrelated facts. He argued that even with vast access to the internet, contemporary AI systems primarily work to fill in gaps in existing human knowledge. According to Wolf, AI as we understand it today often serves as “very obedient students,” rather than scientific revolutionaries.
He pointed out that AI is not incentivized to question or propose ideas that challenge its training data. This limitation confines AI to answering known questions without exploring innovative solutions. “To create an Einstein in a data center, we don’t just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask,” he reiterated.
Wolf believes that the so-called “evaluation crisis” in AI contributes to this unsatisfactory state. He criticizes the benchmarks typically used to assess AI improvements, noting that they often consist of questions with clear, obvious, and “closed-ended” answers. To address this issue, Wolf proposes that the AI industry shifts towards a more nuanced measure of knowledge and reasoning.
This new approach should be capable of assessing whether AI can pursue “bold counterfactual approaches,” make general proposals from minimal hints, and pose “non-obvious questions” that could lead to groundbreaking new research paths. While Wolf acknowledges the difficulty in determining what this measure might look like, he believes the effort would be worthwhile.
“The most crucial aspect of science is the skill to ask the right questions and to challenge even what one has learned,” Wolf concluded. He argues that rather than striving for an A+ AI student who can answer every question with general knowledge, we should aim for a B student who is capable of seeing and questioning what others have overlooked.
In summary, Thomas Wolf advocates for a more thoughtful approach to AI development—one that prioritizes creativity and critical thinking over mere knowledge acquisition. His vision for the future of AI underscores the importance of fostering systems that can challenge the status quo and drive true innovation in the scientific arena.
