Sakana AI, a Tokyo-based artificial intelligence (AI) firm, has introduced its latest AI model that can conduct scientific research, on Tuesday. The company claims that the AI model, dubbed The AI Scientist, can run an end-to-end scientific discovery process in the AI and machine learning (ML) fields on its own. This would include idea generation, running experiments, writing codes, accumulating and analysing results, and writing a scientific paper based on it. Sakana AI has also published a pre-peer-reviewed paper detailing the new AI model.
Sakana AI Introduces The AI Scientist
The company announced the AI model in a post on X (formerly known as Twitter) and said, “From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.”
In a separate blog post, the company said that The AI Scientist can utilise a broad research direction and a starting codebase, such as the open-source code base of older research on GitHub, and start its own discovery process. It also claimed that the AI model can follow the procedure of AI researchers and conduct literature searches, experiment planning, figure generation, manuscript reviewing, and more. It is also said to be able to run in an open-ended loop to improve the next generation of ideas based on the previous generation.
“When combined with the most capable LLMs, The AI Scientist is capable of producing papers judged by our automated reviewer as “Weak Accept” at a top machine learning conference,” Sakana AI claimed.
The company did not share any technical details about the architecture or methodology to develop the LLM. Since the AI model is not currently available to the public, Gadgets 360 was not able to verify any of the claims made by Sakana AI. However, based on the description, it appears the AI model is only able to research existing software-based ideas, and it might remain restricted by hardware limitations.
Since The AI Scientist requires a codebase as the starting, it cannot be said that it is capable of true innovation, which is often required to make a scientific discovery. The efficiency and capabilities of the AI model can only be assessed once it has been launched.
The AI firm did highlight some limitations of the current generation of The AI Scientist. The AI model does not have any computer vision capabilities which limits its ability to fix visual issues with the paper. It is also prone to hallucinations and can incorrectly implement ideas or make unfair comparisons to baselines, which can contaminate the results.
Sakana AI also highlighted that the AI model “makes critical errors when writing and evaluating results.” One particular area where it struggles is comparing the magnitude of two numbers.