Japan’s AI landscape is witnessing significant developments, with funding and innovation in the sector growing rapidly. This trend is further underscored by the latest venture in Tokyo: Sakana AI, a new AI company initiated by two renowned former Google researchers, which has just announced a $30 million funding round dedicated to developing AI models using a novel approach. Among them is Llion Jones, known as the fifth author of the groundbreaking 2017 Google research paper Attention Is All You Need. This paper introduced the transformative “transformer” deep learning architecture, which underpins the generative AI boom, including the widely-used ChatGPT. His co-founder, David Ha, has an impressive background as the head of research at Stability AI and a member of Google Brain.
Advancements in generative AI foundation models have primarily focused on expanding the size of “transformer”-based models since the release of the influential paper. However, Sakana AI is charting a different course. According to Jones in a statement to Reuters, their approach will concentrate on developing innovative architectures for foundation models, diverging from the trend of merely enlarging existing transformer models.
“Rather than building one huge model that sucks all this data, our approach could be using a large number of smaller models, each with their own unique advantage and smaller data set, and having these models communicate and work with each other to solve a problem,” said Ha, though admitted that this concept is currently just in the idea stage.
While Sakana AI has not disclosed specific funding details, the movement of these notable authors from Google to new ventures has drawn substantial venture investor interest. This pattern is evident in the activities of their peers, such as Noam Shazeer, who now leads the AI chatbot startup Character.AI, and Aidan Gomez, the founder of the large language model startup Cohere. Each of these ventures, rooted in the authors’ pioneering work on transformer AI architecture, has attracted millions in funding, signaling a robust and dynamic AI funding environment in Japan.
