Illia Polosukhin, co-author of the groundbreaking 2017 paper that introduced the transformer architecture, offered a unique perspective on the current state and future of artificial intelligence (AI) in a recent interview. As one of the “founding fathers” of generative AI, Polosukhin’s thoughts on the technology’s origins, its rapid evolution and the challenges it faces, is fascinating.
Reflecting on the transformer paper’s impact, Polosukhin expressed excitement about its widespread applicability.
“We started working on language, and this is kind of what I worked at Google on,” he said. “But we see it not just on, you know, images, videos, sound, audio, but also on DNA, on time series, on predicting purchasing behavior in credit card companies.”
The development of the transformer architecture at Google stemmed from a collaborative effort to address limitations in existing deep learning models. Polosukhin recalled the breakthrough moment: “After lunch, I went back to my computer and made a really quick prototype of translating English to French. And it worked. It was not completely random. It was picking up some signal.”
While the potential of transformers was evident early on, Polosukhin left Google to pursue his own ventures. He reflects on the challenges large companies face in rapidly deploying new technologies.
“For Google, it doesn’t make sense to launch something that’s not a $1 billion idea.” This perspective clears up why smaller, more agile organizations like OpenAI have taken the lead in commercializing generative AI.
Polosukhin advocates for a more open, user-centric approach to AI development. He argues that the current dominance of large tech companies in the field poses risks.
“The challenge with this for profit companies and again, the public companies have this need to keep increasing revenue means that their models, eventually, one way or another, will be optimized for increasing revenue,” said Polosukhin.
To address these concerns, Polosukhin proposes a different model: “How do we focus on the user and actually benefit the user?” He sees a future where AI models are more transparent and customizable, allowing users to select their preferred algorithms and data sources.
Despite concerns about AI’s potential negative impacts, Polosukhin remains optimistic about its future. He dismissed doomsday scenarios, stating: “AI is not a human, it’s a system. And the system has a goal, right? So unless somebody goes and says, let’s kill all humans. Uh, it’s not going to go in like, you know, magically do that.”
Instead, Polosukhin sees AI as a tool to enhance human capabilities.
“What we’re doing right now, we’re building systems to improve ourselves, you know, further what our minds can do,” he said, believing that responsible development and deployment of AI can lead to significant advancements across various fields, from chip design to medical research.
As the AI landscape continues to evolve, Polosukhin’s insights offer a valuable perspective on the technology’s past, present, and potential future. His vision of a more open, user-centric approach to AI development provides food for thought as society grapples with the implications of this transformative technology.
