Eye on AI: Beyond Foundation Models – Innovations and Challenges in the AI Landscape

Hello there! Welcome to this week’s edition of Eye on AI. In this newsletter, we’ll explore why a legal AI startup demonstrates that the AI boom encompasses much more than just foundational models. We’ll also share news about Zoox’s robotaxi rides in San Francisco, discuss the troubling ease of jailbreaking LLM-powered robots, and ponder whether the progress in foundation models is nearing its limit.

The AI Progress Dilemma

Recent discussions have erupted around the potential stagnation of general-purpose foundation models and its implications for the AI boom. Some industry skeptics, like Gary Marcus, are forecasting a tumultuous period akin to the dotcom crash. Tomorrow, I’ll be diving deeper into this topic at the Web Summit in Lisbon, where I’ll be moderating a panel titled “Is the AI Bubble About to Burst?” Tune into the Web Summit livestream at 4:25 PM local time to catch the conversation with Moveworks CEO Bhavin Shah and AI Now Institute’s Sarah Myers West.

My take? Even if foundation model advancements slow down, companies focused on delivering AI solutions for specific industries may not feel the impact as heavily as giants like OpenAI, whose massive $157 billion valuation hinges on creating artificial general intelligence (AGI). If the trend of building ever-larger LLMs doesn’t yield substantial capability advantages to justify their cost, those companies could find themselves in a tricky position.

Solutions Over Models

The key for many AI application-based companies lies in selling solutions tailored to specific industry challenges, rather than promoting a singular AI model or a nebulous idea like "general purpose intelligence." Often, these solutions don’t necessitate AGI or further AI breakthroughs. Simply combining existing models and refining them for specific professional tasks can create substantial business value.

A perfect case in point is Robin AI. Founded in 2019 by Richard Robinson, a former lawyer, and James Clough, a seasoned machine learning researcher, Robin AI doesn’t just peddle a piece of software. Instead, it offers legal services to major corporations, with various services delivered automatically through AI or by human lawyers and paralegals who leverage Robin’s technology.

“We’re blending capabilities of current models with investments in what’s just out of reach today, all while utilizing humans in the loop to bridge the gap,” Robinson explains.

Closing the Expectations Gap

Robinson acknowledges a disparity between public expectations of AI models and their real-world capabilities. While state-of-the-art AI excels in summarization and translation, it struggles with complex tasks like negotiating legal documents or drafting briefs reliably. “They can handle parts of the task, but not the entire job,” he notes.

However, Robin AI proves that there’s a viable business even when there are persistent capability limitations. Whether clients simply use their software or outsource entire legal tasks, Robin finds ways to efficiently deliver results at competitive prices—thanks to their technology-enhanced workflows.

Notably, Robin employs skilled legal professionals in high-cost locations like New York, London, and Singapore, but their technology significantly boosts efficiency without resorting to labor arbitrage.

Investment Confidence in Robin AI

In a show of assurance for Robin’s future, the company recently closed a Series B Plus funding round of $25 million, building on a prior $26 million Series B round. This brings their total fundraising to $61.5 million, with investors including PayPal’s venture arm, Michael Bloomberg’s Willets family office, and the University of Cambridge—all of whom also happen to be customers.

Despite having adequate runway from their previous funding, Robinson stated the need for additional capital to enhance their popular “Reports” product, which allows users to consult multiple questions about document sets—powered by Anthropic’s Claude model.

Real-Life Impact of Robin AI

One illustrative example of Robin AI’s impact involves a U.S. biotech firm facing a data breach. In a matter of hours, Robin reviewed 10,000 contracts across 30 types to assess notification obligations. Using its Reports feature and human expertise, the biotech identified the top 50 contracts needing urgent notification and formulated an action plan for all 10,000 contracts within just 72 hours. This efficiency reportedly saved the firm 93% of the time and 80% of the estimated $2.6 million it would have cost to hire an external law firm. That’s the tangible value companies are gaining from AI today.

Fresh Developments in AI

• Zoox Launches Robotaxi Services: The autonomous taxi company, Zoox, has begun offering rides in San Francisco. Initially available only to employees, these robotaxis will operate in the SoMa neighborhood. This marks Zoox’s expansion following its success in Las Vegas.

• AlphaFold 3 for Researchers: Google DeepMind and Isomorphic have released the model weights and code for AlphaFold 3 for academic use. This powerful model can predict biological molecule structures and interactions, aiding research significantly.

• Tencent’s Hunyuan-Large Model: Tencent has unveiled its Hunyuan-Large model, claiming it surpasses Meta’s Llama 3.1 in benchmarks, showcasing the competitive nature of AI development.

The Jailbreak Concern

A growing alarm among AI researchers is the ease of jailbreaking LLM-powered robots, which poses significant risks. At the University of Pennsylvania, the RoboPAIR software successfully found prompts to bypass defense mechanisms in various robots. This laxity raises safety concerns, especially when LLMs control machines that can act autonomously in the real world.

Reflecting on AI Scaling Laws

On the topic of model progress, it appears that the scaling laws proposed by OpenAI could be losing their efficacy. New insights suggest that merely increasing model size and data might yield diminishing returns. Recent developments from OpenAI indicate that their latest model, Orion, has not outperformed its predecessor GPT-4 in critical tasks, prompting the necessity for further fine-tuning.

This potential setback could reshape the trajectory of the AI landscape, steering companies to explore alternative architectures that are more efficient and environmentally friendly.

Conclusion

As we explore these intriguing developments, it’s clear that the AI landscape is more dynamic than ever, with innovative companies paving the way for practical solutions. The AI Buzz Hub team is excited to see where these breakthroughs take us. Want to stay in the loop on all things AI? Subscribe to our newsletter or share this article with your fellow enthusiasts!