Can AI Pass the Bar? Linklaters Tests Robots on Legal Exams
In a groundbreaking move, one of London’s prestigious “magic circle” law firms, Linklaters, is putting artificial intelligence to the test. They’re giving robots law exams to see if these sophisticated systems can handle legal tasks. With equity partners earning a whopping average of £1.9 million, you can bet that the firm is serious about innovating the legal landscape.
What’s This All About?
Linklaters has developed the LinksAI English law benchmark, a tailored assessment designed to measure the prowess of large language models—those cutting-edge AI systems like OpenAI’s GPT series. This test aims to evaluate not only the ability of these AI to answer complex legal questions but also how well they reference strategies and avoid “hallucinations,” a term used in the AI community to describe instances when models generate incorrect information.
The Tests and Their Challenges
Linklaters’ assessments are no walk in the park. The questions posed are designed to match the complexity typically encountered by a lawyer with at least two years’ post-qualification experience in specialized fields. AI is awarded a score of up to ten points per question, based on accuracy and citation relevance.
Despite the challenges, the firm has noted that AI performance has seen significant strides over the past two years. In 2023, they tested four different AI models—GPT 2, GPT 3, GPT 4, and Bard. Bard led the pack with a score of 4.4 out of 10, though evaluators were quick to warn that all the models frequently produced inaccuracies and fictional references. Fast forward to the latest round of tests, and OpenAI’s Model O1 impressed with a score of 6.4, closely followed by Gemini 2.0 with a 6 out of 10.
The Bigger Picture
While these AI models are improving, Linklaters’ assessors have made it clear that human oversight is vital. The firms keep cautioning that these systems aren’t quite ready to slash hefty legal bills for clients yet—"expert human supervision" remains essential for any legal advice based on AI assessments. However, there is a glimmer of hope: the future of legal work may involve AI in drafting first drafts and cross-checking documents, especially in standard areas of law.
A Glimpse into the Future
Imagine a world where junior lawyers collaborate with AI to streamline their work, making the legal process more efficient without necessarily reducing the human touch that’s so vital in understanding the nuances of law. Linklaters alludes to this transformative potential in their findings—although the AI systems are far from infallible, they could become useful tools in an attorney’s arsenal.
As technology evolves, we may see AI handling summarizations of well-established legal principles, giving lawyers more time to focus on complicated, nuanced cases that need the human brain’s subjectivity and intuition.
Conclusion
The implications of AI in the legal arena could be staggering, reshaping how firms recruit, train, and work. As we witness these innovations, it’s clear that a collaborative future between humans and AI is on the horizon.
The AI Buzz Hub team is excited to see where these breakthroughs take us. Want to stay in the loop on all things AI? Subscribe to our newsletter or share this article with your fellow enthusiasts.
