To make AI models truly useful in real-world settings, we need better ways to measure their performance. This talk will focus on how we can improve benchmarks, ensuring LLMs are tested in ways that reflect actual business challenges.
Jonathan will discuss how using real user feedback and industry-specific examples can create more meaningful tests for AI models. We’ll explore ways to measure AI performance based on practical tasks that require applying the model’s conceptual understanding. This will complement the many existing benchmarks that focus on evaluating AI models across a range of conceptual understanding tasks.
By designing evaluation methods that reflect real-world use, we can help bridge the gap between research and business, making AI more effective and reliable in everyday applications.
About the Speaker:
Jonathan Siddharth
Founder and Chief Executive Officer, Turing
Jonathan Siddharth is the Founder and CEO of Turing, one of the world's fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems. Turing helps customers in two ways: working with the world’s leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM and frontier knowledge; and leveraging that expertise to build real-world AI systems that solve mission-critical priorities for Fortune 500 companies and government institutions.
Siddharth is a rare blend of AI scientist and serial tech entrepreneur, with a track record of building transformative AI systems and scaling successful ventures. He helped pioneer natural language search at Powerset, which was acquired by Microsoft, and went on to architect large-scale AI platforms at Rover—a content discovery engine, he co-founded and led as CEO, that was acquired by Revcontent—and at Turing, where he continues to lead cutting-edge innovation.
Beyond his work at Turing, Siddharth has served on the board of Quora, the global knowledge-sharing platform, and is an active investor and advisor to StartX, Stanford’s premier startup accelerator, where he supports the next generation of founders.
He earned his master’s degree in computer science from Stanford University, graduating with distinction in research for his work applying machine learning to web search.