As AI systems become more capable, evaluation is quickly becoming one of the most important challenges in AI development. But measuring quality in production is often much harder than running a benchmark.

Join Elicit’s Eval team for a candid discussion on the realities of evaluating AI systems in the real world. We’ll explore:

How evaluation has evolved alongside advances in AI

How to choose, adapt, or create meaningful benchmarks

Common pitfalls in AI evaluation and the lessons we’ve learned

Whether you’re building AI products or deploying them inside your organization, you’ll leave with practical lessons, candid hot takes, and actionable guidance for measuring what matters, avoiding costly mistakes, and building confidence in AI-powered workflows.

Beyond Benchmarks: Lessons from Building AI Evals in the Real World