The Unsexy Side of AI: Evals, Constraints, and Saying No

Shipping AI isn’t a prompt problem. It’s an ops problem.

In this talk, I’ll walk you through how we built a production-grade AI support system using evals, controlled experimentation, and strict conversation design. You’ll see how we measured reliability, prevented regressions, constrained model behavior, and defined clear “no-AI zones” for high-risk interactions.

If you’re building real AI products—not demos—this talk will give you concrete patterns you can apply immediately.