Ryan Marinelli will be holding a tutorial at PyData, Amsterdam. Evaluating large language models is expensive. Standard benchmarking practices run fixed numbers of trials regardless of how quickly signal emerges, wasting compute when differences are obvious and under-sampling when they are subtle. This talk introduces Bayesian sequential stopping rules as a principled alternative. Rather than […]