Three Levels of Evaluation System

The evaluation system is built on three levels: (1) evaluation environment – where to test (instrumental, human-in-the-loop, simulated); (2) methods – how to judge (dataset design, metrics, LLM-as-a-Judge, pairwise comparisons); (3) decisions – what to do with the results (model selection, architecture optimization, continuous iteration). It is necessary to evaluate not the model itself, but the combination of the model and Harness. Two ways to diagnose: ablation (disable a component of Harness) and model swapping (bottleneck in the model or in the Harness?). No measurement – no improvement.