Four Principles of a Rubric

Scale AI Principles: (1) Lean on expertise – capture key facts and steps, typical pitfalls; (2) Coverage – accuracy, coherence, completeness, safety plus explicit pitfalls; (3) Weight by importance – must-have/should-have/nice-to-have items and a veto mechanism (hallucination zeroes the score regardless of other points); (4) Self-containment – each item is checkable without domain knowledge, instead of “deep understanding” – “cited at least two theories”. Each gradation – concrete checkable behavior plus examples and edge cases. The rubric is a product of iterative refinement.

Related: LLM-as-a-Judge, [Process reward and outcome reward]