Reward for Process vs. Reward for Outcome

Key choice for multi-step tasks: outcome reward (only at the end, maximum freedom to explore, but sparse signal) vs. process reward (each step, easier learning, but risk of limiting innovation). “Reward Outcome, Constrain Process” (RLVP) compromise: R = O + β·Φ, where Φ is a verifiable path signal: penalty for each verifiable violation and partial reward for achievable progress. Dense signal rescues groups of solid failures (zero variance in GRPO → no gradient). Penalty is universally applicable; progress reward is only when progress is achievable.

Related: [Four Principles of Rubric], [RLVR and Verifiable Rewards], GRPO