Safety Boundaries of Self-Modification

Self-evolution turns a single mistake into a long-term risk. Three boundaries: (1) separation of evidence and instructions—raw external content isn’t written to Skill directly, only via LLM generalization with version control; (2) candidate and official capabilities—new artifacts first in a candidate space off real traffic, with sandboxing and supply chain scanning; (3) prohibition of self-modifying safety mechanisms—the agent doesn’t change the checks, tests, release thresholds, audit logs, and backups approving its own updates. Otherwise, lowering the bar enough looks like progress.

Related: [Check - Release - Rollback], [Deadly Triad], [Prompt Injection]