Sim2Real

The gap between simulation and reality: simulation does not accurately reproduce physics, visual, and hardware characteristics.

Domain Randomization: widely and randomly vary parameters (friction, masses, lighting, delays, noise) forcing the policy to learn a robust representation – the real world becomes “one sample” from the distribution.

Two engineering points:

  • calibration of the randomization range based on empirical data from the real environment (not randomly);
  • visual alignment (camera calibration, replacing the real background in rendering).

Successes: Dactyl (Rubik’s Cube), ANYmal (rough terrain). RGB-based grasping without examples: alignment + visual randomization + physical – all three are necessary.

Related: VLA Models, Simulation Environment, Data and Environment are more important than the Algorithm

// связанные заметки
↔VLA Models ↔Simulation Environment →Data and Environment Matter More Than the Algorithm