THE USEFUL PART
- A simulator supplies a modeled world; a training process changes a controller.
- Inference runs a learned policy and does not necessarily keep training it.
- Moving from simulation to hardware requires checks against the real machine.
Follow one walking behavior
Imagine a small robot trying to walk forward without falling. We can describe its body, place it on a virtual floor, let a controller choose motor commands and observe what happens. That sequence contains several distinct jobs. The simulator calculates how the modeled world changes. A training process may improve the controller. Later, a deployed controller chooses actions from new observations. The same moving robot can appear during all three stages.
Keeping those stages separate helps you understand product demonstrations and choose a project. A simulator running on your laptop does not automatically mean an AI is learning. A neural policy running on a robot does not necessarily mean it is changing its own parameters. Start by asking which software is being run, what it receives and whether anything is being updated as a result.
Simulation supplies the environment
MuJoCo is an open-source physics engine used in robotics research and development. Its documentation describes a system of bodies, joints, contacts and actuators. Given a model and control inputs, the engine advances the simulated state. In our walking example, it estimates how the robot's pose and motion change when motors act and feet touch the floor. The result depends on the model's assumptions and parameters.
Simulation is useful even with a simple hand-written controller. You might inspect a range of movement, test a command sequence or discover that a camera mount blocks a joint. Reachy Mini's simulation guide provides an example of application prototyping without hardware. It can help you learn the software interface, while physical details such as real sound, cable routing and surface behavior still need separate evaluation.
Training changes how actions are chosen
In reinforcement learning, a policy selects actions and a training procedure uses experience and a reward signal to update it. For walking, a reward might favor progress toward a commanded velocity while penalizing unwanted motion. The difficult part includes deciding what observations, actions and rewards represent the task. A controller can become very good at the specified objective while producing behavior the designer did not intend.
Microduck's separate training repository documents MuJoCo-based environments, a reinforcement-learning workflow and exported policies for the robot. The separation makes the roles visible: the software that improves a policy and the software that runs the finished policy are different pieces. Training can be computationally expensive even when the resulting policy is small enough to execute on the robot's onboard computer.
Inference uses the learned behavior
During inference, the system feeds current observations to a trained model and obtains an output. For a movement policy, that output helps determine the next action. Microduck's runtime repository describes a 50 Hz control loop that uses neural policies to drive its servos. That is evidence about the published control architecture, not an independent measurement by Robots at Home. It explains why the robot can respond continuously without retraining for every step.
Inference can run on different hardware from training, and it can occur in simulation as well as on a physical robot. Loading a saved policy into a virtual scene is often part of evaluating it. A video of that evaluation may look much like a training visualization. Ask whether the model's parameters are frozen and whether the clip shows an evaluation episode, a training run or a hardware deployment.
The real floor does not read the model
A physical robot has imperfect measurements, friction, delays and component variation. A policy that succeeds in one precisely modeled scene may struggle when those conditions change. NVIDIA's Isaac Lab documentation identifies domain randomization as a way to improve robustness: vary selected properties during training so the controller encounters a wider set of conditions. This is a technique for handling uncertainty, not a guarantee that every possible real environment is covered.
Before transfer, confirm that the hardware, joint conventions, observation format and action limits match what the policy expects. Begin with the project's documented deployment procedure and a controlled test. Record what worked, what failed and which settings changed. The valuable outcome is an understood connection between simulation and hardware, including the conditions under which the behavior remains dependable.
Programming is a useful starting point
You can learn a great deal without training a neural network. Petoi's getting-started guide moves from built-in actions to joint control, command sequences and custom skills. Those activities teach the relationship between instructions and movement. They also make calibration and mechanical limits concrete. A sequence you design in an app is a programmed behavior; calling it training can obscure what you actually changed.
Choose a first experiment that you can explain in a few sentences. Run a documented example, change one parameter and observe the difference. When you move to learning, keep the simulator, reward, policy and deployment steps clearly named in your notes. You will then be able to tell whether an improvement came from a better model of the world, a better controller or a correction to the physical setup.