Robotics
Which assumption moves a walking robot’s score most in simulation?
An actuator is the motor unit that moves one joint. It packs an electric motor, a gearbox, an encoder that reads the joint’s angle, and a driver board. Most robots are chains of these units joined by rigid links. That holds for a humanoid, a robot dog, a biped and an arm.
One view in robotics holds that the actuator is the hard and expensive part. On that view the robot’s form, meaning its frame, shell and joint layout, is a costume over the same modules. The software that decides what to do sits on top. It may be a learned walking policy, a vision-language-action model (VLA) or a world model.
Lucen Robotics builds its robots this way. Its open SDK drives a 12-joint biped, a wheeled biped, a pair of arms and a single arm. All four use the same RobStride actuators, the same motor driver and the same interface board.
A robotics lab must decide what to assume about each of these layers before it scores a policy in simulation. We asked which assumption moves the score most. We ran Lucen’s walking policy for its biped, biped_Laika, in the MuJoCo simulator. Each run changed one assumption about the actuator, the form or the ground.
The actuator model moved the score most, and the actuator’s strength moved it least. Within the stiffness range the policy was trained on, the biped turned at 52% to 99% of the rate it was asked for. Flat soles in place of the foot meshes moved that share by 4 points.
Setup
We changed one assumption at a time in Lucen's own simulator tool
biped_Laika has six joints in each leg. Three models of RobStride motor drive them. The RS06 drives the hip pitch and knee joints, with a torque limit of 18 N·m. The RS02 drives hip roll and ankle pitch, at 17 N·m. The RS00 drives hip yaw and ankle roll, at 11 N·m.
Each motor runs a position controller in its own firmware. It applies the torque kp·(target − angle) − kd·speed, clipped at the torque limit. The gain kp is the joint’s stiffness, which sets how hard it pulls toward its target. The gain kd is its damping, which sets how hard it resists fast motion.
The policy, walk.onnx, is a small neural network that runs 50 times a second. It reads the body’s tilt and spin, the joint angles and speeds, and a velocity command. It writes a target angle for each joint. Latency is the delay between the policy writing a target and the motor acting on it. Lucen measures one to two control steps of 20 ms on the real robot.
Every run used Lucen’s tools/sim2sim.py at commit 34cc92e, in the reference setting of its README. That setting scales the policy’s output by 0.8 and adds two steps of latency. It also offsets each starting joint angle by up to 0.05 rad at random. The commands were 0.2 m/s forward and 0.3 rad/s turning, the speeds Lucen uses on the robot. Each run lasted 20 s, with 5 seeds per command, 260 runs in all.
Lucen trained the policy with stiffness drawn at random between ×0.8 and ×1.2, damping between ×0.7 and ×1.3, and latency between zero and three steps. Training on randomized values, called domain randomization, is meant to make a policy robust to not knowing them exactly.
We scored each run on four measures. Two are the forward speed and the turn rate, each as a share of its command. The third is the change of heading over 20 s while the biped is told to walk straight. The fourth is falls. At the reference setting the biped reached 101% of the forward command, 49% of the turn command, and drifted −20° in heading. Across seeds these vary by 1.7 points, 2.3 points and 12°.
Stiffness
Inside the training range, stiffness changed the turn rate from 52% to 99%
At ×0.8 stiffness the biped turned at 52% of the command and walked at 80% of it. At ×1.2 it turned at 99% and walked at 112%. Its heading on a straight command changed by −6° in 20 s at ×0.8 and by +91° at ×1.2.
No run fell between ×0.8 and ×1.2. Below that range, at ×0.7, 2 of 10 runs fell, each within 1.6 s of starting to walk.
Each point is the mean of the upright runs out of 5 seeds per command. Across seeds at the reference setting, forward speed varies by 1.7 points, turn rate by 2.3 points and heading by 12°.
Domain randomization did what it promises: the policy survived the whole band it was trained on. It did not make the policy behave the same across that band. A turn score taken at nominal stiffness is 49%; the same policy on joints 20% stiffer scores 99%. At only 10% stiffer the score is already 72%.
Lucen sends kp with every motor command, so the commanded stiffness is known. The delivered stiffness can still differ. Lucen ships a check for it, called scale-check. It steps a hoisted joint by ±0.12 rad and reads kp back from the torque per radian. Lucen wrote it because a motor encoded with another model’s scale otherwise goes unnoticed.
Latency
The real robot's own latency range moves the turn by 19 points
Lucen measures one to two steps of latency on the real robot. At one step the biped turned at 68% and drifted +5°. At two steps it turned at 49% and drifted −20°. With the latency drawn afresh between one and two steps at every step, it turned at 60% and drifted −23°.
The response is not monotonic. Three steps turned more than two, and zero steps drifted further than one. A score at one latency therefore cannot be interpolated from scores at its neighbours; the latency has to be measured.
| Latency | Forward | Turn | Heading, 20 s | Falls |
|---|---|---|---|---|
| 0 steps | 103% | 58% | +107° | 0 |
| 1 step | 108% | 68% | +5° | 0 |
| 2 steps (reference) | 101% | 49% | −20° | 0 |
| 3 steps | 119% | 66% | +56° | 0 |
| 4 steps | 128% | 75% | +104° | 1 |
Strength
The walk did not change when the torque limit fell to 60%
At the reference setting the highest torque any joint reached was 70% of its limit. At ×0.8 the limit was never reached, and the runs matched the reference exactly. At ×0.6 the joints did reach the limit, yet the biped still walked at 100%, turned at 49% and drifted −23°.
The walk began to change at ×0.5, where forward speed fell to 86%. At ×0.4 it fell to 45%, and 1 of 10 runs fell. For this walk the motors’ strength has a margin of about 40%. Their stiffness has a margin of less than 10%.
| Torque limit | Forward | Turn | Heading, 20 s | Peak | Falls |
|---|---|---|---|---|---|
| ×1.0 (reference) | 101% | 49% | −20° | 70% | 0 |
| ×0.8 | 101% | 49% | −20° | 87% | 0 |
| ×0.6 | 100% | 49% | −23° | 100% | 0 |
| ×0.5 | 86% | 58% | −28° | 100% | 0 |
| ×0.4 | 45% | 58% | +37° | 100% | 1 |
Actuator, form, ground
Assumptions about the actuator outweigh the sole and the ground
The form change replaced the convex hulls of the foot meshes with flat box soles. Lucen notes that neither is the real sole and that a policy should pass with both. With box soles the biped turned at 45% and drifted −38°. A ground friction of 0.5 in place of 1.0 gave 47% and +12°.
Each stiffness, damping and latency change stayed inside a range the policy was trained on. Together they gave turn rates from 46% to 99% and heading changes from −41° to +91°. Damping alone took the heading from −41° at ×1.3 to +48° at ×0.7.
Actuator
Form
Ground
No run in this figure fell. Forward speed for every row is in the download.
What it means
A score in simulation measures the policy and an assumed actuator together
In this test the change of form moved the score little, and the actuator’s strength did not move it at all. What moved it was the simulator’s assumption about how the actuator responds: its stiffness, its damping and its latency. The hard engineering may well sit in the actuator. The hard part of evaluation is knowing which actuator you have.
A VLA that drove this biped would send it velocity commands, as a person on our try page does. Each turn it asked for would arrive at between 52% and 99% of its size. The share would depend on a stiffness that the policy’s training left open. The score that VLA gets in simulation measures the VLA and an assumed actuator together. A world model trained on video of the robot faces the same question, because the video was made by one particular set of motors.
For a real-to-sim evaluation the order follows from these numbers. Measure stiffness and latency on the real robot first. Lucen’s driver checks the delivered stiffness, and its deploy tool writes a log to compare with the simulator. Fix them in the simulator. Then score the policy. Domain randomization belongs in training, where it bought survival here. It does not replace measurement in evaluation, where it left the turn score free to fall anywhere between 52% and 99%.
Try it yourself
You can be the layer that sends the commands.
- Lucen's walking policy on biped_Laika in MuJoCo, drawn live in your browser
- Steer it from the keyboard: forward, sideways and turning
- Read what you asked for next to how fast it actually walks and turns
- Set the stiffness, damping, torque limit and delay before it starts, or walk your own stamped policy
Up to a minute a walk. New accounts get three free walks.
Walk the biped- Data and code: every run’s metrics, the job list and the aggregation script are in actuator-sweep.zip. Each run is one call of Lucen’s sim2sim.py with the flags it lists.
- One robot and one policy. Stiffness, damping and torque were scaled on all twelve joints at once; a real robot’s error differs from joint to joint.
- Simulation only. No real-robot logs were used, so these results bound how far a score can move, not where the real robot sits within that range.
- Each figure is a mean over the upright runs of 5 seeds, over the whole 20 s, start included. Falls are counted separately, out of 10 runs per condition (two commands, five seeds).
- The form change was one change of sole geometry. Larger changes of form, such as leg length or mass, were not tested.
More head-to-head evals
| Measure | Physical Agent | Qwen3.5-4B |
|---|---|---|
| LIBERO frames answered with a tool call | 400 / 400 | 18 / 400 |
| Target movement per cm of object movement (median) | 0.02 cm | — |
| Measure | Particle-1 | DeepSeek V4.1 Flash | GLM-5.3-Flash |
|---|---|---|---|
| Rows right, blind | 91.2% | 89.4% | 90.8% |
| Median seconds per run | 375 | 315 | 144 |
| Measure | GLM-5.3 | DeepSeek V4.1 Flash |
|---|---|---|
| Right, of 75 | 75 | 71 |
| Seconds a question, median | 3.9 | 1.8 |
| Measure | First run | Best since |
|---|---|---|
| Landscape, seconds | 937 | 233 |
| Memo, seconds | 759 | 602 |
| Measure | Qwen3 235B | GLM-5.2 | Gemini 3.8 Flash | Claude Opus 5 |
|---|---|---|---|---|
| Seconds to the plan, median | 27 | 48 | 57 | 94 |
| Plans written, of three runs | 2 | 3 | 3 | 3 |
| Measure | SubQ | jev | Decider | Nimble |
|---|---|---|---|---|
| Found, mean of five columns at section length | 64% | 65% | 66% | 68% |
| Mechanism of action, found at section length | 57% | 51% | 61% | 54% |
| Measure | ask | Apollo | treg |
|---|---|---|---|
| Usable leads | 13 of 40 | 7 of 40 | 6 of 28 |
| Cost per usable lead | $0.037 | $0.27 | $0.23 |
| Measure | ask with jev | ask with Decider |
|---|---|---|
| Questions right, of 92 | 89 | 89 |
| Wrong answers | 1 | 2 |
| Measure | jev | ask |
|---|---|---|
| Our 370 clinical decisions, right | 94.6% | 93.0% |
| PubMedQA, right (best EdgeEvals scored: 62.2%) | 77.0% | 74.4% |
| Measure | ask | Claude Opus 5.5 |
|---|---|---|
| Questions right, of 95 | 59 | 63 |
| Small companies right, of 20 | 13 | 9 |
| Measure | realestate-us | Best of four assistants |
|---|---|---|
| Floor-plan rents right, of 26 | 26 | 23 |
| Amenities claimed the listing doesn't state | 12 | 129 |
| Measure | ask | jev |
|---|---|---|
| Right, of the questions answered | 98% | 79% |
| Wrong answers, of 92 | 2 | 19 |
| Measure | Trained localiser | Best of four frontier models |
|---|---|---|
| Mean overlap with the defect (IoU, 0 to 1) | 0.421 | 0.109 |