Blog
hedwigaihedwigaihedwigai

Robotics

Which assumption moves a walking robot’s score most in simulation?

An actuator is the motor unit that moves one joint. It packs an electric motor, a gearbox, an encoder that reads the joint’s angle, and a driver board. Most robots are chains of these units joined by rigid links. That holds for a humanoid, a robot dog, a biped and an arm.

One view in robotics holds that the actuator is the hard and expensive part. On that view the robot’s form, meaning its frame, shell and joint layout, is a costume over the same modules. The software that decides what to do sits on top. It may be a learned walking policy, a vision-language-action model (VLA) or a world model.

Lucen Robotics builds its robots this way. Its open SDK drives a 12-joint biped, a wheeled biped, a pair of arms and a single arm. All four use the same RobStride actuators, the same motor driver and the same interface board.

A robotics lab must decide what to assume about each of these layers before it scores a policy in simulation. We asked which assumption moves the score most. We ran Lucen’s walking policy for its biped, biped_Laika, in the MuJoCo simulator. Each run changed one assumption about the actuator, the form or the ground.

The actuator model moved the score most, and the actuator’s strength moved it least. Within the stiffness range the policy was trained on, the biped turned at 52% to 99% of the rate it was asked for. Flat soles in place of the foot meshes moved that share by 4 points.

The chain from a command to a step. Each run changed one assumption in the lower two layers and kept the upper two fixed.
Velocity commandyou on the try page, or a VLA · a few times a secondfixed: 0.2 m/s ahead, or 0.3 rad/s turningPolicy (walk.onnx)a target angle for each of 12 joints · 50 times a secondfixed: the same network in every runActuatortorque = kp·(target − angle) − kd·speed, clipped · in the motorchanged: kp, kd, latency, torque limitForm and groundlinks, masses and soles turn torque into a walkchanged: box soles; friction 0.5

Setup

We changed one assumption at a time in Lucen's own simulator tool

biped_Laika has six joints in each leg. Three models of RobStride motor drive them. The RS06 drives the hip pitch and knee joints, with a torque limit of 18 N·m. The RS02 drives hip roll and ankle pitch, at 17 N·m. The RS00 drives hip yaw and ankle roll, at 11 N·m.

Each motor runs a position controller in its own firmware. It applies the torque kp·(target − angle) − kd·speed, clipped at the torque limit. The gain kp is the joint’s stiffness, which sets how hard it pulls toward its target. The gain kd is its damping, which sets how hard it resists fast motion.

The policy, walk.onnx, is a small neural network that runs 50 times a second. It reads the body’s tilt and spin, the joint angles and speeds, and a velocity command. It writes a target angle for each joint. Latency is the delay between the policy writing a target and the motor acting on it. Lucen measures one to two control steps of 20 ms on the real robot.

Every run used Lucen’s tools/sim2sim.py at commit 34cc92e, in the reference setting of its README. That setting scales the policy’s output by 0.8 and adds two steps of latency. It also offsets each starting joint angle by up to 0.05 rad at random. The commands were 0.2 m/s forward and 0.3 rad/s turning, the speeds Lucen uses on the robot. Each run lasted 20 s, with 5 seeds per command, 260 runs in all.

Lucen trained the policy with stiffness drawn at random between ×0.8 and ×1.2, damping between ×0.7 and ×1.3, and latency between zero and three steps. Training on randomized values, called domain randomization, is meant to make a policy robust to not knowing them exactly.

We scored each run on four measures. Two are the forward speed and the turn rate, each as a share of its command. The third is the change of heading over 20 s while the biped is told to walk straight. The fourth is falls. At the reference setting the biped reached 101% of the forward command, 49% of the turn command, and drifted −20° in heading. Across seeds these vary by 1.7 points, 2.3 points and 12°.

Stiffness

Inside the training range, stiffness changed the turn rate from 52% to 99%

At ×0.8 stiffness the biped turned at 52% of the command and walked at 80% of it. At ×1.2 it turned at 99% and walked at 112%. Its heading on a straight command changed by −6° in 20 s at ×0.8 and by +91° at ×1.2.

No run fell between ×0.8 and ×1.2. Below that range, at ×0.7, 2 of 10 runs fell, each within 1.6 s of starting to walk.

Stiffness kp scaled on every joint, all else at Lucen’s reference setting. Top: speed reached as a share of the command. Bottom: heading change over 20 s while told to walk straight. Shaded: the stiffness range the policy was trained on.
40%60%80%100%120%as commanded×0.7×0.8×0.9×1.0×1.1×1.2×1.32 fell1 fell1 fell
turn rate, of 0.3 rad/sforward speed, of 0.2 m/strained range ×0.8–1.2
-50°0°50°100°150°200°straight×0.7×0.8×0.9×1.0×1.1×1.2×1.3
heading change in 20 s, straight command

Each point is the mean of the upright runs out of 5 seeds per command. Across seeds at the reference setting, forward speed varies by 1.7 points, turn rate by 2.3 points and heading by 12°.

Domain randomization did what it promises: the policy survived the whole band it was trained on. It did not make the policy behave the same across that band. A turn score taken at nominal stiffness is 49%; the same policy on joints 20% stiffer scores 99%. At only 10% stiffer the score is already 72%.

Lucen sends kp with every motor command, so the commanded stiffness is known. The delivered stiffness can still differ. Lucen ships a check for it, called scale-check. It steps a hoisted joint by ±0.12 rad and reads kp back from the torque per radian. Lucen wrote it because a motor encoded with another model’s scale otherwise goes unnoticed.

Latency

The real robot's own latency range moves the turn by 19 points

Lucen measures one to two steps of latency on the real robot. At one step the biped turned at 68% and drifted +5°. At two steps it turned at 49% and drifted −20°. With the latency drawn afresh between one and two steps at every step, it turned at 60% and drifted −23°.

The response is not monotonic. Three steps turned more than two, and zero steps drifted further than one. A score at one latency therefore cannot be interpolated from scores at its neighbours; the latency has to be measured.

Latency in control steps of 20 ms, all else at the reference setting. Falls are out of 10 runs.
LatencyForwardTurnHeading, 20 sFalls
0 steps103%58%+107°0
1 step108%68%+5°0
2 steps (reference)101%49%−20°0
3 steps119%66%+56°0
4 steps128%75%+104°1

Strength

The walk did not change when the torque limit fell to 60%

At the reference setting the highest torque any joint reached was 70% of its limit. At ×0.8 the limit was never reached, and the runs matched the reference exactly. At ×0.6 the joints did reach the limit, yet the biped still walked at 100%, turned at 49% and drifted −23°.

The walk began to change at ×0.5, where forward speed fell to 86%. At ×0.4 it fell to 45%, and 1 of 10 runs fell. For this walk the motors’ strength has a margin of about 40%. Their stiffness has a margin of less than 10%.

Torque limit scaled on every joint, all else at the reference setting. Peak is the highest torque as a share of the scaled limit.
Torque limitForwardTurnHeading, 20 sPeakFalls
×1.0 (reference)101%49%−20°70%0
×0.8101%49%−20°87%0
×0.6100%49%−23°100%0
×0.586%58%−28°100%0
×0.445%58%+37°100%1

Actuator, form, ground

Assumptions about the actuator outweigh the sole and the ground

The form change replaced the convex hulls of the foot meshes with flat box soles. Lucen notes that neither is the real sole and that a policy should pass with both. With box soles the biped turned at 45% and drifted −38°. A ground friction of 0.5 in place of 1.0 gave 47% and +12°.

Each stiffness, damping and latency change stayed inside a range the policy was trained on. Together they gave turn rates from 46% to 99% and heading changes from −41° to +91°. Damping alone took the heading from −41° at ×1.3 to +48° at ×0.7.

One change at a time against Lucen’s reference setting (dashed line: turn 49%, heading −20°). The stiffness, damping and latency changes stay inside the ranges the policy was trained on.

Actuator

Stiffness ×0.8turnheading
Stiffness ×1.2turnheading
Damping ×0.7turnheading
Damping ×1.3turnheading
Latency 1 stepturnheading
Latency 1–2 steps, varyingturnheading
Torque limit ×0.6turnheading

Form

Flat box solesturnheading

Ground

Friction 0.5turnheading

No run in this figure fell. Forward speed for every row is in the download.

What it means

A score in simulation measures the policy and an assumed actuator together

In this test the change of form moved the score little, and the actuator’s strength did not move it at all. What moved it was the simulator’s assumption about how the actuator responds: its stiffness, its damping and its latency. The hard engineering may well sit in the actuator. The hard part of evaluation is knowing which actuator you have.

A VLA that drove this biped would send it velocity commands, as a person on our try page does. Each turn it asked for would arrive at between 52% and 99% of its size. The share would depend on a stiffness that the policy’s training left open. The score that VLA gets in simulation measures the VLA and an assumed actuator together. A world model trained on video of the robot faces the same question, because the video was made by one particular set of motors.

For a real-to-sim evaluation the order follows from these numbers. Measure stiffness and latency on the real robot first. Lucen’s driver checks the delivered stiffness, and its deploy tool writes a log to compare with the simulator. Fix them in the simulator. Then score the policy. Domain randomization belongs in training, where it bought survival here. It does not replace measurement in evaluation, where it left the turn score free to fall anywhere between 52% and 99%.

Try it yourself

You can be the layer that sends the commands.

  • Lucen's walking policy on biped_Laika in MuJoCo, drawn live in your browser
  • Steer it from the keyboard: forward, sideways and turning
  • Read what you asked for next to how fast it actually walks and turns
  • Set the stiffness, damping, torque limit and delay before it starts, or walk your own stamped policy

Up to a minute a walk. New accounts get three free walks.

Walk the biped
Continue in
  • Data and code: every run’s metrics, the job list and the aggregation script are in actuator-sweep.zip. Each run is one call of Lucen’s sim2sim.py with the flags it lists.
  • One robot and one policy. Stiffness, damping and torque were scaled on all twelve joints at once; a real robot’s error differs from joint to joint.
  • Simulation only. No real-robot logs were used, so these results bound how far a score can move, not where the real robot sits within that range.
  • Each figure is a mean over the upright runs of 5 seeds, over the whole 20 s, start included. Falls are counted separately, out of 10 runs per condition (two commands, five seeds).
  • The form change was one change of sole geometry. Larger changes of form, such as leg length or mass, were not tested.

More head-to-head evals

Does Physical Agent aim its tool calls at the object?Physical Agent writes a robot tool call on 400 of 400 frames, but moving the object 16 cm leaves its targets where they were.
MeasurePhysical AgentQwen3.5-4B
LIBERO frames answered with a tool call400 / 40018 / 400
Target movement per cm of object movement (median)0.02 cm—
Read the comparison
Particle-1, DeepSeek V4.1 Flash and GLM-5.3-Flash in Claude Code with hedwigai workbooksParticle-1, DeepSeek V4.1 Flash and GLM-5.3-Flash, 180 runs graded blind, rows right: 91.2%, 89.4% and 90.8%, intervals overlapping.
MeasureParticle-1DeepSeek V4.1 FlashGLM-5.3-Flash
Rows right, blind91.2%89.4%90.8%
Median seconds per run375315144
Read the comparison
GLM-5.3 against DeepSeek V4.1 Flash on 75 questionsTwo models answer the same 75 questions; one is always right, the other 14× cheaper.
MeasureGLM-5.3DeepSeek V4.1 Flash
Right, of 757571
Seconds a question, median3.91.8
Read the comparison
How we made a long-running AI agent for knowledge work 4× fasterOne cited knowledge work task through 13 experiments: 937 s to 233 s, with the model held constant.
MeasureFirst runBest since
Landscape, seconds937233
Memo, seconds759602
Read the report
Ten models plan a change to a spreadsheet: which do it in under a minute, traced call by callTen models drive one agent harness through the same spreadsheet request; six write the plan, timed and traced call by call.
MeasureQwen3 235BGLM-5.2Gemini 3.8 FlashClaude Opus 5
Seconds to the plan, median27485794
Plans written, of three runs2333
Read the comparison
SubQ, jev, Perplexity's Decider and Bespoke's Nimble on biotech diligence memos: finding the sentences a column is aboutFour decision models judge every sentence of ten biotech diligence memos for five columns, scored against sentences a person marked.
MeasureSubQjevDeciderNimble
Found, mean of five columns at section length64%65%66%68%
Mechanism of action, found at section length57%51%61%54%
Read the comparison
Leads with a reason to write: hedwigai ask, Apollo and treg on one brief40 leads each, every one checked: the right person, an inbox that accepts, and a dated reason to write.
MeasureaskApollotreg
Usable leads13 of 407 of 406 of 28
Cost per usable lead$0.037$0.27$0.23
Read the comparison
Self-hosted decisions for ask: Perplexity's open Decider in place of jev on 92 company questionsThe decisions inside hedwigai ask made by Perplexity's open Decider on two NVIDIA RTX 3090s, against jev.
Measureask with jevask with Decider
Questions right, of 928989
Wrong answers12
Read the comparison
Clinical decisions: jev and ask on System One, PubMedQA, SciFact and a 370-decision setEdgeEvals' System One board, PubMedQA, SciFact, and a clinical set built to System One's rules.
Measurejevask
Our 370 clinical decisions, right94.6%93.0%
PubMedQA, right (best EdgeEvals scored: 62.2%)77.0%74.4%
Read the comparison
Executive lookup for US-listed companies: Claude Opus 5.5 and ask on 95 questionsWho runs a public company, and how to email them: a frontier model from memory against hedwigai ask.
MeasureaskClaude Opus 5.5
Questions right, of 955963
Small companies right, of 20139
Read the comparison
Rental comps: floor-plan rents and amenities on 14 listings, five assistants comparedRent per floor plan and amenities for US rentals: hedwigai realestate-us against Claude, GPT, Gemini and Perplexity with web search.
Measurerealestate-usBest of four assistants
Floor-plan rents right, of 262623
Amenities claimed the listing doesn't state12129
Read the comparison
Company screening from SEC filings: jev, Vela 2.0 and ask on 92 yes/no questionsYes-or-no questions about public companies, answered by two decision models alone and by hedwigai ask.
Measureaskjev
Right, of the questions answered98%79%
Wrong answers, of 92219
Read the comparison
Defect localization in industrial images: four frontier models and a trained localizerFour frontier models asked to put a box around what they just described.
MeasureTrained localiserBest of four frontier models
Mean overlap with the defect (IoU, 0 to 1)0.4210.109
Read the comparison