HumanTracker Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Dairu Liu1,3,* Zekun Qi2,3,* Jiayu Zeng3,* Ruixi Yu2,3 Yu Guan2,3 Yintianrun Zhang3 Xuchuan Chen2,3 Sikai Liang3,4 Zekai Li2,3 Chenghuai Lin3 Xinqiang Yu3 Wenyao Zhang4 He Wang3,5,† Li Yi2,3,6,†
1Nankai University 2Tsinghua University 3Galbot 4Shanghai Jiao Tong University 5Peking University 6Shanghai Qi Zhi Institute

*Equal contribution    †Corresponding authors

Paper Code Dataset

HumanTracker overview

HumanTracker overview showing the HumanScore reward model and four-category evaluation benchmark
HumanTracker combines a preference-aligned trajectory metric with a large, category-aware benchmark for humanoid motion tracking.

Tracking quality should agree with what people see.

Kinematic errors average frame-level pose differences, but often miss the physical artifacts that matter most in video: unstable support, foot skating, incorrect contact timing, and failed recoveries.

HumanTracker addresses both sides of evaluation. Its benchmark supplies broad, diagnostic motion coverage, while HumanScore learns trajectory-level quality from human preferences and predicts which rollout people prefer.

90.83% alignment with human preferences on the test set 6.78 points above the strongest conventional diagnostic.
153hours of optical motion
25Klabeled motion clips
24professional performers
4diagnostic motion families

HumanScore learns the temporal evidence behind human judgments.

HumanScore trajectory reward model from the paper with a temporal Transformer and masked mean pooling

6,000 original pairs. Six expert annotators provide strict preferences, similar judgments, and cannot-compare judgments. Bilateral mirroring yields 12,000 preference records.

Five-second context. Each 250-frame window is encoded as a 539-dimensional frame-token sequence with explicit padding masks.

Readable output. Window rewards pass through a sigmoid and are averaged from 0 to 100, weighted by the number of actual frames.

Test set preference alignment

HumanScore
90.83
KPT position MAE
84.05
Joint velocity error
84.04
MPJPE
80.49
Foot contact accuracy
78.82

Family-balanced alignment rate (%) on original, unmirrored strict-preference comparisons from the motion-disjoint test set.

What the model uses

HumanScore feature sensitivity analysis on the test set
Measured contact features matter most for Ground motions; future reference does not improve the baseline.
HumanScore temporal context analysis on the test set
Longer temporal context reveals sliding, jitter, drift, and recovery.

See what HumanScore sees.

Daily

Kneeling and high-knee steps

37.32 s

GMT

#4

HumanScore 5.70MPJPE (rad) 0.158

Humanoid-GPT

#1

HumanScore 86.75MPJPE (rad) 0.044

SONIC

#2

HumanScore 73.00MPJPE (rad) 0.103

TWIST2

#3

HumanScore 24.37MPJPE (rad) 0.081

Daily

Walking, turns, and high-knee steps

35.72 s

GMT

#4

HumanScore 0.95MPJPE (rad) 0.155

Humanoid-GPT

#1

HumanScore 82.29MPJPE (rad) 0.043

SONIC

#2

HumanScore 79.13MPJPE (rad) 0.075

TWIST2

#3

HumanScore 23.11MPJPE (rad) 0.076

Daily

Straight walking

36.40 s

GMT

#4

HumanScore 1.62MPJPE (rad) 0.166

Humanoid-GPT

#1

HumanScore 79.72MPJPE (rad) 0.038

SONIC

#2

HumanScore 79.28MPJPE (rad) 0.060

TWIST2

#3

HumanScore 5.83MPJPE (rad) 0.066

0:00 / 0:00

Four families expose different failure regimes.

Every clip includes a family label, a natural-language description, a fitted SMPL sequence, and a robot-space reference trajectory.

Motion taxonomy from the paper showing Daily, Highly Dynamic, Interaction, and Ground families

Daily

Steady locomotion, turning, and routine gestures reveal stability and residual drift.

Hours
89.29
Clips
9,739

Highly Dynamic

Jumps, kicks, acrobatics, and fast footwork stress impact handling and phase timing.

Hours
11.01
Clips
2,676

Interaction

Object and environment interactions require hand, arm, and whole-body coordination.

Hours
47.78
Clips
10,940

Ground

Low postures and multi-contact transitions test contact geometry, balance, and recovery.

Hours
4.59
Clips
1,640

Zero-shot results

Succ (%) ↑, MPJPE (rad) ↓, HumanScore ↑ on a 0–100 scale, computed on the test split. The GMT, TWIST2, SONIC, and Humanoid-GPT rows match Table 2 of the paper. SONIC 1.1 and nine additional released policies were evaluated under the same protocol.

Zero-shot tracking results for four motion families.
Method Daily Highly Dynamic Interaction Ground
SuccMPJPEHScore SuccMPJPEHScore SuccMPJPEHScore SuccMPJPEHScore
GMT 17.00.2502.4 36.20.1967.0 81.40.20511.7 0.00.4564.0
TWIST2 60.10.10510.1 39.90.11216.9 91.30.11128.3 0.00.3414.5
SONIC 93.80.10249.5 82.10.11841.0 97.60.12854.6 20.10.23126.5
SONIC 1.1 94.40.09343.6 84.70.10838.5 97.90.11148.9 13.40.19833.9
Humanoid-GPT 94.40.04654.7 86.90.04749.2 97.20.07056.8 32.90.21624.9
ScaleBFM-M 90.80.09444.5 72.40.09139.3 96.50.08454.0 17.70.18643.4
HEFT 93.20.05646.1 75.70.06140.5 97.50.06351.6 15.90.17631.7
ScaleBFM-XL 90.10.09344.2 74.30.08937.5 96.70.08251.2 17.10.15840.8
GRIT v0.0.1 90.20.10540.2 65.70.11541.4 96.50.11448.5 11.00.24124.9
TeleopIT 84.00.09634.0 54.90.11035.8 96.20.10545.7 13.40.22947.1
MimicLite-PPO 81.10.08028.0 58.60.09628.1 95.20.08937.9 7.90.23920.1
MimicLite-ROA 83.10.08024.8 59.00.11226.9 95.20.09335.1 2.40.29026.6
HoloMotion 84.00.08820.4 63.40.10127.2 94.90.09140.4 18.30.17235.1
Mimic Lite v1.1 80.10.08025.0 51.50.11422.8 94.40.08734.3 7.30.25223.1

Every tracker keeps its native observation and action-processing stack, receives the same retargeted references, and is measured by the same evaluator under the SONIC termination criterion.

One score, used with complementary diagnostics.

HumanScore measures perceived quality.

It summarizes trajectory-level preference over contact, stability, smoothness, and reference fidelity. Higher is better within the same evaluation setting.

Success measures completion.

A rollout may look accurate before an early failure. Success prevents a short failed trajectory from being mistaken for robust tracking.

Kinematic errors diagnose pose mismatch.

MPJPE and related errors remain useful for locating specific problems; HumanScore is not a replacement for every analytic diagnostic.

HumanTracker

@misc{liu2026humantrackercomprehensivehumanalignedmotion,
  title         = {HumanTracker: Towards Comprehensive and Human-Aligned
                   Motion Tracking Benchmark},
  author        = {Dairu Liu and Zekun Qi and Jiayu Zeng and Ruixi Yu and
                   Yu Guan and Yintianrun Zhang and Xuchuan Chen and
                   Sikai Liang and Zekai Li and Chenghuai Lin and
                   Xinqiang Yu and Wenyao Zhang and He Wang and Li Yi},
  year          = {2026},
  eprint        = {2608.13555},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2608.13555}
}