#4
GMT
HumanScore 37.09
HumanTracker Towards Comprehensive and Human-Aligned Motion Tracking Benchmark
1Nankai University
2Tsinghua University
3Galbot
4Shanghai Jiao Tong University
5Peking University
6Shanghai Qi Zhi Institute
*Equal contribution †Corresponding authors
Tracking quality should agree with what people see.
Kinematic errors average frame-level pose differences, but often miss the physical artifacts that matter most in video: unstable support, foot skating, incorrect contact timing, and failed recoveries.
HumanTracker addresses both sides of evaluation. Its benchmark supplies broad, diagnostic motion coverage, while HumanScore learns trajectory-level quality from human preferences and predicts which rollout people prefer.
90.83%
alignment with human preferences on the test set
6.78 points above the strongest conventional diagnostic.
153hours of optical motion
25Klabeled motion clips
24professional performers
4diagnostic motion families
HumanScore learns the temporal evidence behind human judgments.
6,000 original pairs. Six expert annotators provide strict preferences, similar judgments, and cannot-compare judgments. Bilateral mirroring yields 12,000 preference records.
Five-second context. Each 250-frame window is encoded as a 539-dimensional frame-token sequence with explicit padding masks.
Readable output. Window rewards pass through a sigmoid and are averaged from 0 to 100, weighted by the number of actual frames.
Test set preference alignment
HumanScore90.83
KPT position MAE84.05
Joint velocity error84.04
MPJPE80.49
Foot contact accuracy78.82
Family-balanced alignment rate (%) on original, unmirrored strict-preference comparisons from the motion-disjoint test set.
What the model uses
See what HumanScore sees.
Each grid synchronizes four zero-shot rollouts of the same reference motion. A failed rollout holds on its final frame while the remaining trackers continue.
#4
#1
#3
#2
#4
#1
#3
#2
#4
#1
#3
#2
#4
#2
#1
#3
#4
#1
#2
#3
#3
#2
#1
#4
#4
#1
#2
#3
#4
#1
#2
#3
#4
#3
#2
#1
#3
#2
#1
#4
#4
#3
#1
#2
#3
#4
#1
#2
Four families expose different failure regimes.
Every clip includes a family label, a natural-language description, a fitted SMPL sequence, and a robot-space reference trajectory.
Daily
Steady locomotion, turning, and routine gestures reveal stability and residual drift.
- Hours
- 89.29
- Clips
- 9,739
Highly Dynamic
Jumps, kicks, acrobatics, and fast footwork stress impact handling and phase timing.
- Hours
- 11.01
- Clips
- 2,676
Interaction
Object and environment interactions require hand, arm, and whole-body coordination.
- Hours
- 47.78
- Clips
- 10,940
Ground
Low postures and multi-contact transitions test contact geometry, balance, and recovery.
- Hours
- 4.59
- Clips
- 1,640
One score, used with complementary diagnostics.
HumanScore measures perceived quality.
It summarizes trajectory-level preference over contact, stability, smoothness, and reference fidelity. Higher is better within the same evaluation setting.
Success measures completion.
A rollout may look accurate before an early failure. Success prevents a short failed trajectory from being mistaken for robust tracking.
Kinematic errors diagnose pose mismatch.
MPJPE and related errors remain useful for locating specific problems; HumanScore is not a replacement for every analytic diagnostic.
HumanTracker
@misc{liu2026humantrackercomprehensivehumanalignedmotion,
title = {HumanTracker: Towards Comprehensive and Human-Aligned
Motion Tracking Benchmark},
author = {Dairu Liu and Zekun Qi and Jiayu Zeng and Ruixi Yu and
Yu Guan and Yintianrun Zhang and Xuchuan Chen and
Sikai Liang and Zekai Li and Chenghuai Lin and
Xinqiang Yu and Wenyao Zhang and He Wang and Li Yi},
year = {2026},
eprint = {2608.13555},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.13555}
}