services/motion is an offline pipeline on the Mac. It decodes a short clip, estimates one hand’s landmarks with MediaPipe Hand Landmarker, retargets the finger motion to a simulated Shadow Dexterous Hand (right or left) with dex-retargeting, and renders a labeled replay in MuJoCo.
Frames without a usable hand are kept and reported as invalid. They’re never filled with invented motion. The replay is kinematic: it plays back recorded motion and isn’t a learned policy.
Run it
run writes hand_track.json, motion.json and replay.mp4, which shows the source clip with landmarks next to the robot. Inputs are assumed unmirrored, like Quest passthrough. Pass --mirrored for selfie footage.
As a gateway worker
scalpal.robot_trajectory.v1 trajectory (what the dashboard replay reads), the replay video, the hand estimates and a quality report. No hand found fails without retry. Network errors retry.