HandUMI: An 8x Cheaper Way to Collect Data for Bimanual Robots
Hand-worn, open-source device for collecting bimanual manipulation data without a robot in the loop (~$110 per unit plus the VR headset of your preference). Interchangeable gripper tips make demonstrations retargetable across parallel-jaw robot arms. Built and presented at Founders, Inc.'s Off Season II in San Francisco.
Technologies
Overview
HandUMI is a hand-worn, open-source variant of the Universal Manipulation Interface (UMI) for collecting bimanual manipulation data without a robot in the loop, designed for robot arms with parallel grippers. It mounts on the operator's thumb and index/middle fingers, opens and closes with a natural pinch, and uses interchangeable gripper tips to target different parallel-jaw robot grippers. One unit costs roughly $110 in parts, plus the VR headset of your preference (PICO 4 Ultra or Meta Quest 3).
Traditional leader-follower teleoperation is expensive (at least four arms for bimanual data) and lab-constrained (the follower arm must physically move to wherever data is collected). HandUMI removes both constraints by moving the collection interface onto the operator's hand: demonstrations are captured directly from human motion, anywhere, and later retargeted across bimanual arms with the [HandUMI software](https://github.com/robonet-ai/handumi-sw). Current target tips include AgileX Piper, ARX X5, Dream Gripper (TRLC), Trossen WidowX AI, and the original UMI gripper.
I built HandUMI during Off Season II at Founders, Inc. (Fort Mason, San Francisco) and presented the full system (hardware, software and VR app) at the program's festival in August 2026.
Key Features
- No robot in the loop: an 8x cheaper way to collect data for bimanual robots, anywhere
- Modular design: only the detachable gripper tip changes per robot
- Direct gripper-width sensing with a Feetech servo encoder (no fiducials or segmentation)
- SE(3) wrist pose from a VR headset and its controllers (PICO 4 Ultra or Meta Quest 3), no offline camera SLAM
- Wrist-view fisheye camera for the observation used in training and deployment
- Fully open source: CAD, STLs and bill of materials (~$110 per unit in parts)
Reflection
Scalable real-world data is the main bottleneck in robot learning. HandUMI attacks it by making high-quality bimanual demonstrations cheap to collect anywhere, with no lab and no robot on site, so datasets can grow at the pace of human activity rather than robot availability.