Robots don’t learn to load a dishwasher by reading about it — they learn from watching it happen, ideally from the vantage point of the person doing it.
What “egocentric” actually means
A head- or chest-mounted camera captures a task exactly as a human sees it: hands entering and leaving the frame, natural occlusion, the same viewpoint a robot’s onboard camera would eventually have.
Why perspective matters
Third-person footage — a fixed camera watching a room — teaches a model what a task looks like from the outside. Egocentric footage teaches it what the task looks like from the position it will actually operate from.
Capturing it at scale
Contributors wear lightweight capture rigs while performing everyday tasks: cooking, folding laundry, assembling furniture. Each session is annotated with the task’s steps and any tools involved.
Depth and hand pose
Paired depth data and hand-pose tracking turn a flat video into a 3D-aware training signal — critical for a robot arm that needs to reason about where a hand actually is in space, not just where it appears on screen.
From dataset to demonstration
Teleoperation sessions go a step further, pairing a human operator’s actions with the robot’s own sensors — the closest thing to a labeled demonstration a learning system can ask for.
