{"id":686,"date":"2026-09-16T08:33:31","date_gmt":"2026-09-16T08:33:31","guid":{"rendered":"https:\/\/perit.ai\/blogs\/inside-egocentric-video-training-data-for-physical-ai\/"},"modified":"2026-09-16T14:10:23","modified_gmt":"2026-09-16T14:10:23","slug":"inside-egocentric-video-training-data-for-physical-ai","status":"publish","type":"post","link":"https:\/\/perit.ai\/blogs\/inside-egocentric-video-training-data-for-physical-ai\/","title":{"rendered":"Inside Egocentric Video: Training Data for Physical AI"},"content":{"rendered":"

Robots don’t learn to load a dishwasher by reading about it \u2014 they learn from watching it happen, ideally from the vantage point of the person doing it.<\/p>\n

What \u201cegocentric\u201d actually means<\/h2>\n

A head- or chest-mounted camera captures a task exactly as a human sees it: hands entering and leaving the frame, natural occlusion, the same viewpoint a robot’s onboard camera would eventually have.<\/p>\n

Why perspective matters<\/h3>\n

Third-person footage \u2014 a fixed camera watching a room \u2014 teaches a model what a task looks like from the outside. Egocentric footage teaches it what the task looks like from the position it will actually operate from.<\/p>\n

Capturing it at scale<\/h2>\n

Contributors wear lightweight capture rigs while performing everyday tasks: cooking, folding laundry, assembling furniture. Each session is annotated with the task’s steps and any tools involved.<\/p>\n

Depth and hand pose<\/h3>\n

Paired depth data and hand-pose tracking turn a flat video into a 3D-aware training signal \u2014 critical for a robot arm that needs to reason about where a hand actually is in space, not just where it appears on screen.<\/p>\n

From dataset to demonstration<\/h2>\n

Teleoperation sessions go a step further, pairing a human operator’s actions with the robot’s own sensors \u2014 the closest thing to a labeled demonstration a learning system can ask for.<\/p>\n","protected":false},"excerpt":{"rendered":"

First-person video of real people doing real tasks is quietly becoming the most valuable dataset in robotics.<\/p>\n","protected":false},"author":1,"featured_media":726,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[],"class_list":["post-686","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-physical-intelligence"],"_links":{"self":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/686","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/comments?post=686"}],"version-history":[{"count":1,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/686\/revisions"}],"predecessor-version":[{"id":727,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/686\/revisions\/727"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media\/726"}],"wp:attachment":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media?parent=686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/categories?post=686"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/tags?post=686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}