Depth, aligned to the frame it came from.
Egocentric capture with a depth channel — stereo headsets, LiDAR-equipped phones or custom stereo rigs — so a model can learn where things are, not only what they look like.
- Environment
- Living room, afternoon light
- Captured with
- Stereo passthrough headset, RGB + aligned depth
- Task
- Pack a mug into a box and close the flaps
Perit has shipped 4,000+ hours of speech through this bench. Physical-AI capture runs on the same recruiting, consent and QA machinery, and the first collections are being scoped with early-access partners now. Nothing on these pages is a delivered volume — the first hours will be listed here the way our speech hours are, after they ship.
What gets captured, and why it is worth having.
The same tasks and rooms as egocentric capture, with a second stream: a per-frame depth map aligned to the RGB image, metric where the sensor supports it. Hand–object distance, container interiors, the gap between a plate and the shelf it is going onto — the geometry a monocular camera has to guess.
Rigs range from a consumer stereo headset to a LiDAR phone to a calibrated stereo pair; calibration files ship with every episode so the depth can be trusted downstream.
- Depth estimation and 3D scene understanding
- 3D hand and object pose
- Spatial reasoning for manipulation
- Sim-to-real gap measurement
- Primary rig
- Stereo passthrough headset (Quest-class)
- Also
- LiDAR phone — RGB plus depth (iPhone Pro-class)
- Also
- Custom calibrated stereo camera rig
- Also
- RealSense-class depth camera on a head mount
- Video
- H.264 in MP4, RGB stream
- Depth
- Per-frame depth map aligned to RGB, sensor-native resolution
- Resolution
- 1080p RGB, 4K on request
- Frame rate
- 30 fps
- Calibration
- Intrinsics and extrinsics per rig, per session
- Episode
- 2–10 minutes
Where it is recorded.
What people are asked to do.
Tasks are recorded where they actually happen — not in a studio dressed to look like a kitchen. Hover a room to see the kind of setting the protocol calls for.
- RGB video and aligned depth per episode
- Calibration files per rig
- Task and environment metadata per episode
- QC status and reviewer chain per episode
- Consent reference on every file
- Quality report per batch: accepted, rejected, and why
- Weekly drops to your bucket, region pinned at kickoff
- 3D hand pose ground truth
- Object masks with depth
- Scene graphs
- Captions
Labelled on the same bench as our speech work, to a written guide, behind the same calibration test.
Small first, on purpose.
- 01Briefday 0
The task family, the environments, the rig, the sensors, the episode count. One page, agreed before anything is bought or anyone is recruited.
- 02SOP and kitweek 1
A written capture protocol — framing, lighting, where a task starts and ends, what counts as a failed take — and the kit list. Operators are trained and tested on it before a single episode is paid for.
- 03Pilot episodesweek 2
A small batch, ten to thirty episodes, delivered with QC status and a quality report. This is where the protocol breaks — on purpose, and cheaply.
- 04Review and scaleongoing
You review the pilot, we fix the protocol, then the run scales on a weekly delivery cadence with the same report attached to every batch.
Scope a depth pilot.
The task family, the environment, the rig and the episode count — one page, and we come back with a protocol and a quote.