Physical Intelligence · Teleoperation

An operator, a robot, and a task the policy has to learn.

Trained operators drive leader–follower arms or VR-controlled robots through repeatable tasks. Every episode logs synchronised camera streams with joint states and actions — the data a policy is actually trained on.

What it is

What gets captured, and why it is worth having.

Environment
Small robotics workshop, LED work lamp
Captured with
Bimanual leader–follower arms, two camera views
Task
Stack three bowls
Small robotics workshop, LED work lamp · Bimanual leader–follower arms, two camera views

Teleoperation is the embodiment-matched end of the spectrum: the demonstration is performed on the robot the policy will run on, so there is no transfer gap to close. It is slower and dearer per episode than human video, which is why the protocol matters — every episode has to count.

Operators pass the same calibration gate as the rest of the bench, on the task and the rig, before a paid episode is recorded. Interventions and failed takes are logged, not discarded, because they are training data too.

What it trains
  • Behaviour cloning and diffusion policies
  • Policy evaluation and intervention data
  • Recovery-from-failure demonstrations
  • Embodiment-matched fine-tuning
The spec

How it is captured, and what you receive.

How it is captured
Primary rig
Bimanual leader–follower arms (ALOHA-class)
VR rig
Headset and controllers driving a single or dual arm
Your robot
Bring-your-own-stack, driven by our trained operators
Cameras
Wrist, front and overhead RGB, 2–4 synchronised streams
Logging
Joint state, gripper state and leader commands per step
Operators
Calibrated on the task and the rig before a paid episode
What you receive
Cameras
2–4 synchronised RGB streams, 480×640 to 1080p
Frame rate
30 fps
State
Joint positions, velocities and gripper state per step
Actions
Leader-arm or controller commands per step
Format
LeRobot- or RLDS-style layout on request; HDF5 or Parquet
Episode
20 seconds – 3 minutes
Environments and tasks

Where it is recorded, and what people are asked to do.

Tasks are recorded where they actually happen — not in a studio dressed to look like a kitchen. Hover a room to see the kind of setting the protocol calls for.

Kitchens

Indian, European and Brazilian home kitchens, lived-in and cluttered, plus small commercial kitchens and pantries.

Cooking preparationWashing upCupboards and drawersAppliancesPouring and scooping

Living rooms and bedrooms

Real Western homes in Spain, Germany, France and Italy, and apartments in India, Brazil, Japan and Korea. Sofas, wardrobes, shelves, laundry.

Tidying and foldingMaking bedsShelving and storageLaundryFurniture interaction

Offices

Open-plan and small offices: desks, pantries, meeting rooms, filing and cable clutter.

Desk tasksDocument handlingCables and peripheralsPantry tasksDrawers and storage

Retail and small shops

Kirana stores, groceries, pharmacies and supermarket aisles, with real stock and real customers out of focus.

Shelf restockingPicking to a listBagging and countersInventory checksCrates and boxes

Warehouses and light industrial

Racking aisles, packing benches, courier depots and small production lines with pallets, trolleys and parts bins.

Pallet to shelfParcel sortingPacking and tapingParts movementInspection

Workshops and labs

Mechanical and electronics workshops, repair benches and robotics labs where a follower rig can be set up.

Tool handlingAssembly and fasteningPegboards and racksBench tidyingTeleoperation setups

Agriculture and outdoor

Polytunnels, packing sheds, market stalls and yards, in daylight that changes from one episode to the next.

Sorting produceCrates and traysPruning and pickingWeighing and baggingLoading

Client-defined and confidential

Your site, or a space recreated to your drawings, run by an NDA'd crew with limited-visibility workflows and no clip leaving your region.

Proprietary workflowsRecreated spacesStealth programsOn-site with your rigYour robot, our operators
What ships

Deliverables, and the labels you can add.

Deliverables
  • Synchronised camera streams per episode
  • State and action logs per step
  • Task and environment metadata per episode
  • QC status and reviewer chain per episode
  • Consent reference on every file
  • Quality report per batch: accepted, rejected, and why
  • Weekly drops to your bucket, region pinned at kickoff
Annotation add-ons
  • Success and failure labels per episode
  • Sub-task boundaries
  • Language instruction per episode
  • Operator interventions flagged

Labelled on the same bench as our speech work, to a written guide, behind the same calibration test.

How a pilot runs

Ten to thirty episodes, then the volume.

  1. 01day 0
    Brief

    The task family, the environments, the rig, the sensors, the episode count. One page, agreed before anything is bought or anyone is recruited.

  2. 02week 1
    SOP and kit

    A written capture protocol — framing, lighting, where a task starts and ends, what counts as a failed take — and the kit list. Operators are trained and tested on it before a single episode is paid for.

  3. 03week 2
    Pilot episodes

    A small batch, ten to thirty episodes, delivered with QC status and a quality report. This is where the protocol breaks — on purpose, and cheaply.

  4. 04ongoing
    Review and scale

    You review the pilot, we fix the protocol, then the run scales on a weekly delivery cadence with the same report attached to every batch.

Turnaround
48 hrs
Written spec and quote
5 days
Sample episode in your format
2 weeks
Pilot batch with its QA report
Weekly
Delivery cadence once the pilot clears
Before an episode counts

What QC checks, every time.

  • Framing

    The task stays in frame; hands and workspace inside the protocol's bounds for the whole episode.

  • Exposure and focus

    No blown highlights or focus hunting beyond the tolerance set for the room.

  • Motion

    No camera whip, no dropped frames; IMU continuous where the rig has one.

  • Sync

    Every stream aligned within one frame; timestamps monotonic across cameras, state and action logs.

  • Task compliance

    Start and end states match the SOP; no skipped or reordered steps unless the protocol allows it.

  • Duration

    Inside the episode window for the task family; overruns are trimmed to the protocol, not the clip.

  • Metadata

    Environment, task, rig, operator id, session and consent reference present and valid on every episode.

  • Privacy

    No bystander faces, screens or documents without consent; blurred where the protocol flags it.

  • Calibration

    Intrinsics and extrinsics on file per rig and session for depth, stereo and multi-camera capture.

  • Labels

    Where annotation is ordered, agreement against the gold set above the threshold before the batch ships.