At frame zero, the umbrella is still closed.
Nothing about its future is uncertain. The simulator knows the position of every rib, the hinge that will rotate, the surface that will stretch and the exact frame in which the canopy reaches its full span. It knows where the camera is. It knows what will be visible when one metal strut passes behind another.
The source umbrella is striped red and white. The instruction asks for something else: an egg-yolk-yellow canopy with a broad navy band, silver-gray ribs and a black handle. The task sounds clean. Keep the opening motion. Replace the appearance.
A modern video model receives the draft and the prompt. The umbrella opens. Its geometry survives. The ribs spread with the right rhythm. The canopy remains attached.
It is still red and white.
This is one of the failure cases in WorldDraft, a benchmark released in November 2026 for a new division of labour in generative video. Code constructs an executable four-dimensional world. A video model is asked to render that world photorealistically. In the authors’ example, Seedance 2.5 follows the complex umbrella motion from a detailed conflicting draft but retains the colours it was explicitly told to overwrite. [1]
The output is not nonsense. That is what makes the error interesting. It is a plausible video of the wrong contract.
The contract
Act I: stop asking one model to invent everything
Text-to-video systems are often described as world models. The phrase suggests a machine that can infer objects, physics, causality, motion, camera behaviour and appearance inside a single generative process. A prompt asks for a forklift disappearing behind warehouse shelving, and the model must invent the forklift, the shelving, the path, the occlusion, the reappearance, the camera and every pixel that makes the result look photographed.
That is an enormous bundle of jobs.
WorldDraft begins by unbundling them. A coding agent or simulation pipeline can build an executable scene in a system such as Blender. Code is good at explicit state: the forklift exists once, follows a declared trajectory and cannot quietly become two forklifts after passing behind a shelf. A renderer is good at surface statistics: painted steel, woven fabric, concrete, soft illumination and the small irregularities that make an image feel real.
WorldDraft split · executable world → draft video → model renders target appearance
In this framing, the video model is not the physicist. It is the cinematographer and finishing department. The code says what happened. The model decides what it should look like.
The distinction resembles a familiar production pipeline. A film’s previs establishes staging and camera movement before final visual effects. A game engine separates simulation state from rendering. A robotics simulator exposes object poses, contacts and trajectories before any domain-randomized texture is applied. WorldDraft turns that production intuition into a controlled evaluation.
The promise is larger than prettier clips. If the executable world remains the source of truth, every generated frame can arrive with an answer key: object identity, projected boxes, visible masks, event boundaries and camera trajectory. Synthetic video stops being an opaque pile of pixels and becomes a traceable transformation of known state.
One world becomes nine drafts
The benchmark contains 48 executable world families across six kinds of difficulty: coordinating multiple entities, multi-stage dynamics, occlusion and reappearance, camera motion, non-rigid deformation and hybrid interactions. The scenes include an urban intersection, an excavator loading a truck, a warehouse forklift, a moving interior camera, an opening umbrella and a marble triggering a field of dominoes. [2]
Each world is rendered nine ways. The first axis controls how much detail the draft provides:
- Abstract proxy, P0: minimal shapes and cues.
- Simulation, P1: a more informative synthetic scene.
- Previs, P2: the richest structural and visual draft.
The second axis controls the relationship between source appearance and target prompt:
- Neutral, AN: source visuals do not strongly compete with the requested appearance.
- Aligned, AA: source and target appearances agree.
- Conflicting, AC: the draft says blue metal while the prompt asks for red plastic, or supplies red-and-white fabric where the target asks for yellow and navy.
That factorial design is the heart of the work. The motion and target prompt stay fixed while one source property changes. Comparing P0 with P2 asks whether extra draft information helps. Comparing aligned with conflicting drafts asks whether the model can distinguish structure that must survive from appearance that must be replaced.
The released dataset makes the contract unusually inspectable. It contains 432 draft videos at 832×480, 81 frames and 16 frames per second, plus preprocessing metadata, prompts, Blender scenes and ground truth exported from those scenes. [3]
At a little over five seconds per draft, the clips are short. Their job is not to simulate an entire film. It is to isolate one failure mechanism cleanly enough to measure.
Two kinds of correctness pull in opposite directions
A generative renderer has to satisfy two masters.
World preservation asks whether the entities, positions, motions, interactions and camera path remain faithful to the draft. If twelve sailboats enter the world in a specified arrangement, twelve should remain. If a forklift passes behind a shelf, it should reappear on the same path. If the camera arcs left, the rendered camera should not suddenly zoom into the nearest object.
Visual realization asks whether the output adopts the requested colours, materials, textures, environment and lighting—and keeps those attributes attached to the correct object over time.
The benchmark’s twelve models do not dominate both sides together. SolarRenderer records the highest world-preservation score in the main table, 63.28, but a lower appearance-realization score of 69.97. Wan 3.0 reaches the highest realization score, 80.40, while falling to the lowest preservation score, 44.37. [1]
Those values are benchmark composites rather than intuitive percentages of perfect videos. Their usefulness is comparative: the system that paints most convincingly may be the system that disturbs the prescribed world most aggressively.
The overall ranking reflects this compromise. The authors weight rendering performance at 50%, gains from draft granularity at 25% and robustness to source-target conflict at 25%. LynnReal-Omni leads at 64.89, only 0.16 points ahead of commercial Seedance 2.5 at 64.73. [4]
A single rank conceals the more valuable result. There is no universal “best video model” here. There are different ways to violate the contract.
Act II: more information makes one problem worse
The obvious prediction is that a richer draft should make rendering easier. Give the model clearer geometry, better surfaces and more complete motion, and it has less to invent.
That prediction is half right.
Moving from abstract proxies toward detailed previs generally improves structural support. The model can see where objects belong and how the world is organized. But the richer draft also carries a stronger visual opinion. Synthetic shading, colour and material become harder to treat as disposable scaffolding.
In the paper’s main comparison, appearance and binding gains are negative for nine of ten models as draft granularity rises. OmniWeaving shows the sharpest example: structure improves, while its reported appearance and binding changes fall by 14.47 and 15.95 points. Seedance 2.5 is the sole exception in that comparison, posting positive changes on both. [1]
The umbrella makes the paradox visible. A detailed draft helps the model preserve the articulated canopy and ribs. The same detail makes its red-and-white stripes stubborn. The renderer is not merely reading geometry from the pixels. It is absorbing an appearance prior that competes with the text.
The draft has two channels hidden inside one image
Some pixels describe structure: edges, depth, identity and motion. Other pixels describe an appearance that should be discarded. Today’s models do not reliably know which channel a pixel belongs to.
This matters beyond generative video. Synthetic training pipelines often assume that adding detail makes a sample more useful. WorldDraft suggests a more careful question: detail about what? A dense input can improve spatial supervision while making domain transfer harder. Fidelity to the source is not always fidelity to the target.
The colour that refuses to leave
WorldDraft calls the most direct version of this error source leakage. Under a conflicting condition, the evaluator asks whether a colour belonging to the source draft remains visible even though the target prompt requires another colour. Lower is better.
Wan2.1-VACE reaches 64.17% source-colour leakage in the paper’s aggregated conflict result. UniVideo reaches 52.24%. The stronger commercial and larger models are not immune: Wan 3.0 is at 27.39%, MiniMax-H3 at 28.64%, LynnReal-Omni at 30.54% and Seedance 2.5 at 41.57%. [4]
These figures do not mean that 64% of every generated frame is wrong or that a model ignores 64% of prompts. The metric applies to audited conflicting source-colour questions in the AC cells. It measures a particular failure: the unwanted source colour surviving when it should have been overwritten.
The failure can take stranger forms. In the twelve-sailboat case, the prompt specifies an ordered 3×4 colour layout. Under conflict, LynnReal-Omni sometimes creates an additional cluster of green sailboats. The model appears to satisfy colour pressure by changing topology—adding entities that the executable world never contained. [1]
That is the synthetic-data equivalent of correcting a spreadsheet total by inserting imaginary rows.
How do you grade a world?
A conventional video benchmark may ask whether a clip looks attractive or matches a caption. WorldDraft has access to something richer: the scene that existed before the clip.
Blender exports visible-instance masks, projected entity boxes, event boundaries and camera trajectories. Generated videos are then evaluated on several separate axes:
| Question | Representative measurement | What it catches |
|---|---|---|
| Are the right entities present? | Exact global and local counts | Missing or invented objects |
| Did objects stay where they belong? | Bounding-box IoU and matched-track coverage | Spatial drift and broken tracks |
| Did the intended interaction occur? | Interaction success and event-boundary coverage | Motion without the required outcome |
| Did the camera follow the draft? | Rotation, translation and camera-motion error | Unprompted zooms and changed paths |
| Was the target appearance realized? | Attribute QA, image quality and text alignment | Wrong colour, material, environment or lighting |
| Did attributes remain attached? | Appearance persistence | Colour or texture changing midway |
| Did the source leak through? | Conflicting source-colour probes | Scaffolding mistaken for final appearance |
This is not a perfect oracle. The repository documents important limitations. Bounding-box scoring uses detector and tracker outputs, with a ground-truth-box fallback when detection fails. Camera trajectories are compared through a ViPE estimate rather than directly against Blender extrinsics. Event timing coverage checks whether boundaries can be localized, not how close every predicted boundary is to the true time. Attribute and event questions are judged by Qwen3.6-27B at fixed settings. [5]
The authors compare automated decisions with majority labels from five independent human judges on sampled cases and report agreement ranging from 87.5% to 94.2%, averaging 91.0%. That supports the evaluation while leaving room for disagreement. It does not turn every automated judgment into ground truth.
The methodological lesson is excellent: when code supplies exact state, use deterministic measurements where possible, model-based judges where necessary and human review to estimate where the judges diverge.
Act III: the fix is a mark on the first frame
The proposed mitigation is surprisingly modest. WorldDraft-Primer does not retrain the video model or require privileged access to its weights. It changes the first draft frame.
One set of cues emphasizes objects that are easy to overlook, reinforcing the topology that should survive. Another places subtle target-attribute cues around entities and supporting surfaces, helping the model bind requested colours and textures without repainting the whole source frame.
On the authors’ benchmark, the primer increases OmniWeaving’s preservation by 2.99 points and realization by 4.56, reduces source leakage by 3.42 points and raises its overall score by 2.51. For MiniMax-H3, realization increases by 3.83 points and the overall score rises from 63.63 to 65.56—above the unmodified leaders in the original table. [1]
Those are results from the benchmark’s authors on two representative models, not proof that every renderer or production pipeline will improve. The larger idea is still compelling: the interface between simulator and renderer may matter as much as the renderer itself.
A draft is not merely a lower-quality video. It is a control surface. Its palette, abstraction, emphasis and first-frame composition influence which parts of the source the model treats as binding.
The real product is the executable contract
WorldDraft points toward a different way to procure synthetic data.
Buying “10,000 generated videos” says almost nothing about whether the desired events occurred, whether identities persisted or whether the camera silently changed. Buying executable scene families plus rendered variants creates an auditable object:
render contract · attributes to preserve · attributes to replace
evidence · draft · generated video · masks · boxes · camera · event log
quality record · deterministic checks · model judgments · human review · failure reason
For one model, the benchmark’s 432 cases produce 432 outputs. Across twelve models, the design implies 5,184 evaluated generations. At five seconds each, that is only 7.2 hours of generated video—but it spans controlled counterfactuals that an unstructured seven-hour collection would almost never contain.
The value is not duration. It is paired variation.
This has a direct physical-intelligence application. A robot-learning team may know the desired trajectory and contact event from simulation but need visually diverse footage for perception or pretraining. A driving system may want the same interaction under different weather, materials and lighting without letting the renderer move vehicles or change the camera. A video model may need to learn that surface appearance is mutable while identity and causality are not.
Perit’s role in such a pipeline would not be limited to drawing boxes after generation. The higher-value work is designing the variation matrix, specifying which state must survive, reviewing model-judge disagreements and delivering failure packs that distinguish topology changes from appearance errors. Existing video-annotation workflows already separate task phases, outcomes and review decisions. Executable-world data adds a machine-readable reference behind those judgments.
A practical workflow for verifiable synthetic video
Declare entities, counts, trajectories, event order and camera motion that no renderer may change.
Cross draft detail with aligned, neutral and deliberately conflicting appearances.
Record renderer, settings, seed, input transformation and complete output—not only selected clips.
Check count, geometry, tracks, events and camera before scoring aesthetic quality.
Sample automated passes and inspect cases where deterministic and learned metrics conflict.
Package source leakage, topology changes and temporal breaks as training and debugging data.
The order matters. If aesthetic scoring comes first, a beautiful wrong world can pass. If the world contract is evaluated first, visual quality becomes meaningful only after the scene has survived.
The umbrella opens again
Return to the first frame.
The simulator knows the umbrella is one object assembled from many moving parts. It knows when the canopy opens. It knows that the red-and-white surface is temporary scaffolding and the yellow-and-navy instruction is the final appearance.
The video model sees pixels.
That gap explains the entire benchmark. Humans can look at a previs and understand an implicit production rule: preserve the choreography, replace the costume. Today’s renderers often treat both as visual evidence of equal authority. A richer draft can make the choreography clearer while making the old costume harder to remove.
WorldDraft does not show that video models cannot represent worlds. It shows that the word world hides several contracts—identity, geometry, dynamics, camera and appearance—and current systems do not reliably keep them separate.
The path forward may be less romantic than asking one model to dream reality from a sentence. Code can keep the books. Simulation can preserve causality. Generative models can supply the visual richness. Evaluation can verify the hand-off.
When those parts work together, the umbrella should not merely open beautifully.
It should open as the same umbrella, in the same world, wearing only the colours it was given.
References and notes
- Z. Hou et al., “WorldDraft: If Code Builds the World, Can Video Models Just Render It?”, arXiv:2611.16217, 12 November 2026. Primary source for the benchmark design, results, failure cases, human-alignment study and WorldDraft-Primer.
- WorldDraft authors, project page and interactive exhibition. Source for the six displayed case families, controlled 3×3 comparisons and downloadable result table.
- WorldDraft authors, dataset card. Source for the Apache 2.0 dataset, video dimensions, frame count, frame rate, Blender scenes and exported ground truth.
- WorldDraft authors, main results table. Model-level preservation, realization, leakage and composite scores. The overall score uses benchmark-specific weights and should not be read as a universal percentage of correct videos.
- WorldDraft repository, metric protocol. Definitions, sampling rules, tracking fallbacks, VLM judging, camera-estimation caveats and aggregation procedure.
- WorldDraft authors, code repository. Evaluation package, data formats and Apache 2.0 licence.