WORLDCAST

Exploring Intelligent Visual Generation
in World State Prediction

  • Wenhao Yan1,*
  • Haodong Li2
  • Xiaotong Li3,†
  • Yaxin Luo5
  • Zhengrong Yue6
  • Jialv Zou7
  • Xu Huang4
  • Haojin Yang4
  • Nan Jia7
  • Yan Feng3
  • Peng Pei3
  • Xunliang Cai3
  • 1THU
  • 2CUHK
  • 3Meituan LongCat
  • 4PKU
  • 5MBZUAI
  • 6SJTU
  • 7HUST

*Work done during internship at Meituan.†Project Lead.

Paper Code Full Benchmark Case Gallery BibTeX

01Overview

WorldCast evaluates whether visual generative systems can forecast successive future world states from multimodal observations and heterogeneous references. The benchmark targets L3–L4 capabilities, where understanding and generation must be tightly integrated to model physical changes, action sequences and visual reasoning.

320

manually designed cases

1,352

requested visual states

8

task categories, L3 and L4

12

image and video models

Figure 1. (a) The visual generative capability tree organizes intelligent visual generation into a four-layer hierarchy (L1–L4). (b) WorldCast targets L3–L4 capabilities of dynamically transforming real-world context into successive future world states through unified multimodal understanding and generation.
Abstract

Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of linguistically structured planning and reasoning while also being deeply unified to faithfully simulate real-world states.

02Task Example

An In-Context Procedural Learning case (L4). The model must interpret the manual, plan the assembly order, and render each intermediate state while preserving the work surface and component appearance.

Adapted from Figure 2 of the paper. GPT-Image-2.5 completes the key milestones; Wan 3.0 fails to follow the assembly order.

03Task Categories

Categories are distinguished by where the information governing a future process is encoded and what reasoning is required to translate it into visual state transitions.

Figure 3: representative examples of the eight categories
Figure 3. Representative examples of the eight WorldCast categories. The benchmark comprises 320 manually designed cases with 1,352 requested visual states.

04Leaderboard

Twelve image and video generation systems evaluated under the same state-sequence protocol. Scores range from 0 to 100; higher is better.

GPT-Image-2.5 (Sunburst) is paired with GPT-6 Astra and GPT-Image-2 with GPT-5.5. The table reports baseline systems without agentic enhancement. Each model uses its native output interface: interleaved, grid, video, or plan & iterate.

05Findings

Finding 1

Frontier systems exhibit early-stage L4-level visual state prediction.

GPT-Image-2.5 scores above 60 on all five L4 categories and retains 71.6 on L4, only 2.2 below its L3 score. The best open-source model, SenseNova-U1, drops from 24.4 to 15.0: performance gaps widen at higher levels.

Finding 2

Strong generators draw on their underlying language models.

Pairing Nano Banana Pro with GPT-6 Astra as the enhancer improves every category, raising its overall score from 42.8 to 61.2, yet it still trails the GPT-Image family. Replacing GPT-5.5 with GPT-6 for GPT-Image-2 yields only a marginal gain, with a decline in Physical Simulation.

Finding 3

Interleaved generation helps, but accumulated context remains a challenge.

Switching GPT-Image-2.5 from interleaved to grid output reduces its overall score by 22.7 points, most sharply on Logic and Rules (74.3 → 23.0). Across four generation steps, weaker models degrade as context accumulates, while GPT-Image-2.5 remains stable.

Qualitative failure patterns

Models exhibit distinct failure patterns across the three evaluation dimensions; scores are on a 0–3 scale.

Figure 4. Qualitative examples on WorldCast.

06Towards Agentic Visual Generation

We build two Codex-based harnesses with the same generation budget: an inspect-and-revise loop without coding tools, and a coding-first loop that constructs and inspects intermediate visual artifacts before generation.

Iterative refinement and visual coding raise the ceiling.

The inspection loop lifts GPT-Image-2.5 from 72.8 to 80.2 and GPT-6-enhanced Nano Banana Pro from 61.2 to 72.5. Coding tools push the best overall score to 82.4, with the largest gains on Visual Goal Realization, Logic and Rules, and Physical Simulation, where code makes intermediate states explicit.

82.4best overall score
coding-first inspection loop
+20.0GPT-Image-2.5 on Visual
Goal Realization
Figure 8. Visual coding as procedural guidance. Left: results with different model harnesses. Right: the agent simulates the full steps, notices that its first attempt without the simulation fails at step 3, then regenerates by referencing the simulation result together with the other steps.

08Comparison

Pairs of complete sequences with a clear difference on one metric. Each pair states only which side is better.

09Evaluation

Image grids, interleaved images and videos are all evaluated as ordered sequences of keyframes along three complementary dimensions.

TC

Task Completion

Whether the sequence visibly realizes the intended task: a weighted average of Progress Alignment and Outcome Completion.

CP

Causal Plausibility

Whether transitions follow physical laws, logical rules and task-specific dynamics, penalizing impossible transformations or reversed cause and effect.

VC

Visual Consistency

Whether identity, count, geometry, scale, material, camera and background persist where the task does not require change.

1

Agentic video adapter

GPT-6 Astra selects task-aligned keyframes from generated videos, following an inclusion-first principle.

2

Human-annotated hints

Judge-only hints identify task-critical objects, milestones and constraints.

3

LMM-as-a-judge

GPT-5.5 rates TC, CP and VC on a 0–3 scale for every output sequence.

4

Final score

Rescale(α1TC + α2CP + α3VC)

Citation · BibTeX
@article{yan2026worldcast,
  title   = {WorldCast: Exploring Intelligent Visual Generation in World State Prediction},
  author  = {Yan, Wenhao and Li, Haodong and Li, Xiaotong and Luo, Yaxin and Yue, Zhengrong and Zou, Jialv and Huang, Xu and Yang, Haojin and Jia, Nan and Feng, Yan and Pei, Peng and Cai, Xunliang},
  journal = {arXiv preprint},
  year    = {2026}
}