BAAI Launches Multimodal Representation World Model Wujie·RoboBrain Orca

Wujie·RoboBrain Orca Quick Summary

Wujie·RoboBrain Orca is a general world foundation model released by Beijing Academy of Artificial Intelligence (BAAI) in 2026. It adopts the next-state prediction concept, learning unified world latent representations through multimodal signals such as vision and language, targeting embodied intelligence scenarios like text generation, image prediction, and robot action generation.

- Model Name: Wujie·RoboBrain Orca

- Developer: Beijing Academy of Artificial Intelligence (BAAI) Orca Team

- Release Time: Technical report "Orca: The World is in Your Mind" released in June 2026.

- Main Functions: Supports multimodal tasks including world state modeling, text generation, future image prediction, and robot action generation.

- Usage Requirements: Code and model resources are being opened gradually according to the project plan, supporting researchers in world model research and downstream adaptation.

- Open Source Status: Project repository is public on GitHub; Orca-0.8B and Orca-4B model checkpoints are planned for release.

- Applicable Scenarios: Robot manipulation, embodied intelligence training, video understanding, physical state prediction, digital twins, etc.

- Technical Features: Uses Encoder-Decoder architecture, constructing a unified world latent space through unconscious and conscious learning.

- Price: Commercial API prices and free quotas are not yet announced; primarily aimed at research and development testing.

Wujie·RoboBrain Orca Core Advantages

- Unified World Representation: Orca learns multimodal information such as vision and language through an Encoder and maps it to a unified latent space, then outputs results via text, image, and action modules. The model is trained using 125K hours of video, 160M event annotations, and VQA data, enhancing cross-modal state understanding capabilities.

- Next-State Prediction: Unlike traditional next-word or next-frame prediction, Orca adjusts the goal to predict future world state changes. In text generation tests, Orca-4B achieved a comprehensive score of 51.8 and 34.2 on TemporalBench, demonstrating dynamic scene understanding capabilities.

- Physical Change Prediction: Orca image prediction focuses on state changes after real interactions rather than simply generating images. In PRICE-V0.1 tests, the Orca-4B version achieved an average score of 59.8±10.9, used to evaluate real-world interaction prediction capabilities.

- Robot Generalization Capability: Orca completes task adaptation by freezing the backbone model and training only the action output module. In robot environment and object change tests, the overall rule score reached 32.4, verifying the support capability of world representation for action generation.

- Scale Expansion Capability: Experiments show that as training data and model scale increase, Orca's world learning loss continues to decrease, and downstream text, image, and action tasks improve synchronously, indicating expansion potential for world state modeling.

Wujie·RoboBrain Orca Main Functions

- World State Modeling: Orca converts inputs such as video and language events into a unified world state.

- Video Understanding Q&A: Orca completes visual question answering tasks through the language output module. Tests cover MVBench, TemporalBench, 3DSRBench, and SWITCH, where Orca-4B achieved an average score of 51.8, usable for video event analysis.

- Future Image Prediction: Orca predicts future states based on current images and language conditions.

- Robot Action Generation: Orca supports generating robot action trajectories from world states. Test tasks include five types of dual-arm robot operations: picking up books, stacking bowls, drawing paper, stamping, and scooping sugar.

- Agent Environment Understanding: Orca provides a unified world representation interface, connectable to language, vision, and action modules, allowing agents to reason, plan, and execute based on environmental changes.

Wujie·RoboBrain Orca Technical Principles

- Encoder-Decoder Architecture: Orca uses an encoder and decoder structure. The encoder learns world latent states, and the decoder is responsible for outputting text, images, and actions, enabling one model to support multiple tasks.

- Unconscious Learning: The model uses 125K hours of continuous video to learn natural state changes, establishing basic world knowledge by observing object motion, occlusion relationships, and environmental changes.

- Conscious Learning: The model combines 160M event annotations and VQA data to learn event relationships, task goals, and language descriptions, making state predictions contain semantic information.

- State Transition Prediction: Orca focuses on predicting the "next state," considering both implicit dynamic factors and explicit conditions, such as physical laws, environmental changes, and user instructions.

- Frozen Backbone Training: After pre-training, the world model backbone is frozen, and only lightweight output modules are trained, reducing downstream task adaptation costs.

Wujie·RoboBrain Orca Comparison with Mainstream Models

In the officially announced VQA benchmark tests, Orca-4B achieved a comprehensive score of 51.8, making it the best-performing 4B-level vision-language model in this comparison. Compared to Qwen3.5-4B, Orca leads by 9 points in TemporalBench temporal reasoning tests, by 4 points in 3DSRBench spatial understanding tests, and by 12.8 points in SWITCH tests. Capability analysis shows Orca exceeds Qwen3.5 in four core capabilities: state transition understanding, dynamic motion understanding, common sense reasoning, and spatial relationships. State transition understanding improved by 12.27%, and dynamic motion understanding improved by 8.52%, reflecting the advantages of the next-state prediction training paradigm in temporal reasoning, physical change understanding, and embodied intelligence scenarios.

How to Use Wujie·RoboBrain Orca

- Obtain Project Resources: Visit the Orca official project page or GitHub repository to view technical reports, model architecture, and latest release updates. The paper and project code repository are currently public, while model weights and inference code are being opened gradually.

- Understand Model Architecture: Read technical documentation to understand core concepts such as Next-State-Prediction, unconscious learning, conscious learning, and world latent representation, facilitating understanding of the model's working mechanism in text, image, and robot tasks.

- Wait for Model Release: According to the project roadmap, Orca-0.8B and Orca-4B model weights will be opened gradually. Developers can follow GitHub updates to obtain inference code, training scripts, and example cases.

- Perform Downstream Adaptation: After the model is opened, it can be adapted for text generation, future state prediction, visual question answering, or robot control tasks, completing specific application deployment through lightweight readout modules.

- Conduct Embodied Intelligence Research: Researchers can explore applications and verification in directions such as robot planning, environment prediction, agent training, and digital twins based on Orca's unified world representation capabilities.

Wujie·RoboBrain Orca Related Resources

- Project Official Website: https://orca-wm.github.io/

- GitHub Repository: https://github.com/orca-wm/Orca

- HuggingFace Model Library: https://huggingface.co/papers/2606.30534

- arXiv Technical Paper: https://arxiv.org/pdf/2606.30534

Wujie·RoboBrain Orca Limitations

- Model Resource Limitations: The current code repository is public, but complete model weights for Orca-0.8B and Orca-4B are not yet fully opened, so ordinary users cannot directly deploy the complete model temporarily.

- Insufficient Parameter Information: Public materials have not yet disclosed complete parameter scale, context length, API prices, and other information, so commercial deployment costs still await subsequent materials.

- Limited Real-Scenario Verification: Current robot tests focus on five types of dual-arm tasks. Although the OOD rule score reached 32.4, complex environments such as industrial robots and autonomous driving still require further verification.

Wujie·RoboBrain Orca Typical Application Scenarios

- Robot Manipulation: Input camera footage and task instructions; the model analyzes object states and generates operation actions, outputting behaviors such as grasping and moving, used to improve robot environmental adaptability.

- Embodied Intelligence Training: Input robot execution data and environmental information; learn world states to reduce repetitive data collection, helping agents complete different tasks.

- Video Understanding: Input long videos and question descriptions; the model analyzes event changes and generates answers, suitable for video analysis and event reasoning.

- Digital Twins: Input current environmental state and change conditions; the model predicts future states, used for industrial simulation and virtual testing.

- Agent Planning: Input visual and language environmental information; the model provides world state representation, helping agents complete complex task planning.

Wujie·RoboBrain Orca FAQ

How to use Wujie·RoboBrain Orca?

Wujie·RoboBrain Orca is mainly used for research deployment. It requires preparing video, language, or robot data and connecting corresponding output modules. Current model resources are being opened gradually; actual usage requires following project updates.

How is Wujie·RoboBrain Orca billed?

Wujie·RoboBrain Orca currently has no public API prices or free quotas, released mainly as a research project. Deployment costs depend on computing equipment and subsequent service forms.

Which is better, Wujie·RoboBrain Orca or V-JEPA 2.1?

The two have different directions. Orca focuses on world state prediction, while V-JEPA 2.1 focuses on visual representation. Orca's robot OOD rule score is 32.4, higher than V-JEPA 2.1's 17.0, making it more suitable for embodied intelligence research.

Does Wujie·RoboBrain Orca support real-time robot control?

Orca supports action generation research, but there is currently no public real-time control latency data. Actual robot deployment requires optimization combining control systems, hardware performance, and task requirements.

Does Wujie·RoboBrain Orca have a free quota?

There is currently no public free quota information. Users can follow the project repository to obtain the latest code and resources. Before deployment, computing equipment must be prepared and model open licenses confirmed.

© Copyright Notice

The articles on this site are copyrighted by AI Toolbox. Any form of reprinting without permission is prohibited.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.