Alibaba Tongyi Launches Physical World Foundation Model Suite Qwen-Robot Suite

Qwen-Robot Series: Thinking, Walking, and Acting, Taking the First Step into the Physical World

Qwen series multimodal foundation models have made significant progress in physical world understanding. Qwen-VL can parse complex spatial relationships, identify objects in cluttered scenes, execute multi-step instructions, and reason about physical configurations—laying the cognitive foundation for physical agents. Vision-language models can already plan at the language level: "Go to the kitchen, find the red cup, pick it up, and put it on the shelf."

However, understanding the physical world and acting within it are two different things. A model capable of planning the above steps cannot generate the motion commands to truly execute them. This is essentially an alignment problem—language instructions and physical actions exist in different representation spaces, and perception alone cannot bridge this gap. A greater challenge lies in the fact that embodied data differs vastly from internet text—naturally heterogeneous, high collection cost, and limited diversity. A navigation trajectory, a teleoperated grasp, and a driving record each exist in different action spaces, observation formats, and robot morphologies. Simply mixing them together yields conflict rather than synergy.

The Qwen-Robot Suite bridges this gap with three foundation models: Qwen-RobotNav, Qwen-RobotManip, and Qwen-RobotWorld, which align language with different types of physical actions respectively. We have made significant progress in both instruction generalization capability and physical law consistency. This article will also explore the possibility of using these models as underlying tools to build general agent systems.

- Qwen-RobotNav: The Action Entry Point for Physical Agents — Through controllable observation encoding and tool interfaces, it connects vision-language capabilities to motion control, unifying four types of tasks: instruction following, point/goal navigation, target tracking, and autonomous driving

- Qwen-RobotManip: The Interaction Cornerstone for Physical Agents — Through standardized state-action spaces and end-effector incremental poses in camera coordinates, it connects vision-language capabilities to manipulation control, achieving large-scale multi-embodiment training based on a corpus of >38,100 hours built entirely from open-source data

- Qwen-RobotWorld: The Infinite World for Physical Agents — Through a natural language action interface, it connects vision-language capabilities to world dynamic prediction, allowing the same world model to predict physically consistent futures across manipulation, driving, and navigation scenarios.

Qwen-RobotNav Physical Agent Action Entry Point

For an agent to manipulate any object, it must first be able to reach the target location. However, different navigation tasks have vastly different requirements for historical information—instruction following requires retaining long-range context, while target tracking focuses almost exclusively on the most recent few frames. No fixed visual allocation strategy can satisfy both needs simultaneously.

The core idea of Qwen-RobotNav is to parameterize the visual allocation strategy itself: task mode selects navigation behavior (instruction following, target search, target tracking, autonomous driving), and adjustable parameters (visual token budget, temporal decay, single-camera weight, frame sampling mode) determine the encoding method of visual history. The model is trained on 15.6 million samples, jointly with vision-language data to retain perception capabilities, with one set of weights unifying five navigation tasks.

This parameterized interface also makes Qwen-RobotNav naturally suitable as a navigation module in agent systems—the upper-level planner decomposes long-range goals into sub-tasks, dynamically switching task modes and context strategies during execution, combining complex behaviors by repeatedly calling the same model. Thus, the entire system possesses persistent memory and long-range reasoning capabilities, capable of handling complex tasks such as multi-step navigation, environmental forensics, and evidence-based Q&A.

Core Highlights:

- Unified Multi-Domain Navigation: Single model, single set of weights, achieving SOTA in 5 navigation domains — VLN-CE RxR 76.5% SR, HM3Dv2 Target Search 75.6% SR (RGB only, surpassing all depth methods), EVT-Bench 90.0% Tracking Rate, NAVSIM 91.4 PDMS, and new records on 3 EQA benchmarks — consistent improvement from 2B to 8B parameters

- Controllable Observation Protocol: Four control axes (visual token budget, temporal decay, per-camera weight, frame sampling mode) exposed as inference-time parameters, randomized per sample during training, adaptable to any configuration without retraining or modifying the Qwen3-VL architecture

- Agent Navigation System: As a reconfigurable navigation primitive in a two-layer system, the upper-level planner (Qwen3.6-Plus) decomposes long-sequence goals and schedules configurable navigation calls, paired with a two-layer memory mechanism — improving by 15.4% on EXPRESS-Bench, reducing navigation steps by 77%

- Open Environment Generalization: Zero-shot deployment on Unitree Go2 quadruped robot, using only a single low-resolution camera, demonstrating strong generalization capabilities in open real-world environments with free natural language instructions, without any environment-specific fine-tuning

Target Tracking:

https://www.zhihu.com/video/2050196745309656543

Instruction Following 1:

https://www.zhihu.com/video/2050196786678178055

Instruction Following 2:

https://www.zhihu.com/video/2050196827610277409

Qwen-RobotManip Physical Agent Interaction Cornerstone

Physical agents need to interact with the real world, such as completing manipulation tasks via robotic arms. However, industrial arms on production lines and service arms in kitchens may perform visually similar grasping actions, but their joint configurations and action spaces are completely different. The core challenge is to achieve compatibility at the representation level for robots of varying morphologies, so that large-scale training across robots and data sources produces synergy rather than conflict.

Qwen-RobotManip uses Qwen3.5-4B VL as the backbone, combined with a flow matching DiT action head, addressing this through three mechanisms. A unified 80-dimensional state-action representation is shared across embodiments such as single-arm, dual-arm, dexterous hands, and mobile platforms. End-effector incremental pose actions in camera coordinates make visually similar motions numerically close across different robots, thereby masking morphological differences. Context strategy adaptation treats execution history as an implicit embodiment identifier, achieving online behavior calibration during inference.

Once the representation system is unified, data barriers are lowered. We utilized 11,320 hours of open-source robot data, 1,933 hours of open-source first-person human video, and a human-machine transfer data synthesis pipeline to synthesize 24,808 hours of robot data across 15 embodiments from human video (totaling >38,100 hours) to train the VLA model. Relying solely on open-source data, the model emerges with generalization capabilities such as robustness to perturbations, zero-shot instruction following, autonomous error recovery, and cross-embodiment transfer.

- Unified Cross-Embodiment Alignment Framework — Unified 80-dimensional state-action representation compatible with diverse embodiments, end-effector incremental pose in camera coordinates makes visually similar motions numerically close, context strategy adaptation treats execution history as implicit embodiment identifier — together achieving consistent signal extraction across embodiments

- Large-Scale Human-Machine Transfer Data Synthesis — Pipeline converts 1,933 hours of first-person human video into 24,808 hours of robot demonstrations across 15 embodiments via action retargeting, hand removal and inpainting, simulation rendering, and depth-assisted fusion, paired with a multi-stage data curation pipeline to ensure data quality

- Out-of-Distribution Generalization: LIBERO-Plus 91.4% (surpassing π0.5 +7.0), RoboTwin-Clean2Rand Hard 69.4% (surpassing π0.5 +21.5), RoboCasa365 Composite-Unseen 14.9% (3x the second place), EBench 45.6% (surpassing second place +18.5); RoboTwin-IF (self-built zero-shot instruction following evaluation) 72.0% (surpassing π0.5 +22.4), confirming reliable language-conditioned control capabilities; RoboTwin-XE (self-built zero-shot cross-embodiment evaluation) zero-shot cross-embodiment transfer performance reaches 3x the previous best

- Real-World Performance: Ranked first in RoboChallenge Table30 v1 General Track with 45% SR, 20% performance improvement over previous SOTA; achieved 2x the previous best on in-domain and out-of-domain tasks, few-shot adaptation, and cross-embodiment skill transfer on real robot platforms

Key Finding — Alignment is the prerequisite for scaling. Only models with unified cross-embodiment representation (UnifiedSpace + UnifiedEEF) can exhibit stable log-linear data scaling curves. Without alignment, adding more data only produces jittery or flat curves—scaling cannot compensate for the inherent defects of misaligned representation frameworks.

Clothes Storage:

https://www.zhihu.com/video/2050196898884198720

Artistic Flower Arrangement:

https://www.zhihu.com/video/2050196933420095001

Hamburger Making:

https://www.zhihu.com/video/2050196957394630265

Qwen-RobotWorld Physical Agent Infinite World

Real-world experience is the scarcest resource in the robotics field. Qwen-RobotWorld addresses this by directly learning the world's state transition function: given current observation and a natural language action, predict what the world will look like next. The key design choice is to express all actions in natural language—this unifies end-effector poses, steering instructions, and navigation waypoints into a single interface, allowing 20+ embodiment types and 500+ action categories to be trained jointly under an embodied world knowledge corpus (8.6 million video-text pairs, over 200 million frames). A 60-layer dual-stream MMDiT deeply couples Qwen2.5-VL's semantic representations with video latent variables. Using a complete multimodal large language model as the action encoder—rather than a lightweight text encoder—is key: it brings internalized world knowledge—arms are rigid bodies, liquids diffuse, objects fall—thereby implicitly constraining generation to physically credible futures. Each domain reinforces the others: manipulation teaches contact physics, driving teaches large-scale 3D geometry, navigation teaches room-level spatial reasoning.

- Language-Driven Unified Action Interface — Natural language standardizes 20+ robot embodiments and 500+ action categories into a unified training interface, enabling joint training of manipulation, driving, navigation, and human-machine transfer; each domain reinforces the others

- Dual-Stream MMDiT + Qwen2.5-VL Action Encoder — Using a complete multimodal large language model (rather than a lightweight text encoder) as the action encoder, parsing complex compositional instructions into precise generation signals and internalizing physical world knowledge, serving as a synthetic data engine, closed-loop policy evaluator, and action planner

- Leaderboard Ranking: EWMBench Total.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.