The Robotics Scaling Recipe

Entering the Age of Robot Intelligence

Large language models learned the structure of human knowledge and communication through the internet. Now, robots need to learn the way we interact in the real, physical world.

Software AI is deep into its "agentic era". Jensen Huang calling himself the "Inference King", Claude Code building a C compiler autonomously to run Doom, and long-running agents continue to improve on benchmarks like METR's time-horizon evaluations.^[1][2][3]

However, the frontier of physical AI is opening; deep learning for physical AI is advancing at a deceptively rapid rate, and the "ChatGPT moment" for robotics is near.

The LLM Parallel

To understand the innovation and intelligence scaling for robotics AI models, it is important to understand how LLMs are trained and their respective scaling laws.

LLM scaling laws are the empirical observation that language model performance improves predictably if you scale three things: model size, training data, and compute.^[4]

In other words, if we train a larger parameter model, with an increasing dataset size, using more compute, we can decrease the model loss.

The basic LLM training pipeline is as follows: generative pretraining, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF).

Pretraining is the stage where the model learns the statistical structure of language, typically using the transformer architecture.^[5] You take massive amounts of text, code, documents, webpages, books, papers, etc., and then train the model on a simple objective: predict the next token.

That model loss we were talking about looks like the following:

Where $x_i$ is the true next token, $x_{<i}$ is all previous tokens, $p_{\theta}(x_i \mid x_{<i})$ is the probability the model assigns to the correct next token, $\theta$ represents the model parameters, and $N$ is the number of tokens.^[6]

A pretrained model is not the same assistant we use in ChatGPT; right now, it is simply an excellent autocomplete. Now we need to fine-tune it to follow instructions and give the helpful answers we expect.

This is where SFT comes in, and it works by creating a curated dataset of conversations where the "correct" assistant answer is provided. Then the model is further trained to predict that response token by token.

Now that the model knows how to answer, we need to finally train the model how to answer well using RLHF. Instead of giving the model one ideal answer, humans compare multiple model-generated answers and rank which ones are better. These rankings are used to train a reward model, which is a separate model that predicts how much a human would prefer a response.

Using this process, and adhering to scaling laws alongside many other techniques I am glossing over, frontier labs like OpenAI dramatically improved the performance of their models over the years.^[7]

LLMs improved because the research field found a repeatable scaling recipe. Robotics is now looking for its own version of that recipe, and there are many parallels to the training process.

Robot Intelligence: The Model Strategy

For LLMs, we input text, and perhaps some other modalities like images, and out comes text. They are autoregressive language models, meaning they generate one discrete text token at a time.

For robotics models, we now need to output motor commands and positions, and this is a challenge because instead of discrete output, robots need to output continuous actions.

Instead of outputting a single action at a time, Action Chunking with Transformers (ACT) can be used to predict a sequence of future actions, or an "action chunk".^[8] This matters because robotic control is extremely sensitive to small errors. If a robot predicts one tiny movement at a time, each mistake can compound over the episode, causing the robot to drift away from the intended behaviour.

Since we know so much about language models, it would be beneficial to work with tokens rather than "action chunks". So Frequency-space Action Sequence Tokenization (FAST) introduced by Physical Intelligence turns chunks of robot motion into something closer to the text tokens used by language models.^[9]

Physical Intelligence's FAST tokenizer uses the discrete cosine transform (DCT), to compress action sequences into dense discrete action tokens for autoregressive Vision Language Action (VLA) models.

Many of today's leading generalist robotics AI models are based on the VLA architecture. The VLA uses a Vision Language Model (VLM) as its backbone. VLMs are multimodal models trained to process images and text together, similar to the consumer AI models people use today, like GPT 5.5, Sonnet 5, Gemini 3, etc.

The key idea is that VLMs already have the useful visual-language knowledge from its internet-scale pretraining. It can already understand and identify a cat in the scene, or that the mug is filled with coffee.

VLAs like $\pi_0$ from Physical Intelligence extend the VLM capabilities by bolting on an "action expert", which is a much smaller and faster model used to map the visual-language understanding into robot actions, or motor commands.^[10]

RLHF teaches LLMs which answers humans prefer. Physical Intelligence introduced RL with Experience & Corrections via Advantage-conditioned Policies (RECAP) to teach a robot which actions actually led to successful outcomes in the real world.^[11]

A VLA trained only on demonstrations has the same limitation as a chat assistant trained only with SFT: the model learns to imitate good examples, but does not know how to handle error cases. This is especially crucial in robotics because small mistakes compound, leading the robot down a path that is outside the training data, evidently leading to errors.

VLAs like $\pi^*_{0.6}$ that use RECAP not only train on demonstrations, but use reward labels, autonomous robot rollouts, and human teleoperated corrections to condition the VLA.

Up until recently, the frontier has been mostly defined by pure VLAs like $\pi_0$ and GR00T.^[12] However, the future of robotics AI foundational models is incorporating world models. A robot should know more than just "what action to take"; it should understand what happens afterwards.

Physical Intelligence's $\pi_{0.7}$ shows one version of this shift.^[13] It is still a VLA, but it is guided by a world model. At inference time, a high-level policy can generate subtask instructions, and a lightweight world model can generate visual subgoal images showing the desired near-future state. This gives the robot a concrete visual target to move toward.

NVIDIA's DreamZero takes this idea further by making the world model part of the action policy itself.^[14] It is a World Action Model (WAM) built on a pretrained image-to-video diffusion backbone. DreamZero works by trying to "imagine" or "dream" the physical future. The observation is that if the model can predict the next few frames, it has a richer signal for choosing actions. So now instead of just predicting actions like a VLA, it jointly predicts future states and the corresponding action.

Robot Experience: The Data Strategy

The model strategy is only half the battle. LLMs are fortunate to have all, if not most of their training data available on the world wide web. Robotics training data is very sparse, low quality, and lacking diversity.

The gold standard is collecting real robot data in the wild; however, this can be costly and inaccessible. So the past three years have primarily consisted of leveraging teleoperation, meaning controlling a robot remotely with a human. While extremely accurate and producing high quality data, the physical upper bound is 24 hours per day, per robot. This is extremely unscalable.

So instead of teleoperation, Stanford researchers developed the Universal Manipulation Interface (UMI), which is a handheld gripper with cameras to demonstrate manipulation tasks.^[15] The demonstrations captured can then be transferred to robot policies. This is also Sunday Robotics' bet on solving the embodiment gap with their Skill Capture Glove.^[16] NVIDIA did similar research with their DexUMI which built on the original UMI.^[17]

What researchers have realized is that there are only two real scalable sources of data: human egocentric data and simulated data. We need something on the same or bigger scale of Tesla and Waymo's autonomous vehicles, which is currently the biggest active data flywheel.^[18]

NVIDIA's EgoScale proved the viability of training on human video.^[19] They trained a VLA on over 20,000 hours of action-labeled first-person human video. The key result is that they discovered a log-linear scaling law, basically demonstrating that robots can scale on human video.

Physical Intelligence noticed a similar phenomenon with human-to-robot transfer. They co-finetuned $\pi_{0.5}$ on human egocentric video and relevant robot data, treating human video like another embodiment.^[20] They observed a roughly 2× improvement on tasks where robot data was limited.

Human egocentric data is one scalable path, but the other is pure simulation. In theory, simulation is the infinitely scalable, perfect engine to generating the training data we need. The problem is the sim-to-real gap. Traditional simulators have struggled to capture the messy details of the real world: contact physics, deformable objects, and friction, object diversity, etc.

NVIDIA's DreamDojo is an attempt at learned simulation instead of hand-engineering it through software programs.^[21] It is a foundation world model trained on over 44,000 hours of egocentric human video, then post-trained on smaller-scale robot data. Given the current scene and a proposed action, DreamDojo predicts the future visual sequence of what should happen next.

Overall, the robotics data strategy is moving from scarce robot-only data to a mixture of real robot demonstrations, human experience, and learned simulation.

The Learning Loop is Key

It is clear to me that world models will become a cornerstone of robotics AI, not only as better policies, but as a way to unlock the data bottleneck.

Inference compute will not only be used to determine the robot's next action. It will also be used to generate and evaluate possible worlds for robots to practice in. A world model can simulate different environments, object layouts, task variations, and failure cases, creating the kind of diversity that real robot data collection struggles to provide.

More compute means more imagined environments. More environments means more experience. More experience means better robot models.

This is not a completely new idea. AI progress has always come from turning hard problems into compute problems. But robotics makes the need obvious: we cannot scale physical intelligence by only building more robots and deploying them in the wild. Robotics has to scale by building more worlds.

I believe that the winning robotics companies will not simply be the ones with the "best VLA" or most impressive humanoid. Instead it will be the ones who have the best learning loop: real robot data, human video, simulation, and feedback that all compound into better policies.