목표 지향적 마인크래프트 범용 AI 모델 'Pan'

At Pantograph, we're working on training fully general robotics models that can act autonomously for hours at a time. Especially in robotics, it's difficult to get diverse data at scale. Learning to act from internet video data could allow models to scale with compute, rather than being limited by small action datasets. In this work, we develop a simple method for learning goal-directed behavior through pretraining on internet-scale video. Usually, goal-directedness is taught in a post-training phase, which limits the extent to which it can generalize. Here, we learn goal-directedness during pretraining on internet-scale video, which greatly improves the models' ability to achieve complex goals. Video games are a useful testing ground for robotics. We're starting with Minecraft because it's open-ended, and supports commanding diverse, long-horizon goals. 1 1 This model can only act within the one game, but we will be training larger models across a broader distribution of visual environments in the coming months. ↩ Our largest model, which we call Pan, is a 4B parameter model that can fight mobs, explore to find specific objects, complete challenging platforming environments, and build structures as commanded. We find it is able to generalize to quite diverse environments that it has never seen before. Reinforcement learning is interesting in part because it can produce models that are superhuman. If we can do reinforcement learning at scale on video, we may be able to produce models that are superhuman in many more areas all at once. # Reinforcement learning from observation-only data We view internet-scale videos as reinforcement learning trajectories that include only observations. 2 2 This lens makes sense for some modalities and not others. Most text data, for example, can't be viewed as a written transcript of an agent interacting with an environment. It could likely work for audio, however. ↩ It's difficult to do reinforcement learning from this kind of data because it lacks rewards and actions, but it turns out there are simple solutions for both. Goal-conditioning is a simple way of side-stepping the need to specify a reward function. The idea is to use what happens later in a video as the goal for the earlier part, a trick sometimes called hindsight relabeling , 3 3 Marcin Andrychowicz et al., Hindsight Experience Replay , NeurIPS 2017. ↩ because it takes what may have been a failure at one goal, and relabels it as a success for whatever actually happens. In principle, you could use any description of what happens later in a video as the goal, but in this work we focus on using frames of video as goals. Although videos don't contain actions, it's still possible to learn functions from video that only depend on states. This includes value functions, 4 4 Typically, value functions are tied to a specific policy. In this very general setting, where any number of policies may be acting in any kind of environment, it makes more sense to use in-context learning to infer the tendencies and level of skill of the policies in a specific video. This way a single model can be trained that can act as a generic value function, capable of evaluating any policy's behavior. ↩ which in goal-conditioned RL represent roughly the probability that a goal frame is achieved in the agent's future, as well as the next frame distribution for a goal-conditioned policy. 5 5 Both of these can be modeled in a number of ways, including likelihood-based models, energy-based models, and contrastive learning. We're excited about future work that could study the trade-offs between these different approaches. ↩ After this action-agnostic pretraining, a smaller dataset containing actions can be used to produce an agent capable of acting in the environment. 6 6 Given a strong value function, learning to act is a much easier problem (Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction , 2nd ed., MIT Press, 2018). A value function effectively converts the very difficult multi-step decision making problem of reinforcement learning into a single-step decision problem, which is generally much more tractable. ↩ We think that this lens of viewing video as RL trajectories is fruitful, and there are many ideas yet to be explored. If this is the kind of work you're interested in, join us ! # Training We pretrained a family of goal-conditioned models from scratch on about 500k hours of diverse videos of Minecraft gameplay. Afterwards, we post-trained them on about 2k hours of contractor trajectories including both videos and action sequences. The pretraining is action-agnostic, and the action space only comes in during post-training. (See Appendix for more details.) # Evaluation We created a suite of 104 evaluation environments. Each environment consists of an initial Minecraft save world and a goal save world. We load the goal save world once per rollout and render a single goal image from it as the model's prompt. We use a mix of programmatic and language model-based graders to assess model performance, depending on the environment. # Comparisons We compared to STEVE-1, an open source goal-conditioned policy for Minecraft. STEVE-1's goal conditioning comes from post-training on a contractor dataset with action labels, on top of VPT, a model pretrained by OpenAI using an inverse dynamics model to label internet videos with estimated actions. 8 8 Shalev Lifshitz et al., STEVE-1: A Generative Model for Text-to-Behavior in Minecraft , 2023; and Bowen Baker et al., Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos , 2022. ↩ We also trained a VLA baseline on the same post-training dataset of contractor trajectories that we used for Pan. This VLA model was initialized from Gemma 4, one of the strongest open-source language models with support for video inputs. 7 7 Google DeepMind, Gemma 4 model card , 2026. ↩ In total, much more compute went into Gemma 4 than into the models we trained, though our models have likely seen more Minecraft data through their pretraining. # Results The models trained with goal-conditioned pretraining are better able to accomplish their goals in our evaluation worlds, including for varied and off distribution goals. We find that model size matters most for complicated, semantic things like mechanisms, and dextrous tasks like building. Our models learn some mechanisms (buckets, flint and steel, crops, fishing) and not others (levers, ender pearls, animal taming). We are excited to see what capabilities emerge from further scale-ups. # Basics We begin simply: if we command a goal a little bit in front of the model, does it care about the goal frame at all? Will it stay there? We find that the Pan models walk to the target position and oscillate around it for the remainder of the rollout, seeming to try to line up their view with the goal frame perfectly. We call this a "goal dance", and it is a useful indicator that the model thinks it has achieved the goal. STEVE and the VLA are less consistent on these simple tasks, tending to wander off rather than stay on the goal, though they do accomplish them sometimes. In Minecraft, building a tower underneath you is almost as basic as walking forward, and all of the models are able to do it. Our models again demonstrate more faithful goal following: whereas STEVE and the VLA tower up way beyond the commanded image, the Pan models frequently stop at the commanded height. What about simple but unusual goals, like looking at a target made out of wool, or swimming through a pool towards a gold panel? This is easier to do, but less common in the training distribution. Here, the difference between model families is larger: Pan models are much more goal directed. # Building We test building ability in various settings, varying how much of the final structure we show in the goal frame, what materials are us