Meet Open Dreamer: JAX/Flax Reproduction of the Dreamer 4 World Model Pipeline, with Full Training Recipe Published

A small group of AI researchers (Reactor) released Open Dreamer, an open source implementation of the Dreamer 4 world model pipeline written in JAX and Flax NNX.

What exactly was sent

Two caches were released. next-state/open- dreamer manages the training pipeline: causal video token, action-conditioned latent dynamics model, output generation, and FVD goal. reactor-team/open-dreator runs a small local output harness that generates frames from MP4 and the same action file.

The third artifact is the browser demo hosted by the Reactor runtime. Streams the generated world of Minecraft in real time and exposes the Game ⟷ transform a dream that moves the stream from the real game to a model of the world in a frame.

The stated objective was to reproduce the Dreamer 4 study. The research team deliberately avoided methods outside of that research paper to keep the search space small. They started with CoinRun, a procedurally generated 2D platformer powered by a single GPU, and scaled the workflow into a Minecraft/VPT-style video game.

Architecture: one core, two models

Both the tokenizer and the dynamics model use the same block-causal transformer backbone. That core alternates between two types of attention. Spatial layers distribute information between elements of a single frame. Causal time layers propagate information between frames.

The tokenizer is a transformer-based Masked Autoencoder rather than a VAE. The team reports about 100× compression and notes that the design does not require KL or adversary losses. Masking, they argue, makes the hidden space more distinct.

The dynamics model makes the following frame assumptions and is trained with diffusion forcing, flow simulation, and intercept models. It also predicts the next action. Rather than switching between a separate change module and a policy, the output is folded into blocks for each time step (previous action, state, policy). Local attention moves within each block; temporal causal attention connects blocks across time.

Importantly, world model tokens cannot read an agent token. Therefore knowledge of work and policy can influence future states only through subsequent action.

Training recipe, as prescribed

The Minecraft configs posted make the recipe concrete.

The dynamics model is 1.6B parameters: 30 block-causal layers, d_model 1920, 30 attention heads, and 3 KV attention heads of questions collected. Every fourth layer is a time-conscious layer. Each time carries 32 read register tokens, too packing_factor: 2 packs neighboring latent tokenizers into each local dynamics token. Timekeeping uses a 192-step sliding window.

Training runs 200,000 steps with Muon, a WSD schedule, and a maximum learning rate of 3e-4. Cross-section/bootstrap samples run at 100,000 steps in a batch fraction of 0.25. The EMA decay is 0.999.

The tokenizer config outputs 512 hidden tokens per frame at a bottleneck width of 16. The 360×640 green frames are merged into 368×640 so that both sizes split into 16×16 patches. The depth of the encoder is 12 d_model 1536; The depth of the decoder is 8 in d_model 1024. MAE masking probability is up to 0.9, and LPIPS is used with a weight of 0.2 for half of the time steps.

VPT actions are divided into 27 binary action channels and 121 phase mouse classes, with no continuous channels.

Computemaxxing and the memory wall

The research team reports a model utilization of 57–58% FLOPs, against the stated benchmark of 60% for healthy transformer training. The reason is the roof conflict. In the B200, the intersection between bandwidth-bound and commute-bound remains at 292 FLOP/byte. Feeding 256 frames per GPU pushes the workload past that line point.

Shading went the other way from expectations. For 1.6B parameters the model state – parameters, gradients, optimizer state, and EMA – takes up about 24 GiB, which is equivalent to B200. Getting started was a real expense. The research team tried data matching, FSDP, tensor matching, and sequence matching, and settled on implicit data matching and activation testing.

Data loading was solved by pre-tokening the entire dataset into .arrayrecord files, then use Grain with a GPU-side prefetch buffer. ffmpeg decoding was not fast enough to keep up with fed GPUs.

The stability phase is the real burden of payment

The research team clearly states that stability has consumed a very large share of their time. Their main observation: many stability problems occur in spite of the loss decreases. The MSE improves smoothly while the generation rate decreases.

Six corrections are written. The Muon replaced the LaProp, which bounced randomly and continuously, between two runs of about 400 hours each B200. EMA weights are treated as a constraint to achieve divergence. Mixed precision is boundary sensitive: parameters are always float32, BF16 includes most matmul implementations and attention inputs, and float32 is reserved for normalization and the head of the dynamics flow.

For loss estimation, they use ix-prediction for v-space loss, which reduces to a weight term similar to Dreamer 4’s but with a squared value. They report a small but noticeable improvement. Minibatch barycentric optimal transport between noise and hidden sequences makes output generation more stable. The μ-parametrization was tested and judged unnecessary, in part because the muon holds tight parameters for all model sizes.

Another result of the CoinRun phase: the iso-FLOPs sweep sets the compute-optimal scaling at approx. $N\propto C^{0.56}$ again $D\propto C^{0.44}$.

What’s not in the box

The repository does not include behavior-cloning or the RL training loop; the Dreamer 4 BC/RL agent loop is listed as an open road item. The CoinRun policy function described in the post was not used for Minecraft and was not released.

The post also doesn’t publish FVD scores, though scripts/eval_fvd.py ships with I3D based harnesses optimized for 4 core frames and a 240 frame horizon.

Key Takeaways

• Open Dreamer reproduces the Dreamer 4 pipeline in JAX/Flax NNX, with training code and a Minecraft demo.
• The dynamics model is 1.6B parameters, 30 layers, d_model 1920, he trained 200K steps with Muon.
• Reported engineering numbers: 57–58% MFU on B200, 256 frames per GPU, ~24 GiB model state.
• Stability, not success, was the barrier; loss curves mask the regression of generation quality.

Check out the blog post and demo, training repo, inference repo, and Reactor on X. All credit for this research goes to the researchers of this project.