Tiny World Model Generates and Plays Games on Raspberry Pi Pico 2 W

Overview

A small neural network project demonstrates generating and playing simple games simultaneously on a Raspberry Pi Pico 2 W microcontroller. The system uses a looped transformer world model to predict the next frame and reward, alongside a streaming reinforcement learning agent that plays the generated game.

How the System Works

• World Model: A single transformer block applied multiple times with shared weights processes game frames and actions. It outputs scores for every cell and token, which are decoded into the next frame using specific rules for backgrounds, singletons, and sprites.
• Agent: The system uses Stream AC(λ) reinforcement learning, operating transition-by-transition without a replay buffer or target network.
• Hardware Split: On the RP2350 chip, core 1 handles the game logic, world model, and agent execution, while core 0 manages USB communication and display tasks.

Implementation Details

• Quantization: The models use int8 weights trained with quantization-aware training (QAT) to fit within microcontroller memory constraints.
• Memory Footprint: Games fit within the Pico 2 W's 4 MB of flash and 512 KiB of SRAM. For example, Breakout uses 361 KB of flash and 299 KB of SRAM, while Pac-Man uses 600 KB and 460 KB.
• Interactivity: Users can press buttons via a connected display pack or over USB to take control from the automated agent, or switch to an on-device learning mode where the agent trains from scratch.

Verification and Reproduction

The project provides Python code for training and simulation, as well as a WebAssembly port to run the engine in a browser. Original source code, models, and firmware instructions are available in the public GitHub repository under SheepSeb/tinyworldformer.