pokeai

a distributed deep-RL framework that trains a neural net to play Pokémon Red on hardware i already owned — a 2016 GTX 1080, a Ryzen 7 5700G, no cloud. two systems train in parallel and blend their "brains" into each other every 5 minutes. i called it asymmetric hardware-heterogeneous cooperative RL because i couldn't find anyone who'd wired it up quite this way.

status: still cooking. workers are on Route 1, Badge 1 not earned yet, repo private until it clears. it never fully worked — but it's an idea i keep coming back to.

the idea

the question was never "can it run Pokémon" — it was "can it learn Pokémon," across 256 parallel instances, on a gaming GPU from 2016. two asymmetric systems, one machine, sharing knowledge:

CPU system — 8 PyBoy emulator workers under a supervisord stack, training a shared PPO policy (Stable-Baselines3). a custom reward engine pushes the agent north through Kanto on a clockwise geographic curriculum.
CUDA system — 256 Game Boy instances living entirely in VRAM via custom kernels for the GTX 1080's Pascal arch (sm_61). a GPU-resident PPO loop keeps every tensor on-device; a health monitor auto-tunes on saturation or collapse.
transfusion loop — every 5 min a host daemon blends the conv/FC weights of both systems into each other (60/40 CPU/GPU → 70/30). CPU contributes reward-correlated navigation; GPU contributes visual diversity from 256 exploration paths. actor/critic heads stay system-specific.

hardware

this started as a gaming PC — every part bought on release day, none of it for research. now it's a dedicated training node running 24/7.

AMD Ryzen 7 5700G · NVIDIA GTX 1080 8GB (2016) · 48GB RAM
Proxmox → Ubuntu 26.04 VM → Docker, PCIe GPU passthrough
~1,800 fps across 256 instances · ~180W under load · 28.6 MB VRAM for all 256 GBInstance structs
no cloud compute, ever

architecture

Proxmox host
└── VM 501 (Ubuntu 26.04, PCIe GPU passthrough)
    ├── docker: pokeai        (CPU system)
    │   └── supervisord ×8 — SB3 PPO trainer, telemetry bridge,
    │       stagnation/watchdog/stack guards, orchestrator, blackbox
    ├── docker: pokeaicuda    (CUDA system)
    │   └── 256 GBInstance structs in VRAM, custom PPO loop,
    │       CUDAHealthMonitor, telemetry server :9090
    └── host
        ├── transfusion_loop.py   (bidirectional weight sharing)
        ├── monitor.sh            (5-pane tmux dashboard)
        └── twitch_panels.py      (live OBS overlays)

reward engine

the reward mirrors biological curiosity — reward what's new and useful, penalize wasted time.

per step    -0.01     time penalty (always)
new map    +50.00     first visit, gated behind badges late-game
north push  +Δ·0.03   proportional to northward progress
gym entry   +5.00     Pewter Gym magnetism (phase 1)
battle      +0.50     HP drop detected — agent is fighting
level up   +10.00     per level
badge    +1000.00     per badge
clockwise   +score·0.05 + Δ·5.00   (phase 2 only)

self-healing

shared-memory bridge — lock-free seqlock segment in /dev/shm instead of sockets; survives supervisor restarts, workers auto-reconnect.
stagnation guard — detects Route 1 blackout loops by tracking HP ratio across workers, restarts the trainer.
CUDA health monitor — entropy boost on EV saturation, checkpoint recovery on collapse, LR dampener in the stable band.

honest log

this isn't a polished paper — it's an engineering log of what i built, what broke, what i fixed, and why. i used Claude as a programming assistant throughout; the ideas, the curiosity, and the 10-year-old GPU are mine, and every commit notes where AI helped.

github (private) ↗