World models
AnyGameRenderer turns greybox games into finished visuals
Alaya Lab's AnyGameRenderer re-renders crude gameplay video in any prompted style while keeping every character where the game put it. A new entity-level benchmark shows how often such renderers still get it wrong.
Coding agents can now write a small playable game in minutes. What they cannot do is make it look good: the output is usually grey boxes, coloured cylinders and flat ground. Alaya Lab’s new paper, AnyGameRenderer, tackles that gap with a model that takes the crude recording and repaints it.
What it does
AnyGameRenderer is a video-to-video renderer. You give it a recording of a rough game and a text prompt, optionally with a style reference image. It returns the same footage, frame-aligned, in the prompted style, with characters, motion detail and effects added. Layout, camera, and each entity’s position and timing stay as the game produced them.
Changing the look of a game then means changing the prompt. The authors show one blockout rendered as a fantasy duel, a voxel scene and a photoreal landscape.
How it works
The team fine-tuned a LoRA adapter on MiniMax-H3 ref2va, a 33-billion-parameter reference-video-conditioned video model. Training used 10,330 paired clips from three sources: 38 games made by an AI game-creation tool, Unreal Engine scenes scripted with groups that change size mid-clip, and real gameplay recordings converted into crude inputs with 3D-box detection.
The input is plain video. Grey blockouts, coloured primitives, semantic maps and wireframes all work, with no per-object labels or engine buffers. That is looser than several concurrent systems the paper cites, which need class labels or low-poly proxies.
The numbers
The paper’s main contribution is a benchmark that scores renderers per entity rather than per frame: 59 held-out cases and 753 ground-truth entities. Against the untuned base model:
- Temporal match (entities appear and disappear on time) rises from 50.9% to 62.9%.
- Position match rises from 57.1% to 61.9%.
- Entities left as unrendered blocks fall from 4.1% to 2.9%.
- Hallucinated entities fall from 52.1% to 46.0%.
Limits
Those figures show a real but modest gain. Even after tuning, more than a third of entity appearances and disappearances are off. Real recordings improved least, with temporal match moving from 46.5% to 51.1%. Top-down games remain the hardest genre, and hallucination rose slightly on top-down and third-person games.
The paper does not claim real-time speed. The code repository lists 123 GB of base weights plus a 2.4 GB LoRA, and asks users to accept the MiniMax-H3 community licence. Treat it as an offline research tool for now, unlike real-time renderers such as EdenLive.
Our take
This is another sign that the field is splitting a game into two jobs: code that keeps the truth of the world, and a generative model that performs it. We describe that layering in AI models as game generators.
The useful lesson is the benchmark. If a model paints over game state, the question that matters is whether the count of enemies, their timing and their positions survive. For anyone building interactive worlds, that bookkeeping is the part to test before trusting a generated look.