Intelligent NPCs
Nemotron 3.5 Bolt: a 4GB open model for on-device NPCs
NVIDIA has released Nemotron 3.5 Bolt E4B 4GB, an open small language model built for local game characters, voice assistants and robots. It fits in about 4GB of memory and is offered through NVIDIA's In-Game Inferencing SDK.
On 23 October NVIDIA released Nemotron 3.5 Bolt E4B 4GB, an open small language model aimed at conversational agents that run locally: voice assistants, robots and, in particular, in-game characters.
The weights are on Hugging Face under the OpenMDW-1.1 licence, and the model is also offered through NVIDIA’s In-Game Inferencing (IGI) SDK. NVIDIA’s developer blog on 27 October pointed to SkyrimNET, a community modding platform for Skyrim, as a place to try it.
How it works
Bolt is post-trained from Google’s Gemma 4 E4B, an 8-billion-parameter model with 4 billion effective parameters. NVIDIA then tuned it with reinforcement learning across environments for instruction following, multi-turn tool use, conversation and structured output. The 4GB build is a Q4_K_M quantisation, which is what keeps the memory footprint small.
It takes text in and gives text out, with a 64K-token context window and support for 35 languages. Two switches matter for games: a conversational mode that encourages short, speech-ready replies, and configurable reasoning that can be turned off, or capped, to hold a latency target.
NVIDIA’s demo has a blacksmith trader NPC that checks a payment, looks up a crafting recipe, gathers ingredients, crafts the item and hands it over, all through tool calls.
The numbers
NVIDIA claims about 40% higher average quality than its previous small model on conversational-agent benchmarks, and up to 80% fewer output tokens than peers when reasoning is on. On its own tests the BF16 version scores 53.1 on MultiChallenge (consistency across turns) against 42.1 for Gemma 4 E4B, and 67.1 on RoleBench (staying in character) against 50.0.
NVIDIA’s speed tables, measured at batch size 1 on llama.cpp, list 3.5GB of VRAM and a 548 ms time to first token on an RTX 5070 for a 3K-token prompt. On a Jetson Orin Nano 8GB the same test takes 5,785 ms.
Limits
The model card is clear about the trade-offs. Alibaba’s Qwen3.5-4B scores higher on Tau2-Bench (75.1 against 55.1) and on the 64K-token RULER long-context test, and Gemma leads on EQ-Bench v3. All figures are NVIDIA’s own, measured in its harness. Bolt is text-only, so speech recognition and voice synthesis are separate models you must fit in the remaining memory. The published speeds also show that low-power hardware is still slow.
Our take
A character that runs on the player’s own machine has no per-line cost and no network delay, and it can read game state directly. The catch is that a 4B model has little room to spare, so the design work moves to what the character is told and remembers. That is the problem we cover in intelligent NPCs: a small model is enough when the game supplies the world, the memory and the rules around it.