Review · AISource: NVIDIA
Nemotron Bolt claims a 24GB footprint; its deployment recipes target larger GPUs
NVIDIA positions Nemotron Bolt for local agentic work, including coding, with a stated 24GB memory footprint. [2] Its published GGUF deployment recipes target discrete GPUs with at least 32GB or 48GB of memory, depending on configuration. [1] NVIDIA announced availability on October 15, 2026. [2] The quantized model card lists 74 billion total parameters and 9 billion active parameters. It supports multimodal inputs and text output. [1] It specifies mixed quantization: IQ2-XXS experts, NVFP4 Mamba layers and Q8_0 embeddings and output. [1] The setup instructions use llama.cpp with a separate LiLiCorr drafter for speculative decoding. [1]
Quantized AI model
NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB GGUF
VerdictWait and see
NVIDIA's published deployment recipes do not establish practical coding performance or demonstrate operation on a 24GB GPU. [1]
- Best for
- Developers evaluating NVIDIA's published GGUF deployment recipes for coding agents [1]
- Pricing
- Commercial and noncommercial use under OpenMDW-1.1; the model card specifies no subscription price or hosted free-tier allowance. [1]
Pros
- Official quantized artifact with llama.cpp deployment instructions [1]
- Documented context and concurrency settings for larger GPUs [1]
- Commercial and noncommercial use under OpenMDW-1.1 [1]
Cons
- Published GGUF recipes target 32GB+ or 48GB+ GPUs [1]
- Memory requirements vary with context length and speculative-decoding settings; the published recipes do not demonstrate a 24GB GPU configuration [1]
- Unpinned engine and model revisions in setup instructions [1]
Why it matters
The smaller recipe specifies 65,536 context tokens and one sequence on a GPU with at least 32GB. [1] The 262,144-token recipe targets GPUs with at least 48GB. [1] The card lists a 256K maximum context. Its quick start uses 98,304 tokens. [1] NVIDIA compares Bolt with Qwen3.6-35B-A3B and Gemma4 26B-A4B in BF16 precision. [1] That comparison does not establish an advantage at equivalent local memory budgets.
Attribution
Sources
- 1NVIDIANVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB GGUF model card
- 2NVIDIANVIDIA announces Nemotron Super VL and Bolt
- 3NVIDIANVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB packed model card
Read the original source for full details and context.