Advertised rates set a throughput hurdle
Hugging Face lists Beam NVFP4 configurations with four A100 GPUs at $10 per hour and four H100 GPUs at $40 per hour. At those advertised rates, the H100 option needs four times the accepted fixes per billed hour to match the A100 option’s compute cost per fix. Faster responses may justify a premium when latency matters. Speed lowers cost per accepted fix only when it produces enough additional accepted work. Sources: Hugging Face
The lowest-cost deployment route remains undetermined: exact checkpoint sizes, loading and runtime memory, recipe execution and repository-fix outcomes lack independent measurements. A purchasing decision also needs current API prices and equivalent rental quotes. ByteForward says it did not run Beam, measure hosting costs or verify a public token rate. Advertised rates and arithmetic estimates provide comparison thresholds, but cannot identify a winner. Sources: ByteForward
Quantization reduces nominal payload
Reflection specifies 501.3 billion total parameters and 23.4 billion active parameters, with BF16, FP8 and NVFP4 releases. The payload calculation uses the total parameter count, assuming two, one or half a byte per parameter, respectively. The table below shows the resulting nominal payloads in decimal gigabytes. These estimates assume uniform precision and exclude metadata, quantization scales, higher-precision tensors and runtime allocations. They are neither measured artifact sizes nor GPU memory requirements. Sources: Reflection
Table
Nominal weight payload by precision, in decimal GB
| Precision | Assumed bytes per parameter | Nominal payload |
|---|---|---|
| BF16 | 2 | 1,002.6 |
| FP8 | 1 | 501.3 |
| NVFP4 | 0.5 | 250.65 |
ByteForward reports roughly one terabyte in the BF16 repository, with shard numbering spanning 188 files. Treat that as a reported repository size, rather than a verified byte sum at a pinned revision. A deployment ledger should distinguish logical file bytes, occupied disk space and network bytes transferred, including retries. Record whether tokenizer files, configuration, container images and an optional drafter count toward storage and startup costs. Sources: ByteForward
Reflection says its NVFP4 release uses post-training quantization with NVIDIA Model Optimizer. It reports Terminal Bench v2.1 scores of 80.1% for BF16, 78.9% for FP8 and 78.4% for NVFP4, using temperature 0.7 and top-p 0.99. FP8 and NVFP4 trail BF16 by 1.2 and 1.7 percentage points, respectively. Those vendor results describe a precision trade-off, but cannot supply the acceptance rate for a repository-fix budget. Sources: Reflection
Context adds memory beyond the weights
The vLLM team describes 43 sliding-window layers retaining 512 tokens and nine full-attention layers, with eight key-value heads of dimension 128. The output gate adds no separate key or value tensor to the key-value cache, or KV cache, calculation. With bounded sliding-window retention, the nine full-attention layers drive continued cache growth. The table below applies that architecture to independent sequences at concurrency one and four. Sources: vLLM
Table
Calculated aggregate logical KV cache, in GiB
| Tokens per sequence | FP8, one sequence | FP8, four sequences | BF16, one sequence | BF16, four sequences |
|---|---|---|---|---|
| 8,192 | 0.183 | 0.730 | 0.365 | 1.461 |
| 32,768 | 0.604 | 2.418 | 1.209 | 4.836 |
| 131,072 | 2.292 | 9.168 | 4.584 | 18.336 |
| 262,144 | 4.542 | 18.168 | 9.084 | 36.336 |
| 1,048,576 | 18.042 | 72.168 | 36.084 | 144.336 |
These are aggregate logical cache estimates, excluding allocation padding, scales, buffers, weights and drafter overhead. They are not per-GPU memory figures. NVIDIA’s Dynamo recipe advertises 1,048,576-token context, FP8 cache, 64-token blocks and DFlash2 with block length eight. Its aggregated H200 configuration uses eight GPUs and FP8 weights, while its Blackwell configurations use NVFP4. NVIDIA provides no measured performance results on that page, so the recipe does not establish million-token capacity on the four-A100 configuration. Sources: NVIDIA
Capacity testing should cover each supported context and concurrency pair in the table. Record peak host process and container memory during download, conversion and loading, separating filesystem cache. Track persistent storage, temporary disk peaks and per-GPU memory after loading, graph capture, prefill and generation. Record engine commits, container digests, CUDA, driver and kernel versions, topology and parallelism. Also capture cache precision, prefix sharing, buffers, drafter settings and failures, including unsupported kernels, memory exhaustion, preemption and context-limit errors.
Managed capacity exposes idle-time costs
Hugging Face calls its catalog deployment verified and advertises 320 GB of GPU memory and 580 GB of host memory for the A100 configuration. Its displayed A100 recipe uses vLLM, tensor parallelism four, FP8 KV cache and a 262,144-token maximum. It also sets 64 maximum sequences and 0.95 GPU-memory utilization. Prefix caching, chunked prefill and CUDA graphs are enabled, with no speculative-drafter argument shown. These settings do not promise maximum context for every sequence simultaneously. Loading an NVFP4 checkpoint also does not establish native NVFP4 arithmetic acceleration. Sources: Hugging Face
Table
Calculated managed endpoint compute charges, in USD
| Billed running time | Four A100 GPUs | Four H100 GPUs |
|---|---|---|
| One hour | 10 | 40 |
| Eight hours | 80 | 320 |
| 24 hours | 240 | 960 |
| 30 days continuously | 7,200 | 28,800 |
Hugging Face says replicas scaled to zero are not billed and may take time to resume. The table above excludes additional charges; budgeting should establish how provisioning, downloads, persistent storage and transfers are billed. Utilization should account for idle capacity while tools execute, tests run and reviewers inspect patches. Consider a hypothetical system producing six accepted fixes per busy hour at 50% utilization. Using the advertised rates, its compute cost would be $3.33 per fix at $10 per hour and $13.33 at $40 per hour. Those are illustrative calculations, not Beam task results. Sources: Hugging Face
Compare the whole bill against accepted work
For hosted API use, count uncached input, cached input, output and separately billed tools or services. For dedicated deployments, count billed replica-hours, storage, transfers and other infrastructure charges. Divide total experiment infrastructure spend by accepted attempts, retaining failed generations, retries, rejected patches and startup failures in the numerator. Report review minutes separately, then add review and operations labor for a labor-inclusive comparison. With zero accepted fixes, report that outcome instead of a finite ratio.
Serving throughput cannot replace accepted-work counts. The vLLM team reports roughly 88,000 total tokens per second per GPU on four GB300s at concurrency 96. Those totals include cached input, with 91.4–95.6% input-cache hits. Its AgentX benchmark replayed coding conversations with NVFP4, FP8 cache, DFlash2, maximum reasoning and temperature 1.0. Each point ran for 20 minutes using vLLM commit 8e0922. That measures serving under a particular workload and cache pattern, rather than accepted repository fixes. Sources: vLLM
- Choose an API when measured per-fix charges undercut dedicated capacity after idle time and operations costs.
- Choose a managed endpoint when sustained accepted throughput covers its hourly bill and operational savings justify the premium.
- Choose rented self-hosted hardware when rental, storage and operations cost less at comparable acceptance quality and latency.
Make the next comparison reproducible
A proposed comparison would run 20 repository tasks three times per feasible configuration, producing 60 attempts for each. Freeze repository commits, prompts, dependencies, acceptance tests, the agent harness, reasoning settings, sampling and token or time budgets. Fix retry rules, network access, tool permissions and treatment of human intervention. Start each attempt from a clean checkout and randomize configuration order where practical. Separate cold starts from warmed service and record how startup costs are allocated.
Retain patches, test logs, regression counts and severity, model usage, latency, reviewer minutes, retry histories and billed spend. Under this proposed protocol, report accepted attempts out of 60, distinct tasks accepted out of 20 and per-task reliability across repetitions. Acceptance should require fixed tests and a review rubric, with human-repaired patches labeled separately. Keep the fixtures, clean snapshots and rubric for the Mirror review. Reflection says Beam was trained together with Mirror, making harness choice relevant to interpretation. Treat harness changes as an explicit experimental factor so deployment savings remain distinguishable from changes in agent behavior. Sources: Reflection