M3.1 Flash versus Mercury 3: what a completed-task cost comparison needs

Launch claims leave completed-task cost open

MiniMax and Inception announced M3.1 Flash and Mercury 3 on October 19, 2026, with competing claims about generation speed and cost. MiniMax claims output above 200 tokens per second and 70% lower agentic usage cost than its predecessor. Inception claims 1,068 tokens per second on NVIDIA GPUs and benchmark performance matching or exceeding GPT-6 Luna and Claude Haiku 5.5. The practical question is whether those advantages survive the tool calls, failed edits and retries needed to finish work correctly. Sources: MiniMax, Inception

This comparison presents a proposed testing protocol; it has no measured four-model outcomes, latency results, spending records or published harness. Inception’s selected partner reports offer narrower evidence: TinyFish reports 50% completion across 50 live Mind2Web tasks. Octen reports median generation of 0.9 seconds and median end-to-end response of 2.2 seconds. Those reports do not establish a shared workload or a winner on completed-task cost across all four models. Sources: Inception

Published prices change with context and promotion

At short-context rates, GPT-6 Luna undercuts M3.1 Flash on uncached input and output. Mercury 3’s launch rates are also lower than MiniMax’s, but its standard rates are higher. The table below compares published schedules, rather than verified account charges. Testing should begin by checking current rates and archiving applicable promotion terms. Sources: MiniMax, Inception, OpenAI

Context length can change the ranking: Luna applies long-context multipliers to the full request. OpenAI prices Batch and Flex at half Standard and Fast at twice Standard. Regional processing carries a 10% premium. A fair evaluation should preserve identical task content, disclose service tiers and record actual token usage. Sources: OpenAI

Verify model identity and freeze the workload

MiniMax’s announcement names M3.1 Flash, while its product and subscription pages refer to M3.1 Flash Preview. Results should identify the deployment actually tested. OpenAI identifies gpt-6-luna, and Anthropic identifies claude-haiku-5-5. Each run should record requested and returned identifiers, provider, endpoint and region. It should also record account tier, SDK version and test window in coordinated universal time. Sources: MiniMax, OpenAI, Anthropic

Provider adaptations also need disclosure. Luna requires Responses for function calling with reasoning; Chat Completions function calling requires reasoning to be set to none. Before scoring begins, each adapter should pass a minimal request, a tool-call round trip and structured-output validation. Adapters should preserve information and permissions. They should record format changes and reject unsupported settings explicitly. Record reasoning settings beside results without assuming that matching names imply matching computational budgets. Sources: OpenAI

The proposed workload has 20 versioned tasks, divided equally among the categories below. The protocol proposes three independent repetitions across four models. That design yields 240 run slots, or 60 per model, with repairs contained within each slot. Hold prompts, tools, repository commits, documents, permissions, resources and tool-output limits constant. Use local service fixtures for structured workflows. Pilot separate tasks, then freeze the proposed limits: 15 minutes, 50 tool invocations and at most two post-submission repair attempts per slot. Time and tool limits apply cumulatively across the initial attempt and both repairs; neither resets between attempts. Grading and delivery of repair feedback consume the time budget, including final acceptance checking.

Grade correctness before interpreting speed

Every task should have private acceptance checks fixed before execution. Repository fixes must reproduce the defect, pass targeted tests and preserve existing behavior. Tool workflows must reach the required state without duplicate effects. Multi-file changes need consistency and compatibility checks. Document tasks require correct facts, arithmetic and supporting citations. Use blinded rubric grading where deterministic checks are insufficient. Hidden tests must remain inaccessible to agents, and repair feedback should follow a predefined rule.

First-attempt success should mean the initial submitted artifact passes every required check within the slot’s time and tool limits. Eventual success should mean an artifact passes every required check within those cumulative limits and the repair budget. Record regressions, malformed tool calls, semantically wrong operations, retries and human interventions separately. Human-rescued runs belong outside autonomous successes, while timeouts, refusals and exhausted budgets remain visible failures. Define infrastructure exclusions before testing and publish their costs.

Each proposed repetition should start independently, without selecting the best answer as the score. Randomize model order within each task and repetition block, then distribute blocks across the test window. Publish overall, category and per-task outcomes with paired comparisons and uncertainty intervals clustered by task. Repeating a task does not broaden the kinds of work covered, so this proposed suite would support limited claims about model superiority.

Measure throughput and usable results separately

The earlier Mercury paper describes Transformer-based diffusion that refines multiple tokens in parallel. Submitted on June 17, 2025, it evaluates Mercury Coder Mini and Small, rather than Mercury 3. That mechanism suggests a hypothesis for testing: faster generation may not proportionately shorten the wait for a usable tool call. Agent benchmarks should therefore keep the following clocks separate. Sources: arXiv

• Request-to-first-token: client dispatch to the first nonempty model-content event, identifying the content channel.
• First usable result: dispatch to a predefined committed answer unit or a complete, schema-valid tool call.
• Output throughput: committed tokens after the first token divided by elapsed time between the first and last committed tokens.
• Full task wall-clock: task release to terminal result, including tools, orchestration, retries, backoff, grading, repair feedback and final checking.

Partial tool-call JSON should not count as usable, and visible answer text should be timed separately from reasoning events. A response arriving in one burst may leave streaming throughput undefined. Tokens divided by total request time needs a separate label. Use a monotonic client clock and retain tool and retry timestamps. This protocol proposes reporting medians and the 90th and 95th percentiles, with sample counts and the quantile method. Separate request-level from task-level distributions, including failure durations and timeout counts.

Publish auditable costs and reproducible results

Completed-task cost should equal spending across all attempts, including failed runs, divided by successful run slots. Place it beside success rate and total spend; if nothing succeeds, report no successful completions. Preserve raw usage fields for uncached input, cache reads, cache writes, visible output, billed reasoning and other charged categories. Check whether output totals already include reasoning. Then add failed-request charges, tools and execution costs without double counting. Reconcile client estimates with provider billing.

Publish cash paid after credits, charges before promotional credits and normalized list-price cost separately. Mercury needs both launch-rate and standard-rate normalization because Inception publishes both schedules alongside a free-token offer. Keep API spending separate from fully loaded execution cost. Record human time with any hourly-rate assumption explicit. Calculate cost per first-attempt success from initial-attempt costs. Treat missing usage data separately from a confirmed zero charge. Sources: Inception

A reproducible release should include harness code, adapters, lockfiles, container digests, repository commits, task fixtures, graders, redacted traces and per-run cost records. Commit private-test hashes before execution and release tests afterward where licensing permits. Provide a way to rebuild every results table without changing usage counters, timestamps or outcome labels during redaction. Pin immutable model versions where possible and retain response metadata for mutable aliases. Before adopting a provider, check whether its ranking holds at standard prices, with cold caches and after execution costs.