vLLM Ascend v0.30.0rc1: a Qwen3.8-27B deployment plan awaiting validation

The release sets specific support boundaries

The vLLM Ascend v0.30.0rc1 release notes declare support for four models within specified hardware and configuration limits. Released on October 11, 2026, the community-maintained hardware plugin aligns with upstream vLLM v0.30.0. These are separate releases, and the plugin’s declared scope does not establish support for other model and hardware combinations.

The notes list DeepSeek V4.1 on Atlas 800I A3 and A2, with quantization, Model Runner V2, torch.compile and prefill/decode separation. They list GLM-5.3 with quantized multi-node serving, sparse packed cache and DSpark on those hardware families. GLM-5.3-Flash lists quantization, prefill/decode separation and key-value cache pooling on A3 and A2. Qwen3.8-27B lists BF16/W8A8 configurations on A3 and A2, plus W8A8-310P on Atlas 300I DUO, with hybrid attention and non-contiguous cache.

The notes also report that default asynchronous scheduling was rolled back pending hardening and a block-table optimization was reverted for correctness. This proposed walkthrough uses Qwen3.8-27B W8A8 on one Atlas 800I A2 node, with optimizations tested separately. The procedure was not run on Ascend hardware. The research did not establish an immutable release-candidate image digest or independently reproduced benchmarks for this release.

Pin the runtime before loading weights

The compatibility matrix pairs vLLM Ascend 0.30.0rc1 with vLLM 0.30.0 and Ascend runtime software CANN 9.2.0. It specifies PyTorch 2.10.0, torch_npu 2.10.0.post4, Triton Ascend 3.2.2 and Mooncake 0.3.11.post1. Python must be at least 3.10 and below 3.13. The development matrix lists a different runtime version and a verified upstream commit, so keep release and development installation instructions separate.

Obtain the release-candidate container tag and immutable digest for the selected hardware, then verify its contents against the matrix. The Quick Start illustrates v0.23.0 images, while the Qwen tutorial describes validation against 0.23.0 and accuracy examples labeled 0.23.0rc1. Those examples do not establish compatibility or accuracy for this release. Follow the hardware-specific device and host-driver requirements, including A3’s additional device-mapping requirements.

Before loading weights, run npu-smi info and python -m pip check, then save python -m pip freeze output as environment.txt. Import torch, torch_npu, vllm and vllm_ascend; check torch.npu.is_available() and record torch.npu.device_count(). Save vllm serve --help and vllm bench serve --help to verify available options. Record the accelerator, usable memory, device mapping, driver, firmware, operating system, Python version, environment variables and container digest. Add model and tokenizer revisions, quantization metadata and file hashes.

Start with a constrained eager baseline

Use a local, revision-pinned W8A8 checkpoint at /models/Qwen3.8-27B-w8a8 and set MODEL_PATH to that directory. Set HCCL_BUFFSIZE=512 and PYTORCH_NPU_ALLOC_CONF=expandable_segments:True. The proposed baseline adapts the Qwen tutorial’s tensor parallelism of two and memory utilization of 0.85. Test eager execution first, then evaluate prefix caching, graph execution and speculative decoding separately.

Launch vllm serve "$MODEL_PATH" with --host 0.0.0.0 --port 8000 --served-model-name qwen3.8 --tensor-parallel-size 2 --data-parallel-size 1 --quantization ascend. Add --max-model-len 8192 --max-num-seqs 8 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.85 --no-enable-prefix-caching --enforce-eager. Verify these options against the saved release-candidate help before execution. This adapted configuration awaits hardware validation.

Retain startup logs showing the accelerator backend, checkpoint format, effective cache allocation and available token capacity. Successful imports alone do not establish serving readiness. Send a request to /v1/chat/completions using model qwen3.8, the prompt “What is 17 plus 25?”, max_completion_tokens of 1024 and temperature of 0.7. Save the response, finish reason, token counts and errors. A successful response establishes only basic functionality.

Change cache behavior one setting at a time

Keep memory utilization at 0.85 and context capacity at 8,192 tokens initially. Measure peak memory and stability before increasing allocation, and leave output headroom when extending input length. Retain the checkpoint and backend’s supported cache precision. Weight quantization does not establish support for an arbitrary cache quantization mode.

Enable prefix caching separately, comparing repeated and unique prefixes under matched workloads. Keep external storage and speculation disabled until the baseline passes correctness checks. Searchable KV Pool documentation describes AscendStoreConnector using Mooncake, Memcache or Yuanrong to store and reuse cache across instances. The full page could not be fetched during research, leaving the current connector configuration unverified.

For a pooling follow-up, record backend version, transport, host memory allocation, solid-state drive configuration, connector roles and cache namespace. Test fresh and warmed stores, eviction and restart separately. DFlash2 documentation says Model Runner V2 requires an eager draft, while the target can use graph execution. Its example disables prefix caching, and eight speculative tokens require graph capture sizes accommodating nine tokens per request. These restrictions do not establish that every feature combination works in this release.

Require correctness checks for every optimization

Create a fixed prompt set and compare results with an independently validated reference configuration. Predetermine acceptable accuracy differences and retain individual failures alongside aggregate scores. Use executable tests for coding tasks, schema validation for structured output and requested-fact scoring for retrieval. Record dataset revisions and generation settings to preserve evaluation conditions across runs.

Exercise long-context retrieval at several input lengths and evidence positions, then test concurrent requests with unrelated prefixes. Compare cold and warm answers when prefix caching is enabled. Apply cache pressure and examine results during preemption, eviction and reuse. For pooling, repeat checks after store restart and on cache misses.

Keep the checkpoint unchanged when comparing eager execution with graphs and non-speculative generation with speculation. Evaluate W8A8 against BF16 separately because that comparison changes numerical representation and may also change execution. Independent key-value cache research supports measuring task quality alongside performance, but its Llama and Mistral experiments do not validate this Ascend release. The older Qwen tutorial’s accuracy examples cannot substitute for results from the pinned environment.

Measure throughput against a latency target

The official benchmark guide demonstrates vllm bench serve with /v1/completions, ShareGPT input and --base-url for remote services. Its ten-prompt example demonstrates functionality rather than production capacity. For a proposed synthetic baseline, use --backend vllm --base-url http://127.0.0.1:8000 --endpoint /v1/completions --model "$MODEL_PATH" --served-model-name qwen3.8. Add --dataset-name random --random-input-len 1024 --random-output-len 256 --num-prompts 1000 --request-rate 1 --seed 42 --save-result --result-dir results/baseline. These workload values are editorial test choices; check the saved help and actual output lengths before comparing results.

Measure input/output profiles of 1,024/256 tokens, 1,024/1,024 tokens and 6,144/256 tokens. Sweep offered rates such as 1, 2, 4 and 8 requests per second, with at least three measured repetitions after warm-up. Separate startup and compilation time from steady-state results. Record successes, failures, achieved request rate, input and output throughput, and median, 95th-percentile and 99th-percentile latency. Include time to first token, time per output token, end-to-end latency, peak device memory, cache occupancy, preemptions and actual token-length distributions.

Record topology, power mode, client placement and warm-up procedure, plus cache hits, draft acceptance or storage transfers when applicable. Publish throughput at an explicit latency target and preserve the previous validated configuration for upgrade comparisons. Indexed excerpts from an issue report described lower GLM-5.2 prefill throughput on v0.26.0rc; they do not establish a v0.30.0rc1 defect. Before adoption, confirm the image digest, qualified checkpoint revision and tested feature combinations with maintainers. Retain the environment manifest, launch script, fixed inputs, correctness report and raw benchmark JSON as reviewable deployment evidence.