SGLang 0.5.22 adds Apple Silicon to its standard PyTorch runner

What the standard runner supports

SGLang announced standard PyTorch backend support for Apple Silicon in version 0.5.22 on October 19, 2026. The change makes a Mac a plausible place to exercise application requests before deploying elsewhere. Its value depends on which behaviors a local test reproduces, from response parsing to cache reuse. Sources: SGLang on GitHub

An earlier contributor report described both Torch/MPS and MLX serving on a Mac. The new implementation integrates Apple support into the standard runner through PyTorch’s Metal Performance Shaders, or MPS, device path. It uses native Torch attention, PyTorch sampling, eager execution, and disabled overlap scheduling. Those choices set the boundaries for a local prototype. Sources: SGLang on GitHub, GitHub Gist

The table below summarizes the implementation’s execution choices and exclusions. Low-rank adaptation, or LoRA, has a backend restriction rather than a blanket prohibition. Any production workflow that depends on an excluded feature needs another environment for that portion of testing. Sources: SGLang on GitHub

Pin the installation and establish a baseline

Record the Mac chip, GPU core count, physical memory, macOS build, power mode, Python version, and background memory pressure. SGLang 0.5.22 requires Python 3.11 or newer. PyPI lists uploads dated October 16, 2026, and six Linux wheels; that file list does not establish a macOS wheel installation route. SGLang-Omni documents a source installation using version 0.5.22 and the all_mps extra, though its installer also handles audio dependencies. Sources: SGLang on GitHub, SGLang-Omni, PyPI

The commands below form an untested reproduction recipe; the benchmark results later in this guide are contributor measurements without independent reproduction. The source installation follows the release and extra documented by SGLang-Omni. Save dependency resolution and build failures, and keep the release pinned rather than silently switching branches. Sources: SGLang-Omni

git clone --branch v0.5.22 --depth 1 https://github.com/sgl-project/sglang.git
cd sglang
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e './python[all_mps]'
python -m pip freeze > environment.txt

Check whether PyTorch exposes an MPS device before launching. The candidate launch follows the implementation’s validation setup, selecting Torch, Qwen3-0.6B, and a static memory fraction of 0.6. Before measuring, replace the model identifier with a downloaded snapshot pinned to a recorded revision. Preserve startup logs and resolved arguments so later runs can be compared. Sources: SGLang on GitHub

python -c 'import torch; print(torch.__version__); print(torch.backends.mps.is_available())'

SGLANG_USE_MLX=0 python -m sglang.launch_server \
  --model-path Qwen/Qwen3-0.6B \
  --device mps \
  --mem-fraction-static 0.6 \
  --host 127.0.0.1 \
  --port 30000

Test application behavior before throughput

Start with a small request through the interface the application uses. The earlier contributor example demonstrates the Chat Completions route, making it a useful test target for this runner. Capture the response status, schema, model identifier, token usage, and finish reason. A successful response alone does not establish compatibility. Sources: GitHub Gist

curl --fail-with-body http://127.0.0.1:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [
      {"role": "user", "content": "Return the word ready."}
    ],
    "temperature": 0,
    "max_tokens": 32,
    "stream": false
  }'

Run the same fixture against production, matching the model, tokenizer, conversation template, and precision wherever possible. If those differ, assess transport compatibility separately from generated content. Use the proposed checks below according to the application’s dependencies. These are tests to perform, not promises that every feature works.

A valid JSON response alone does not establish schema enforcement. Likewise, an accepted sampling parameter does not show that the runner applied it. Test difficult schemas, measurable parameter changes, and malformed requests. Keep unsupported-configuration startup checks separate from valid serving runs, and check that failures are clear and prompt.

Measure memory throughout serving

The implementation sizes memory using Metal’s recommended budget and driver allocations. An idle process reading therefore gives an incomplete picture of whether a workload fits. Measure startup and request peaks alongside steady-state behavior. The proposed sequence below also checks whether memory use recovers after traffic stops. Sources: SGLang on GitHub

1. Measure the idle machine before launching the server.
2. Capture startup, post-load idle, first-request, and warmed-up memory use.
3. Increase input lengths within the configured context limit.
4. Repeat with increasing request concurrency.
5. Compare unique prefixes with a shared prefix.
6. Run sustained mixed traffic with cancellations, then measure recovery during idle.

Capture process-tree resident set size, system memory pressure, swap changes, runtime allocation counters, and configured token capacity. Preserve actual request lengths so comparisons reflect the work performed. Keep accounting definitions separate and check for overlap before adding counters together. Allocator instrumentation must observe the server worker; a separate Python monitoring process would measure its own allocator state.

Separate throughput from user latency

The implementation contributor reported 72.94 output tokens per second for Qwen3-0.6B on an M4 Pro with 48 GB of memory. The workload used 40 requests, 256 input tokens, 128 output tokens, and concurrency eight. Median time to first token was 186.48 milliseconds, and median time per output token was 104.33 milliseconds. Aggregate output rate answers a different question from how long an individual user waits. Sources: SGLang on GitHub

SGLang’s indexed benchmark guide describes concurrency controls, compatible chat backends, and detailed output. Check the installed benchmark’s help output before using the candidate command below. Treat it as a proposed baseline, then repeat with higher concurrency and separate output files. Confirm actual token lengths instead of assuming every request reaches its output limit. Sources: SGLang

python -m sglang.bench_serving \
  --backend sglang-oai-chat \
  --host 127.0.0.1 --port 30000 \
  --model Qwen/Qwen3-0.6B \
  --dataset-name random \
  --random-input-len 256 \
  --random-output-len 128 \
  --random-range-ratio 1 \
  --num-prompts 200 \
  --max-concurrency 1 \
  --output-file mac-c1.jsonl \
  --output-details

Exclude warmup explicitly and repeat the measurements. Report successful requests, failures, timeouts, actual token counts, elapsed time, and output throughput. Include typical and tail latency for the first token and complete response, plus latency between tokens. Keep failures in the denominator and avoid strong tail-latency conclusions from small samples.

Next, mix short and long prompts and add controlled arrival rates to expose queue growth. Separate cold-cache runs from shared-prefix runs while recording power conditions and memory pressure. Client concurrency alone does not establish simultaneous GPU execution. Use logs or instrumentation to distinguish queueing from effective batching.

Define what a passing Mac test establishes

Treat the Mac as a functional staging environment once the application’s own contract tests pass. The implementation’s generation and radix-cache reuse test supports including cache behavior in that checklist. A useful result identifies exactly which request formats, stream parsers, errors, and cancellation behaviors match production. Sources: SGLang on GitHub

Match backend comparisons to the release and workload. An earlier M1 report used 60 input tokens and 10 output tokens for its Torch-versus-MLX comparison. Its older implementation and short workload do not establish a performance gap for version 0.5.22. Choose a backend against the application’s required operations, measured memory peaks, and latency budget. Sources: GitHub Gist

Save a compatibility record with the pinned environment, model revision, passing tests, failures, and workload limits. Test structured outputs, tool calls, log probabilities, and LoRA wherever the application depends on them. Keep production capacity and distributed execution tests on the deployment configuration they are meant to qualify.