ExecuTorch on Cortex-M: Measure firmware size, memory, and latency

What the Cortex-M announcement measured

PyTorch reported DS-CNN inference in 5.825 ms on a single Alif E8 Cortex-M55 core at 400 MHz on October 15, 2026. The chip’s neural processing units were unused. The configuration used 8-bit integer quantization, selective builds, compiler version GCC 15.2.1, and disabled logging. Those settings belong with the result: reproducing performance means reproducing the executable and its operating conditions. Sources: PyTorch

The table below shows the announcement’s four MLPerf Tiny model results, which its authors measured outside an official MLPerf submission. The announcement showcased an existing deployment path; initial Cortex-M support appeared in the 1.0-era release notes. Later releases added layout handling and corrections to the backend. Establish a working deployment before trying to match the published figures. Sources: PyTorch, GitHub

Pin versions and inspect the exported graph

Choose an exact ExecuTorch release or commit and include it in the build record. The Pico 2 tutorial is labeled ExecuTorch 1.5. The backend reference describes the main development branch and labels the Cortex-M backend beta. Release history records changes to Cortex-M lowering and kernels. Do not treat the release and main-branch references as interchangeable. Record PyTorch, TorchAO, the Arm kernel library, the board SDK, and compiler versions alongside the ExecuTorch revision. Sources: PyTorch, GitHub

The backend replaces supported quantized operations with Arm neural-network kernel calls. It selects implementations through the CPU target setting. Documented coverage includes convolution, depthwise convolution, linear operations, pooling, and softmax. Quantization uses symmetric 8-bit integers. Convolution uses per-channel quantization; other supported operations use per-tensor quantization. The library’s 4-bit weights, 16-bit activations, and per-channel fully connected configurations are not integrated into this backend. Inspect the exported operators, shapes, and layouts before assuming an entire model will receive acceleration. Sources: PyTorch

1. Set the model to evaluation mode and export its graph.
2. Prepare the graph with the Cortex-M quantizer and PyTorch’s export quantization workflow.
3. Calibrate with representative inputs, then convert the quantized graph.
4. Re-export, apply the Cortex-M lowering passes, and serialize the program. Sources: PyTorch

Build and flash the Pico 2 example

Use the official Pico 2 example to establish export, firmware compilation, flashing, and logging. Its classifier is a handcrafted demonstration for digits 0, 1, 4, and 7. The tutorial provides a portable 32-bit floating-point variant, abbreviated FP32, and an accelerated 8-bit integer variant, abbreviated INT8. The following commands export and build both variants from the repository root. Preserve the first build’s artifacts before starting the second. Sources: PyTorch

# Export both variants.
cd examples/raspberry_pi/pico2
python export_mlp_mnist.py
python export_mlp_mnist_cmsis.py
cd ../../..

# Build FP32 firmware.
./examples/raspberry_pi/pico2/build_firmware_pico.sh \
  --model=balanced_tiny_mlp_mnist.pte

# Preserve the first build's artifacts before building INT8.
./examples/raspberry_pi/pico2/build_firmware_pico.sh \
  --cmsis --model=balanced_tiny_mlp_mnist_cmsis.pte

The build embeds the model as generated C data and produces executorch_pico.uf2 in the example’s build directory. Flash that file through the Pico’s USB boot-selection workflow. The quantized variant retains floating-point inputs and outputs, with quantization and dequantization inside the graph. Once both variants run, replace the demonstration with a small trained model and a held-out evaluation set. Keep identical serialized inputs for both builds so that measurements describe the same workload. Sources: PyTorch

Measure firmware and account for all memory

Record model size, occupied firmware flash, static memory, and runtime memory peaks separately. Count the model artifact’s exact bytes. Inspect executable sections and the linker map for flash occupancy. Do not use the executable or flashing file’s length as the measure of occupied flash. The proposed worksheet below separates storage, reserved capacity, and observed usage. Fill it out for both builds before comparing their footprints.

Memory outside the tensor arena needs explicit accounting. The Arduino integration reports 563,672 bytes of flash and 47,084 bytes of memory for its UNO Q keyword-spotting example. That example uses board core 0.90.0. In that configuration, a 28 KB arena fails method loading, while the example uses 40 KB. Treat these as observations about that integration. Reconcile the target application’s static storage, allocator peaks, stack, and buffers. If the arena already occupies static storage, avoid adding its capacity again to the total. Sources: GitHub

Time inference and the full application separately

Place a target timer or cycle counter around inference and keep serial printing outside the timed region. Declare a warm-up period and a fixed number of measured runs before collecting results. Report the median, upper-percentile latency, minimum, and maximum, and retain the raw observations. Measure initialization and cold startup separately. Then time the complete application path, including preprocessing and output handling, to establish the budget that matters to users.

The announcement’s Renesas Cortex-M85 µYOLO example illustrates why timing boundaries and memory placement matter. It reports 24 ms inference and 12.7 frames per second for the complete application. Moving expensive operators’ weights into data tightly coupled memory and enabling the instruction cache reduced reported inference time from 191 ms to 24 ms. The repository demo targets simulation; the physical-board port was outside the repository. Record cache settings, weight placement, CPU clock, and enabled instruction features, then repeat measurements with realistic peripheral and interrupt activity. Sources: PyTorch

Plan numerical validation before deployment

The following checklist proposes comparisons for deployment validation. The Pico 2 tutorial demonstrates prediction correctness, but does not provide a full board-output capture and host-comparison procedure. Completing this checklist requires that procedure, including input serialization, output precision, and tolerance checks. Choose acceptance thresholds in advance. Keep evaluation samples separate from calibration data, and use the comparisons below to identify where outputs change. Sources: PyTorch

1. Compare eager floating-point execution with the exported floating-point graph; record maximum absolute error and failed tolerances.
2. Compare the exported floating-point graph with the quantized host graph; record mean absolute error, task quality, and prediction agreement.
3. Compare the quantized host graph with board outputs; retain output errors, mismatch counts, and the worst input cases.
4. Compare eager floating-point execution with board outputs; report the overall change in accuracy or the relevant task metric.

Use identical serialized inputs throughout, preserving preprocessing, tensor order, shape, data type, scales, and zero points. For classification, report agreement on the highest-ranked prediction and accuracy against labels. Matching predictions can still be wrong. Include absolute error; relative error alone can mislead near zero. Archive the model hash, calibration and evaluation data, and output tensors. Also retain compiler settings, selected operators, linker map, executable, and raw timings. Approve deployment against explicit flash, memory, latency, and task-quality budgets. Retain the passing artifacts for comparison after the next update.