OAAX 2.0 is out: one AI model, any accelerator, one runtime API

TL;DR

• OAAX 2.0 is a stable release of the specification, runtimes and toolchains. It replaces OAAX 1.x, which is now end of life.
• One model, any chip: convert an ONNX model once per accelerator and run it through the same C runtime API, with runtimes for CPU, Intel, NVIDIA, DEEPX and Hailo.
• A runtime API built for real applications: several models per runtime, asynchronous requests matched by id, timeouts, and 19 structured status codes.
• Tools to check and fetch: oaax-conformance tests a runtime or toolchain against the specification; oaax-manager downloads checksum-verified builds.
• Docs rebuilt around one example: the Hello World takes about ten minutes at docs.oaax.org.
• Open: everything is Apache-2.0, developed under the LF AI & Data Foundation.

OAAX 2.0 is published as a stable release. You take one ONNX model, convert it once per chip, and run it through the same C runtime API on a CPU, an Intel GPU, an NVIDIA GPU, a DEEPX NPU or a Hailo NPU. Your application code does not change when the hardware does.

That is the problem OAAX exists for. AI hardware is fragmented: every vendor ships its own SDK, its own runtime and its own conversion tools, so moving to a new chip usually means rewriting the integration. OAAX (Open AI Accelerator eXchange) standardizes the two pieces every chip needs: a toolchain, a Docker image that turns an ONNX model into the format the chip wants, and a runtime, a shared library that loads that model and runs inference through one common API. It is an open-source project of the LF AI & Data Foundation, initiated by Network Optix and developed in the open with hardware vendors and software builders.

Today we are releasing version 2.0 of the specification, runtimes and toolchains for five accelerator families, the tools to download and check them, and documentation rebuilt around a first run you can finish in about ten minutes.

Where OAAX 1.x left us

OAAX 1.0 proved the idea: a Docker toolchain on one side, a shared library with a small C API on the other, and ONNX in between. It also showed where a first version is too thin once real applications and several vendors depend on it.

The 1.x runtime API was built around one model and one call at a time:

• One model per runtime. runtime_model_loading took a single file path. Running two models meant two copies of the runtime.
• No way to match an output to its input. send_input and receive_output had no request id, so an application had to rely on the order of the results.
• Polling. receive_output returned non-zero when nothing was ready, with no timeout to wait on.
• Pass or fail, nothing in between. Every function returned 0 or non-zero. Whether a failure was a bad input, a missing device or a bug was only in an error string.
• Lifecycle left to interpretation. The API said initialization is "called only once". What happens on a second call, or after cleanup, was up to each vendor.

The last point matters most for a standard. When two runtimes behave differently in the corners, "swap one for the other without code changes" stops being true.

What 2.0 resolves

OAAX 2.0 is a breaking change, on purpose. It is not compatible with 1.x, and runtimes and host applications need to move to the new API. In exchange:

OAAX 1.x has reached end of life: it will get no further fixes or releases. There is no separate migration guide. The 2.0 API is small, the list below maps each 1.x behaviour to its 2.0 counterpart, and the Specification and the YOLOX example show every call in use.

• One model per runtime, loaded from a file path becomes: runtime_load_models loads several models, each addressed by its index.
• Output matched to input by order becomes: each input carries an id, and the runtime copies it onto the output.
• Polling for results becomes: runtime_retrieve_output waits up to a timeout you give, and many requests can be in flight.
• 0 or non-zero becomes: 19 structured status codes plus a readable error message.
• Lifecycle open to interpretation becomes: runtime_init, runtime_cleanup and a second runtime_init have defined results (ALREADY_INITIALIZED).
• No way to ask a runtime what it is becomes: runtime_get_name, runtime_get_version and runtime_get_info.

The contract around the API got stricter too. The toolchain side has a written contract: a Docker image that takes an input path and an output directory, must not need root and should not need the network. And the standard now comes with the tools to check it:

• oaax-conformance checks a runtime library or a toolchain image against the specification. Each failure names the clause it breaks, and reports come as console, HTML, JSON and JUnit.
• oaax-manager lists and downloads runtimes and toolchains, verifies each file against a pinned SHA-256 and unpacks it all-or-nothing.
• Published versions never change. Every release lists the URL, size and SHA-256 of each file, and released files are write-once in the bucket.

The documentation at docs.oaax.org was rebuilt around one example, the official YOLOX detector, so that every chip page shows the same steps. Every command in it was run in a clean container before we published.

What is released today

Everything below is public and Apache-2.0. Binaries live in the OAAX bucket; each release page lists the URL, size and SHA-256 of every file.

• OAAX Specification v2.0.0: the runtime interface, the toolchain contract, and the docs at docs.oaax.org
• OAAX CPU v2.0.0: the reference runtime and toolchain for Linux x86_64 and aarch64, Windows x86_64
• OAAX Intel v2.0.0: runtime and toolchain for Intel CPU, GPU and NPU
• OAAX NVIDIA v2.0.0: ONNX Runtime and TensorRT backends for Linux x86_64, Jetson, DGX Spark, and Windows for ONNX Runtime
• OAAX DEEPX v2.0.0: runtimes for Ubuntu 20.04, 22.04 and 24.04 (x86_64 and aarch64) and the toolchain
• OAAX Hailo v2.0.0: runtimes for HailoRT 4.17 to 4.20 (x86_64 and aarch64) and the toolchain
• OAAX Examples v2.0.0: host applications, including the YOLOX example used by the docs

And the tools around them:

• oaax-manager: lists and downloads runtimes and toolchains with checksum verification
• oaax-conformance: checks a runtime or toolchain against the specification
• oaax (PythonBindings): Python bindings that load any OAAX runtime and drive it with NumPy arrays

To be precise about "supported": the documentation authors ran the YOLOX example end to end on the CPU runtime, an Intel CPU, integrated GPU and NPU (Linux), an NVIDIA RTX A4000 (ONNX Runtime and TensorRT), a DEEPX NPU and a Hailo-8, and got the same two detections, a bicycle and a dog, on all of them. Hailo-8L, Jetson and DGX Spark builds are published but were not run by the docs authors. The Compliant AI Accelerators page keeps this list current.

Get started in about ten minutes

The fastest way to see OAAX work is the Hello World page. It takes the official YOLOX detector through both stages and prints what it finds in a photo. On the CPU it is a handful of commands on a clean Ubuntu machine:

git clone https://github.com/OAAX-standard/examples.git && cd examples/yolox-example
cmake -S . -B build && cmake --build build     # builds the small C host
bash fetch_model.sh                            # official YOLOX-s, SHA-256 checked
uvx oaax-manager fetch runtime cpu             # the CPU runtime, checksum-verified
./build/yolox_host runtime-cpu-2.0.0-linux-x86_64/libRuntimeLibrary.so models/yolox_s.onnx artifacts/dog-bicycle.jpg


The host prints the runtime's name and the detections, and writes a picture with the boxes drawn:

Runtime: OAAX CPU Runtime 2.0.0
Detections (score >= 0.30, NMS IoU 0.45): 2
 bicycle        0.953  ( 616,  210) - (1228,  806)
 dog            0.934  ( 216,  244) - ( 521,  587)


Then change the chip, not the code. The same host runs with a different runtime library and a model converted for it. uvx oaax-manager list shows every runtime and toolchain; uvx oaax-manager fetch toolchain <chip> downloads a conversion toolchain as a Docker image. Each chip has a page on docs.oaax.org with its requirements and its conversion step.

Where to go next:

• Understand the standard: Welcome to OAAX, the FAQ and the Specification.
• Call a runtime from your own program: the examples repository (C), or the Python bindings: uv pip install oaax.
• Check a runtime or toolchain you already have: uvx --python 3.12 --from 'oaax-conformance[e2e]' oaax-conformance runtime ./libRuntimeLibrary.so --model model.onnx.

Contribute

OAAX is built in the open and the contribution path is meant to be short. There are three good ways in.

1. Bring a new accelerator. A chip needs two things: a runtime library that implements the C API, and a toolchain Docker image that turns ONNX into the chip's format. Start from the reference implementation, read the Specification, and use oaax-conformance as your acceptance check: it tells you which clause a failing behaviour breaks. The Contributing Guide lists the rest.
2. Improve a chip that exists. Each *-acceleration repository has open issues, and a backlog of about forty small ones from the 2.0 documentation review is filed across the organization: docs fixes, CI hardening, test gaps. Pick one.
3. Shape the standard. Open an issue or a discussion on the OAAX repository. Larger changes go to the technical steering committee, whose agenda, meeting notes and roadmap are public in the steering-committee repository.

The practical rules are short: commits need a DCO sign-off (git commit -s), and questions are welcome in GitHub Discussions or in the issue tracker of the repository concerned. Vulnerabilities go to security@oaax.org or through GitHub's private reporting, not to a public issue.

Where OAAX is going

2.0 settles the shape of the standard. The next steps are about closing its remaining gaps and widening the set of chips.

• OAAX 2.1. A few behaviours are deliberately left open in 2.0, for example what a second runtime_load_models call does and what happens when a process exits without calling runtime_cleanup. 2.1 is planned to pin them down, add a bundle manifest so a converted model describes itself, report model inputs and outputs through runtime_get_info, and report failures per request. The manifest proposal is already open as a draft for review.
• More chips. SiMa, MediaTek, MemryX and EdgeCortix are planned for 2.1. If your accelerator is not on the list, the contribution path above is how it gets there.
• A release policy. The proposal is that a published version never changes, and that versions, tags and releases follow one convention across every repository. The technical steering committee takes it up at its next meeting, on 10 November.
• Proof, not promises. The conformance suite exists today, and it is the basis for publishing a conformance report next to each runtime release.
• Versioned documentation, so that 2.0 pages stay readable once 2.1 exists.

OAAX 2.0 is a starting point you can build on today. If you try it, tell us what broke or what was missing: that is the fastest way it gets better.

Thank you

OAAX 2.0 is the work of engineers at Network Optix, DEEPX and MemryX, independent community contributors, and the LF AI & Data Foundation, which hosts the project. Over the past twelve months, five people landed more than 480 commits across the OAAX repositories, three of them from outside Network Optix. Thank you to everyone who wrote code, reviewed the specification, tested a runtime on real hardware, filed an issue or sat on the technical steering committee. A standard is only as good as the people who use it and push back on it.