# ONNX

### ONNX Runtime in PySDK

PySDK supports version 1.20.1 of ONNX Runtime, on Linux, Windows, and Mac.

To install the ONNX runtime, download an 1.20.1 archive for your system [here](https://github.com/microsoft/onnxruntime/releases/tag/v1.20.1). Do not use the "-training-" archives. Then, extract it into an appropriate directory:

* On Windows, extract the archive to `C:\Program Files`, `C:\Program Files (x86)`, or `C:\ProgramData`.
* On Linux, extract the archive to `/usr/local/`.
* On Mac, extract the archive to `/usr/local/`.

{% hint style="warning" %}
Do not rename the `onnxruntime-<os>-<architecture>-1.20.1` directory after you extract it.
{% endhint %}

### AMD Ryzen™ AI in PySDK

PySDK supports AMD Ryzen™ AI version 1.2.

To install AMD Ryzen™ AI, click [this link](https://ryzenai.docs.amd.com/en/latest/inst.html) to read installation instructions.

The installation includes: (1) an NPU driver and (2) a "RyzenAI Software MSI installer."

{% hint style="warning" %}

* Make sure to launch the driver `npu_sw_installer.exe` from the terminal while in admin mode.
* Don't forget to append your `C:\path\to\miniconda3\Scripts` to the `Path` System environment variable before launching the installer.
* Restart the terminal (if using VS Code, restart all open windows) after installation.
  {% endhint %}

Run `conda activate <ryzen-ai-env-name>` to enter the environment created by the installation wizard.

{% hint style="info" %}
You may optionally set set Ryzen™ AI environment variables. PySDK initializes the defaults for your processor type automatically. For more information, see the notes below.
{% endhint %}

At this point, you should be ready to run inference on models prepared for the Ryzen NPU from the DeGirum AI Hub Model Zoo.

#### About environment variables

Ryzen™ AI requires certain environment variables to be set depending on the processor configuration:

* Phoenix (PHX): AMD Ryzen™ 7940HS, 7840HS, 7640HS, 7840U, 7640U.
* Hawk (HPT): AMD Ryzen™ 8640U, 8640HS, 8645H, 8840U, 8840HS, 8845H, 8945H.
* Strix (STX): AMD Ryzen™ Ryzen AI 9 HX370, Ryzen AI 9 365.

#### About your processor

To find your processor configuration, go to **Settings -> System -> About**.

If you want to manually specify the environment variables, then either in Windows CMD, Powershell, or inside a Python script run the following:

* If your processor is Phoenix or Hawk (PHX/HPT):

  * CMD:

  {% code overflow="wrap" %}

  ```bash
  set XLNX_VART_FIRMWARE=%RYZEN_AI_INSTALLATION_PATH%voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin
  set XLNX_TARGET_NAME=AMD_AIE2_Nx4_Overlay
  ```

  {% endcode %}

  * Powershell:

  {% code overflow="wrap" %}

  ```powershell
  $env:XLNX_VART_FIRMWARE="$env:RYZEN_AI_INSTALLATION_PATH"+"voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin"
  $env:XLNX_TARGET_NAME="AMD_AIE2_Nx4_Overlay"
  ```

  {% endcode %}

  * Python:

  {% code overflow="wrap" %}

  ```python
  import os
  os.environ['XLNX_VART_FIRMWARE'] = os.environ['RYZEN_AI_INSTALLATION_PATH'] + 'voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin'
  os.environ['XLNX_TARGET_NAME'] = "AMD_AIE2_Nx4_Overlay"
  ```

  {% endcode %}
* If your processor is Strix (STX):

  * CMD:

  {% code overflow="wrap" %}

  ```bash
  set XLNX_VART_FIRMWARE=%RYZEN_AI_INSTALLATION_PATH%voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin
  set XLNX_TARGET_NAME=AMD_AIE2P_Nx4_Overlay
  ```

  {% endcode %}

  * Powershell:

  {% code overflow="wrap" %}

  ```powershell
  $env:XLNX_VART_FIRMWARE="$env:RYZEN_AI_INSTALLATION_PATH"+"voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin"
  $env:XLNX_TARGET_NAME="AMD_AIE2P_Nx4_Overlay"
  ```

  {% endcode %}

  * Python:

  {% code overflow="wrap" %}

  ```python
  import os
  os.environ['XLNX_VART_FIRMWARE'] = os.environ['RYZEN_AI_INSTALLATION_PATH'] + 'voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin'
  os.environ['XLNX_TARGET_NAME'] = "AMD_AIE2P_Nx4_Overlay"
  ```

  {% endcode %}

#### Changing environment with Python

When setting environment variables with Python, put the code lines before `import degirum` for PySDK to see the changes.

**These variables have to be set every time you open a fresh terminal to run inference.** In case you want to set them permanently, go to `Edit the system environment varibles` in Windows search and add (or edit) the two variables through the GUI.

{% hint style="warning" %}
Incorrect environment variables may lead to a crash. PySDK detects when the value of `env:XLNX_VART_FIRMWARE` doesn't match the CPU type and automatically sets it to the one appropriate for your device to prevent a system crash. However, be careful when manually redefining the environment variables.
{% endhint %}

#### About NPU Configurations:

The instructions above correspond the "Standard Configuration" of the NPU. An additional "Benchmark Configuration" is supported by Ryzen™ AI. The setup process is similar (requires different environment variable values) and is described in detail on the [Ryzen™ AI Runtime Setup Page.](https://ryzenai.docs.amd.com/en/latest/runtime_setup.html)

{% hint style="info" %}
Models will be recompiled when run with a different NPU configuration. PySDK detects when an inference on a model is launched in a configuration different from the one it was compiled for (if the cache exists) and recompiles it at runtime to prevent a crash. So expect some initial delay when trying a different NPU configuration.
{% endhint %}

#### About supported model format

DeGirum AI Hub Model Zoo supplies INT8 symmetrically quantized models required by the Ryzen™ AI NPU. Models come with precompiled caches to bypass the sometimes lengthy compilation process and enable immediate inference. If recompilation is needed or if cached models cause errors, remove the `<model_name>_cache` subdirectory from the downloaded model directory. Inference will then proceed with just-in-time compilation automatically.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.degirum.com/pysdk/runtimes-and-drivers/onnx-runtime.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.