# ONNX

### ONNX Runtime in PySDK

PySDK supports version 1.20.1 of ONNX Runtime, on Linux, Windows, and Mac.

To install the ONNX runtime, download an 1.20.1 archive for your system [here](https://github.com/microsoft/onnxruntime/releases/tag/v1.20.1). Do not use the "-training-" archives. Then, extract it into an appropriate directory:

* On Windows, extract the archive to `C:\Program Files`, `C:\Program Files (x86)`, or `C:\ProgramData`.
* On Linux, extract the archive to `/usr/local/`.
* On Mac, extract the archive to `/usr/local/`.

{% hint style="warning" %}
Do not rename the `onnxruntime-<os>-<architecture>-1.20.1` directory after you extract it.
{% endhint %}

### AMD Ryzen™ AI in PySDK

PySDK supports AMD Ryzen™ AI version 1.2.

To install AMD Ryzen™ AI, click [this link](https://ryzenai.docs.amd.com/en/latest/inst.html) to read installation instructions.

The installation includes: (1) an NPU driver and (2) a "RyzenAI Software MSI installer."

{% hint style="warning" %}

* Make sure to launch the driver `npu_sw_installer.exe` from the terminal while in admin mode.
* Don't forget to append your `C:\path\to\miniconda3\Scripts` to the `Path` System environment variable before launching the installer.
* Restart the terminal (if using VS Code, restart all open windows) after installation.
  {% endhint %}

Run `conda activate <ryzen-ai-env-name>` to enter the environment created by the installation wizard.

{% hint style="info" %}
You may optionally set set Ryzen™ AI environment variables. PySDK initializes the defaults for your processor type automatically. For more information, see the notes below.
{% endhint %}

At this point, you should be ready to run inference on models prepared for the Ryzen NPU from the DeGirum AI Hub Model Zoo.

#### About environment variables

Ryzen™ AI requires certain environment variables to be set depending on the processor configuration:

* Phoenix (PHX): AMD Ryzen™ 7940HS, 7840HS, 7640HS, 7840U, 7640U.
* Hawk (HPT): AMD Ryzen™ 8640U, 8640HS, 8645H, 8840U, 8840HS, 8845H, 8945H.
* Strix (STX): AMD Ryzen™ Ryzen AI 9 HX370, Ryzen AI 9 365.

#### About your processor

To find your processor configuration, go to **Settings -> System -> About**.

If you want to manually specify the environment variables, then either in Windows CMD, Powershell, or inside a Python script run the following:

* If your processor is Phoenix or Hawk (PHX/HPT):

  * CMD:

  {% code overflow="wrap" %}

  ```bash
  set XLNX_VART_FIRMWARE=%RYZEN_AI_INSTALLATION_PATH%voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin
  set XLNX_TARGET_NAME=AMD_AIE2_Nx4_Overlay
  ```

  {% endcode %}

  * Powershell:

  {% code overflow="wrap" %}

  ```powershell
  $env:XLNX_VART_FIRMWARE="$env:RYZEN_AI_INSTALLATION_PATH"+"voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin"
  $env:XLNX_TARGET_NAME="AMD_AIE2_Nx4_Overlay"
  ```

  {% endcode %}

  * Python:

  {% code overflow="wrap" %}

  ```python
  import os
  os.environ['XLNX_VART_FIRMWARE'] = os.environ['RYZEN_AI_INSTALLATION_PATH'] + 'voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin'
  os.environ['XLNX_TARGET_NAME'] = "AMD_AIE2_Nx4_Overlay"
  ```

  {% endcode %}
* If your processor is Strix (STX):

  * CMD:

  {% code overflow="wrap" %}

  ```bash
  set XLNX_VART_FIRMWARE=%RYZEN_AI_INSTALLATION_PATH%voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin
  set XLNX_TARGET_NAME=AMD_AIE2P_Nx4_Overlay
  ```

  {% endcode %}

  * Powershell:

  {% code overflow="wrap" %}

  ```powershell
  $env:XLNX_VART_FIRMWARE="$env:RYZEN_AI_INSTALLATION_PATH"+"voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin"
  $env:XLNX_TARGET_NAME="AMD_AIE2P_Nx4_Overlay"
  ```

  {% endcode %}

  * Python:

  {% code overflow="wrap" %}

  ```python
  import os
  os.environ['XLNX_VART_FIRMWARE'] = os.environ['RYZEN_AI_INSTALLATION_PATH'] + 'voe-4.0-win_amd64/xclbins/strix/AMD_AIE2P_Nx4_Overlay.xclbin'
  os.environ['XLNX_TARGET_NAME'] = "AMD_AIE2P_Nx4_Overlay"
  ```

  {% endcode %}

#### Changing environment with Python

When setting environment variables with Python, put the code lines before `import degirum` for PySDK to see the changes.

**These variables have to be set every time you open a fresh terminal to run inference.** In case you want to set them permanently, go to `Edit the system environment varibles` in Windows search and add (or edit) the two variables through the GUI.

{% hint style="warning" %}
Incorrect environment variables may lead to a crash. PySDK detects when the value of `env:XLNX_VART_FIRMWARE` doesn't match the CPU type and automatically sets it to the one appropriate for your device to prevent a system crash. However, be careful when manually redefining the environment variables.
{% endhint %}

#### About NPU Configurations:

The instructions above correspond the "Standard Configuration" of the NPU. An additional "Benchmark Configuration" is supported by Ryzen™ AI. The setup process is similar (requires different environment variable values) and is described in detail on the [Ryzen™ AI Runtime Setup Page.](https://ryzenai.docs.amd.com/en/latest/runtime_setup.html)

{% hint style="info" %}
Models will be recompiled when run with a different NPU configuration. PySDK detects when an inference on a model is launched in a configuration different from the one it was compiled for (if the cache exists) and recompiles it at runtime to prevent a crash. So expect some initial delay when trying a different NPU configuration.
{% endhint %}

#### About supported model format

DeGirum AI Hub Model Zoo supplies INT8 symmetrically quantized models required by the Ryzen™ AI NPU. Models come with precompiled caches to bypass the sometimes lengthy compilation process and enable immediate inference. If recompilation is needed or if cached models cause errors, remove the `<model_name>_cache` subdirectory from the downloaded model directory. Inference will then proceed with just-in-time compilation automatically.