Installation
PySDK Supported Configurations
The table below lists operating systems, CPU architectures, and Python versions supported by latest released PySDK version.
| Operating System | Supported CPU Architectures | Supported Python Versions |
|---|---|---|
| Ubuntu Linux 20.04, 22.04 | x86-64 | 3.8 ... 3.12 |
| Ubuntu Linux 20.04, 22.04 | ARM AArch64 | 3.8 ... 3.12 |
| Raspberry Pi OS (64 bit) | ARM AArch64 | 3.9 |
| Windows 10/11 | x86-64 | 3.8 ... 3.12 |
| macOS 12 | x86-64 | 3.9 |
| macOS 12 | ARM AArch64 | 3.9 |
Basic Installation of PySDK Python Package
It is recommended to install PySDK in a virtual environment. Please see PySDK Supported Configurations for a list of supported system configurations.
To simplify Python installation and package management, we recommend to use Mniconda. Please follow this link for Miniconda installation instructions.
To install DeGirum PySDK from PyPI.org use the following command:
Install
bash
python3 -m pip install degirum
If python3 command for some reason is not available (typically under Windows Miniconda), try
to either use pip directly, or use python:
Alternative Install
pip install degirum
python -m pip install degirum
To do clean upgrade to the most recent PySDK version use the following command:
Clean upgrade
bash
python3 -m pip uninstall -y degirum && python3 -m pip install degirum
If you use Windows PowerShell, change && to ; (you may also try to use
pip command directly, if python3 command is not available):
Clean upgrade for PowerShell
(pip uninstall -y degirum) ; (pip install degirum)
If during the installation you get the following errors:
Failure
ERROR: Could not find a version that the requirement degirum
ERROR: No matching distribution found for degirum
then try to upgrade pip package by running the following command:
Upgrade pip
bash
python3 -m pip install -U pip
On Windows, install the Visual C++ Redistributable for Visual Studio 2015 and above.
DeGirum Docker Images
For the ease of deployment, DeGirum provides Docker images for the DeGirum AI server and PySDK AI client installations.
Please follow this GitHub link to DeGirum Docker repo for details how to run these images and for corresponding source Dockerfiles.
OpenVINO™ Runtime in PySDK
PySDK supports OpenVINO™ versions 2022.1.1, 2023.2.0, 2023.3.0, and 2024.2.0. Version 2024.2.0 is recommended.
Linux Installation
On Linux, installation via APT Package Manager or via .tgz archive is supported.
NPU Requirement
Currently, NPU requires that OpenVINO™ be installed via .tgz archive.
To install OpenVINO™ via APT Package Manager, click this link or the button below to read installation instructions.
OpenVINO™ Linux Installation via APT
To install OpenVINO™ via .tgz archive, click this link or the button below to read installation instructions.
OpenVINO™ Linux Installation via Archive
Configuring Environment
If you installed OpenVINO™ via .tgz archive, whenever you open a new terminal to use any degirum command, make sure to run
bash
source <OpenVINO Directory>/setupvars.sh
When using PySDK in a python interpreter or a development environment, make sure to start the interpreter or environment from a terminal in which you've run this command.
Windows Installation
On Windows, installation via .zip archive is supported.
To install OpenVINO™ via .zip archive, click this link or the button below to read installation instructions.
OpenVINO™ Windows Installation
Configuring Environment
Whenever you open a new terminal to use any degirum command, make sure to run
<OpenVINO Directory>\setupvars.bat
When using PySDK in a python interpreter or a development environment, make sure to start the interpreter or environment from a terminal in which you've run this command.
Make sure to use basic Command Prompt, and not PowerShell to run this script.
GPU Drivers for OpenVINO™
To install Intel GPU drivers, see the instructions at this link, or the button below.
OpenVINO™ GPU Driver Installation
Note
Currently, PySDK does not support any integrated GPUs.
NPU Drivers for OpenVINO™
The host system must have completed the NPU driver installation process to be able to use the NPU with OpenVINO™. As the drivers are updated at regular intervals, for maximum compatibility, it is recommended to always use the latest version.
The Linux NPU driver installation instructions are available via this link, or the button below.
The Windows NPU driver is available via this link or the first button below. The instructions are at this link or the second button below.
OpenVINO™ Windows NPU Driver OpenVINO™ Windows NPU Driver Instructions
Sometimes, the NPU can be disabled in UEFI by default. Make sure it is enabled. The procedure for enabling the NPU varies based on the UEFI.
ONNX Runtime in PySDK
PySDK supports version 1.19.0 of ONNX Runtime, on Linux, Windows, and Mac.
To install the ONNX runtime, download an 1.19.0 archive for your system here. Do not use the "-training-" archives. Then, extract it into an appropriate directory:
- On Windows, extract the archive to
C:\Program Files,C:\Program Files (x86), orC:\ProgramData. - On Linux, extract the archive to
/usr/local/. - On Mac, extract the archive to
/usr/local/.
Directory Name
Do not rename the onnxruntime-<os>-<architecture>-1.19.0 directory after you extract it.
AMD Ryzen™ AI in PySDK
PySDK supports AMD Ryzen™ AI version 1.2.
To install AMD Ryzen™ AI, click this link or the button below to read installation instructions.
The installation includes: (1) an NPU driver and (2) a "RyzenAI Software MSI installer."
!!! warning "Configuring Environment" 1. Make sure to launch the driver npu_sw_installer.exe from the terminal while in admin mode. 2. Don't forget to append your C:\path\to\miniconda3\Scripts to the Path System environment variable before launching the installer. 3. Restart the terminal (if using VS Code, restart all open windows) after installation.
Run conda activate <ryzen-ai-env-name> to enter the environment created by the installation wizard.
(Optional) Set Ryzen™ AI environment variables
PySDK initializes the defaults for your processor type automatically. For more info, see notes below.
At this point, you should be ready to run inference on models prepared for the Ryzen NPU from the DeGirum AI Hub Model Zoo.
About environment variables
Ryzen™ AI requires certain environment variables to be set depending on the processor configuration:
- Phoenix (PHX): AMD Ryzen™ 7940HS, 7840HS, 7640HS, 7840U, 7640U.
- Hawk (HPT): AMD Ryzen™ 8640U, 8640HS, 8645H, 8840U, 8840HS, 8845H, 8945H.
- Strix (STX): AMD Ryzen™ Ryzen AI 9 HX370, Ryzen AI 9 365.
About your processor
To find your processor configuration, go to Settings -> System -> About.
If you want to manually specify the environment variables, then either in Windows CMD, Powershell, or inside a Python script run the following:
-
If your processor is Phoenix or Hawk (PHX/HPT):
-
CMD:
set XLNX_VART_FIRMWARE=%RYZEN_AI_INSTALLATION_PATH%voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin
set XLNX_TARGET_NAME=AMD_AIE2_Nx4_Overlay
- Powershell:
$env:XLNX_VART_FIRMWARE="$env:RYZEN_AI_INSTALLATION_PATH"+"voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin"
$env:XLNX_TARGET_NAME="AMD_AIE2_Nx4_Overlay"
- Python:
import os
os.environ['XLNX_VART_FIRMWARE'] = os.environ['RYZEN_AI_INSTALLATION_PATH'] + 'voe-4.0-win_amd64/xclbins/phoenix/1x4.xclbin'
os.environ['XLNX_TARGET_NAME'] = "AMD_AIE2_Nx4_Overlay"
- If your processor is Strix (STX):
- CMD:
- Powershell:
- Python:
Changing environment with Python
When setting environment variables with Python, put the code lines before import degirum for PySDK to see the changes.
These variables have to be set every time you open a fresh terminal to run inference. In case you want to set them permanently, go to Edit the system environment varibles in Windows search and add (or edit) the two variables through the GUI.
Incorrect environment variables may lead to a crash
PySDK detects when the value of env:XLNX_VART_FIRMWARE doesn't match the CPU type and automatically sets it to the one appropriate for your device to prevent a system crash. However, be careful when manually redefining the environment variables.
About NPU Configurations:
The instructions above correspond the "Standard Configuration" of the NPU. An additional "Benchmark Configuration" is supported by Ryzen™ AI. The setup process is similar (requires different environment variable values) and is described in detail on the Ryzen™ AI Runtime Setup Page.
Models will be recompiled when run with a different NPU configuration
PySDK detects when an inference on a model is launched in a configuration different from the one it was compiled for (if the cache exists) and recompiles it at runtime to prevent a crash. So expect some initial delay when trying a different NPU configuration.
About supported model format
DeGirum AI Hub Model Zoo supplies INT8 symmetrically quantized models required by the Ryzen™ AI NPU. Models come with precompiled caches to bypass the sometimes lengthy compilation process and enable immediate inference. If recompilation is needed or if cached models cause errors, remove the <model_name>_cache subdirectory from the downloaded model directory. Inference will then proceed with just-in-time compilation automatically.
RKNN (Rockchip) Runtime in PySDK
PySDK supports version 2.0.0, 2.1.0+ of the RKNN runtime on Rockchip systems.
You can check for an existing installation of RKNN Runtime by running the following command:
Example output:
To verify that the Rockchip NPU driver is properly loaded and functioning on your system, you can check the system logs for NPU-related entries. Run the following command:
To install the RKNN Runtime, clone the RKNN repository from this link.
Then, simply copy the librknnrt.so file to the /usr/lib/ directory:
git clone https://github.com/airockchip/rknn-toolkit2.git
cd rknn-toolkit2
sudo cp ./rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so /usr/lib/
Verification
After installing third party runtimes and any needed device drivers, you can verify PySDK access to the runtimes and the devices by using the system information command:
Verification
bash
degirum sys-info
The output should contain all the devices available to PySDK and some information about them. Devices are listed as <runtime>/<device type>, for example, ONNX/CPU for CPU access via ONNX, or OPENVINO/NPU for NPU access via OpenVINO™.
Example of degirum sys-info output
bash
Devices:
<other device types>
<runtime>/<device type>:
- '@Index': 0
<device dependent information, optional>
- '@Index': 1
<device dependent information, optional>
<other devices>
<other device types>