TensorRT

PySDK supports the NVIDIA TensorRT runtime on Linux, Windows, and NVIDIA Jetson hardware. This page walks through installation on each platform.

Supported Versions

PySDK is validated with the following setups:

1. AMD64 host, CUDA GPU compute capability 8.9, CUDA 12.4, TensorRT 10.6.
2. AMD64 host, CUDA GPU compute capability 12.0, CUDA 12.9, TensorRT 10.13.
3. ARM64 host, JetPack 6.2.

CUDA GPU compute capability is determined by the GPU installed in the system. Each level of CUDA GPU compute capability requires a minimum CUDA version (see the "Max CC" column in this compatibility table). In practice, RTX 5000 series GPUs require CUDA versions greater than 12.6 and a matching TensorRT version.

Linux Installation

CUDA

We recommend the local Debian Installer method.

1

Make sure you have a compatible GPU and that its drivers are installed.

2

Download CUDA from the NVIDIA CUDA Toolkit archive.

3

Follow NVIDIA's CUDA installation guide for Ubuntu.

4

Update PATH and LD_LIBRARY_PATH, replacing <VERSION> with the version you installed:

export PATH=/usr/local/cuda-<VERSION>/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-<VERSION>/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

5

Verify the installation:

nvcc --version

As an alternative, the default system repository often includes a supported version of CUDA.

TensorRT

We recommend the local Debian Installer method.

1

Follow NVIDIA's TensorRT download and installation instructions.

2

Install the full tensorrt package. cuDNN is not required.

3

The .deb installer handles LD_LIBRARY_PATH automatically.

Windows Installation

CUDA

Follow NVIDIA's CUDA installation guide for Windows. Python wheels are not required.

TensorRT

1

Follow steps 1–4 of NVIDIA's TensorRT zip-file installation guide.

2

On Windows, TensorRT does not update PATH automatically. Make sure you complete step 4 from the guide above.

JetPack Installation

CUDA and TensorRT are included in JetPack. PySDK supports JetPack 6.2. See NVIDIA's JetPack installation guide for details.