# TensorRT

## Supported Versions

PySDK is validated with the following setups:

1. AMD64 host, CUDA GPU compute capability 8.9, CUDA 12.4, TensorRT 10.6.
2. AMD64 host, CUDA GPU compute capability 12.0, CUDA 12.9, TensorRT 10.13.
3. ARM64 host, JetPack 6.2.

{% hint style="info" %}
CUDA GPU compute capability is determined by the [GPU installed in the system](https://developer.nvidia.com/cuda-gpus). Each level of CUDA GPU compute capability requires a minimum CUDA version (see the "Max CC" column in [this compatibility table](https://stackoverflow.com/questions/28932864/which-compute-capability-is-supported-by-which-cuda-versions/28933055#28933055)). In practice, RTX 5000 series GPUs require CUDA versions greater than 12.6 and a matching TensorRT version.
{% endhint %}

## Linux Installation

### CUDA

We recommend the local Debian Installer method.

{% stepper %}
{% step %}
Make sure you have a compatible GPU and that its drivers are installed.
{% endstep %}

{% step %}
Download CUDA from the [NVIDIA CUDA Toolkit archive](https://developer.nvidia.com/cuda-toolkit-archive).
{% endstep %}

{% step %}
Follow NVIDIA's [CUDA installation guide for Ubuntu](https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html#ubuntu).
{% endstep %}

{% step %}
Update `PATH` and `LD_LIBRARY_PATH`, replacing `<VERSION>` with the version you installed:

{% code overflow="wrap" %}

```bash
export PATH=/usr/local/cuda-<VERSION>/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-<VERSION>/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
```

{% endcode %}
{% endstep %}

{% step %}
Verify the installation:

{% code overflow="wrap" %}

```bash
nvcc --version
```

{% endcode %}
{% endstep %}
{% endstepper %}

As an alternative, the default system repository often includes a supported version of CUDA.

### TensorRT

We recommend the local Debian Installer method.

{% stepper %}
{% step %}
Follow NVIDIA's [TensorRT download and installation instructions](https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html#downloading-tensorrt).
{% endstep %}

{% step %}
Install the full `tensorrt` package. cuDNN is not required.
{% endstep %}

{% step %}
The `.deb` installer handles `LD_LIBRARY_PATH` automatically.
{% endstep %}
{% endstepper %}

## Windows Installation

### CUDA

Follow NVIDIA's [CUDA installation guide for Windows](https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html#windows). Python wheels are not required.

### TensorRT

{% stepper %}
{% step %}
Follow steps 1–4 of NVIDIA's [TensorRT zip-file installation guide](https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html#zip-file-installation).
{% endstep %}

{% step %}
On Windows, TensorRT does not update `PATH` automatically. Make sure you complete step 4 from the guide above.
{% endstep %}
{% endstepper %}

## JetPack Installation

CUDA and TensorRT are included in JetPack. PySDK supports JetPack 6.2. See NVIDIA's [JetPack installation guide](https://docs.nvidia.com/sdk-manager/install-with-sdkm-jetson/index.html) for details.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.degirum.com/pysdk/runtimes-and-drivers/tensorrt.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.