# Loading an AI Model

## Connect to an Inference Engine and Model Zoo

The `degirum.connect()` function is the starting point for interacting with PySDK. It establishes a connection with the appropriate AI inference engine and model zoo based on the configuration you provide.

{% code overflow="wrap" %}

```python
import degirum

degirum.connect(
    inference_host_address = "@local",
    zoo_url = "workspace/zoo",
    token = "<your_token>"
)
```

{% endcode %}

When you call `degirum.connect()`, you will:

* Specify an inference host to run AI models
* Specify a model zoo from which AI models can be loaded
* Authenticate with the AI Hub using a token

`degirum.connect()` creates and returns a `ZooManager` object. This object enables:

* Searching for models available in the connected model zoo.
* Loading AI models and creating appropriate AI model handling objects for inference.
* Accessing model parameters to customize inference behavior.

### degirum.connect()

`degirum.connect()` takes three parameters: `inference_host_address`, `zoo_url`, and `token`. These parameters define where the inference will be run, what zoo will be used, and the token used for AI Hub authentication if an AI Hub model zoo will be used.

#### Zoo URL Parsing Behavior

The `zoo_url` parameter is handled the following ways:

* **AI Hub Inference** (`inference_host_address="@cloud"`)
  * `zoo_url` may be given as `https://hub.degirum.com/workspace/zoo` or simply `workspace/zoo`.
* **Local Inference** (`inference_host_address="@local"`)
  * When `zoo_url` starts with `http://` or `https://` or contains exactly one slash (e.g. `workspace/zoo`), it is treated as an AI Hub zoo.
  * If the `zoo_url` is instead formatted differently, it will be handled like a local path. Prefix the path with `file://` to be explicit that it is a local path. If the path does not exist, `degirum.connect()` raises `"incorrect local model zoo URL: path does not exist"`. The path can point to either a directory or a model .json file.
* **AI Server Inference** (hostname or `host:port`)
  * An empty `zoo_url` or a value starting with `aiserver://` selects the AI server's local zoo.
  * Any other value must be a valid AI Hub zoo URL; otherwise, the error `"incorrect cloud model zoo URL"` is raised.

{% hint style="info" %}
The `zoo_url` parameter is handled differentely for PySDK versions including and prior to 0.16.2. If you need help configuring the `zoo_url` parameter, contact the DeGirum team.
{% endhint %}

{% code overflow="wrap" %}

```python
# Function Signature: degirum.connect() 
degirum.connect(inference_host_address, zoo_url=None, token=None)
```

{% endcode %}

The following table lists all possible combinations of inference and zoo types with `degirum.connect():`

<table><thead><tr><th width="128">Inference Type</th><th width="200">Model Zoo Type</th><th>Usage Example</th></tr></thead><tbody><tr><td>AI Hub</td><td>Specified AI Hub Model Zoo</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="@cloud",
    zoo_url="workspace/zoo",
    token="&#x3C;your_token>"
)
</code></pre></td></tr><tr><td>AI Server</td><td>Default: AI Server Local Zoo</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="host:port"
)
</code></pre></td></tr><tr><td>AI Server</td><td>Specified AI Hub Model Zoo</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="host:port",
    zoo_url="workspace/zoo",
    token="&#x3C;your_token>"
)
</code></pre></td></tr><tr><td>AI Server</td><td>AI Server Local Zoo</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="host:port",
    zoo_url="aiserver://"
)
</code></pre></td></tr><tr><td>Local</td><td>Default: Current Directory</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="@local"
)
</code></pre></td></tr><tr><td>Local</td><td>Specified AI Hub Model Zoo</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="@local",
    zoo_url="workspace/zoo",
    token="&#x3C;your_token>"
)
</code></pre></td></tr><tr><td>Local</td><td>Local Folder</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="@local",
    zoo_url="file://path/to/zoo"
)
</code></pre></td></tr><tr><td>Local</td><td>Local File</td><td><pre class="language-python"><code class="lang-python">degirum.connect(
    inference_host_address="@local",
    zoo_url="file://path/to/model.json"
)
</code></pre></td></tr></tbody></table>

## Retrieve Supported Devices, Filter Models, then Load Models

### ZooManager.supported\_device\_types()

The `ZooManager.supported_device_types()` method returns a list of runtime and device combinations (in `"RUNTIME/DEVICE"` format) that the connected inference engine supports.

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Connect to DeGirum Application Server and an AI Hub model zoo
inference_manager = dg.connect(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)
supported_types = inference_manager.supported_device_types()
print(supported_types)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
['N2X/ORCA1', 'TFLITE/EDGETPU', 'OPENVINO/CPU']
```

{% endcode %}

In this example, the inference engine returns a list of supported device types on the application server. In thise case, it's the application server hosted on `@cloud`.

### **ZooManager.list\_models()**

After obtaining a `ZooManager` object, you can use the `ZooManager.list_models()` method to retrieve and filter the list of available AI models.

{% code overflow="wrap" %}

```python
# Method Signature: ZooManager.list_models()
degirum.zoo_manager.ZooManager.list_models(*args, **kwargs)
```

{% endcode %}

This method:

* Filters models based on various criteria such as model family, runtime, device type, precision, postprocessor type, and more.
* Returns a list of model names that can be used later when loading models for inference.

#### **Use Cases**

* Exploring available model families (e.g., mobilenet, YOLO).
* Filtering models based on target hardware.
* Selecting models for specific precision or density.

#### Example Usage

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Connect to DeGirum Application Server and an AI Hub model zoo
inference_manager = dg.connect(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)
model_list = inference_manager.list_models(device_type=["OPENVINO/CPU"])
print(model_list)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
['clip_RN50--224x224_quant_openvino_cpu_3', 'clip_rn50_image_encoder--224x224_float_openvino_cpu_1', 'clip_rn50_text_encoder--1x77_float_openvino_cpu_1']
```

{% endcode %}

In this example, the inference engine returns a list of models that run using the OpenVINO runtime on a CPU.

#### **Available Filtering Parameters and Their Sources**

The `ZooManager.list_models()` method filters models based on information retrieved from the model name string and also the model JSON fields.

For models named based on our recommended [model naming conventions](/pysdk/user-guide-pysdk/organizing-models.md#json-model-file-naming-conventions), the model name string store the `model_family`, `precision`, and `pruned` parameters.

The [model JSON file](/pysdk/user-guide-pysdk/model-json-structure.md) specifies the `RuntimeAgent`, `DeviceType`, and `SupportedDeviceTypes` fields.

<table><thead><tr><th width="183">Parameter</th><th width="256">Possible Values</th><th>Source of Information</th></tr></thead><tbody><tr><td><code>model_family</code></td><td>Any valid substring like <code>"yolo"</code>, <code>"mobilenet"</code></td><td>Extracted from the model name</td></tr><tr><td><code>precision</code></td><td><code>"quant"</code> (quantized model), <code>"float"</code> (floating-point model)</td><td>Inferred from the presence of precision-related fields in the model name</td></tr><tr><td><code>pruned</code></td><td><code>"dense"</code> (dense model), <code>"pruned"</code> (sparse/pruned model)</td><td>Determined from suffixes indicating density in the model name (e.g., <code>"pruned"</code> or <code>"dense"</code>)</td></tr><tr><td><code>runtime</code></td><td>See<a href="/pages/FtxwmI6NxRjdB6bqlc1Z"> </a><a href="/pages/Kcs9VG2k5sSJrh9ZRL4A#supported-hardware">Supported Hardware</a> for the full list.</td><td>Combines information from <code>"RuntimeAgent"</code> and <code>"SupportedDeviceTypes"</code></td></tr><tr><td><strong><code>device</code></strong></td><td>See<a href="/pages/FtxwmI6NxRjdB6bqlc1Z"> </a><a href="/pages/Kcs9VG2k5sSJrh9ZRL4A#supported-hardware">Supported Hardware</a> for the full list.</td><td>Combines information from <code>"DeviceType"</code> and <code>"SupportedDeviceTypes"</code></td></tr><tr><td><code>device_type</code></td><td>See<a href="/pages/FtxwmI6NxRjdB6bqlc1Z"> </a><a href="/pages/Kcs9VG2k5sSJrh9ZRL4A#supported-hardware">Supported Hardware</a> for the full list.</td><td>Extracted from the <code>"SupportedDeviceTypes"</code></td></tr></tbody></table>

#### **Combine list\_models() with supported\_device\_types() to Find Supported Models**

To find only the models that are compatible with the current inference engine, you can use the `supported_device_types()`method as a filter for `list_models()`.

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Connect to DeGirum Application Server and an AI Hub model zoo
inference_manager = dg.connect(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)

# Retrieve supported device types
supported_types = inference_manager.supported_device_types()

# List models that match any supported runtime/device combination
supported_models = inference_manager.list_models(device_type=list(supported_types))

print("Models supported by the current inference engine:")
for model in supported_models:
   print(model)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
Models supported by the current inference engine:
clip--224x224_float_tensorrt_gpu_1
clip_RN50--224x224_float_n2x_orca1_2
clip_RN50--224x224_float_tensorrt_gpu_1
```

{% endcode %}

### ZooManager.load\_model()

Once you have obtained supported models from filtering `list_models() with supported_device_types()` methods, you can load all of the resulting models for inference using the `degirum.zoo_manager.ZooManager.load_model()` method.

#### **Basic Usage**

To load a model, pass the model name string as the `model_name` argument to `load_model()`.

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Connect to DeGirum Application Server and an AI Hub model zoo
inference_manager = dg.connect(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)

# Retrieve supported device types
supported_types = inference_manager.supported_device_types()

# List models that match any supported runtime/device combination
supported_models = inference_manager.list_models(device_type=list(supported_types))

# Load the first model from the list
model = inference_manager.load_model(model_name=supported_models[0])

# Print the model object
print(model)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
<degirum.model._CloudServerModel object at 0x000001FA43E2CF80>
```

{% endcode %}

If a model with the specified name is found, the method returns a `degirum.model.Model` object that you can use to run inference.

If the model is not found, an exception will be raised.

#### **Passing Model Properties as Arguments**

You can pass additional model properties as keyword arguments to customize the behavior of the loaded model. These properties are directly assigned to the model object.

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Connect to DeGirum Application Server and an AI Hub model zoo
inference_manager = dg.connect(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)

# Retrieve supported device types
supported_types = inference_manager.supported_device_types()

# List models that match any supported runtime/device combination
supported_models = inference_manager.list_models(device_type=list(supported_types))

# Load the first model from the list
model = inference_manager.load_model(
    model_name=supported_models[0],
    output_confidence_threshold=0.5, 
    input_pad_method="letterbox"
)
print(model)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
<degirum.model._CloudServerModel object at 0x0000026CAE6FD070>
```

{% endcode %}

In this example:

* `output_confidence_threshold=0.5` sets a confidence threshold for inference results.
* `input_pad_method="letterbox"` specifies the padding method to maintain the input aspect ratio.

## Convenience Functions

### **degirum.get\_supported\_devices()**

You can retrieve supported device types using the `degirum.get_supported_devices()` function. This function combines the arguments of both `degirum.connect()` and `degirum.zoo_manager.ZooManager.get_supported_devices()`, allowing you to list supported devices with a single call.

#### Function Signature:

{% code overflow="wrap" %}

```python
degirum.get_supported_devices(inference_host_address, zoo_url='', token='')
```

{% endcode %}

#### Example:

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Retrieve supported device types
supported_devices = dg.get_supported_devices(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token
)
print(supported_devices)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
dict_keys(['AKIDA/NSOC_V2', 'AXELERA/METIS', 'DEEPX/M1A', 'DUMMY/DUMMY', 'HAILORT/HAILO8', 'HAILORT/HAILO8L', 'MEMRYX/MX3', 'N2X/CPU', 'N2X/ORCA1', 'ONNX/CPU', 'OPENVINO/CPU', 'OPENVINO/GPU', 'OPENVINO/NPU', 'RKNN/RK3566', 'RKNN/RK3568', 'RKNN/RK3588', 'TENSORRT/DLA', 'TENSORRT/GPU', 'TFLITE/CPU', 'TFLITE/EDGETPU'])
```

{% endcode %}

### **degirum.list\_models()**

You can retrieve the list of models using the `degirum.list_models()` function. This function combines the arguments of both `degirum.connect()` and `degirum.zoo_manager.ZooManager.list_models()`, allowing you list models with a single call.

#### Function Signature:

{% code overflow="wrap" %}

```python
degirum.list_models(inference_host_address, zoo_url, token=None, **kwargs)
```

{% endcode %}

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# List models from the AI Hub model zoo with specific filtering criteria
model_list = dg.list_models(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token, 
    device_type=["OPENVINO/CPU"]
)
print(model_list)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
{'clip_RN50--224x224_quant_openvino_cpu_3': <degirum.aiclient.ModelParams object at 0x0000015B74417DB0>, 'clip_rn50_image_encoder--224x224_float_openvino_cpu_1': <degirum.aiclient.ModelParams object at 0x0000015B7FEEC430>}
```

{% endcode %}

In this example, the method connects to the specified model zoo and returns a list of models that run with the OpenVINO runtime on a CPU.

### **degirum.load\_model()**

For convenience, you can directly load a model without explicitly obtaining a `ZooManager` object with `degirum.connect()`. The `degirum.load_model()` function combines the arguments of `degirum.connect()` and `ZooManager.load_model()`, allowing you to load models with a single call.

**Function Signature:**

{% code overflow="wrap" %}

```python
degirum.load_model(model_name, inference_host_address, zoo_url=None, token=None, **kwargs)
```

{% endcode %}

**Example:**

{% code overflow="wrap" %}

```python
import degirum as dg

# Set your inference host address, model zoo, and token in these variables.
your_host_address = "@cloud" # Can be "@cloud", host:port, or "@local"
your_model_zoo = "degirum/public"
your_token = "<your_token>"

# Retrieve supported device types
supported_devices = dg.get_supported_devices(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token, 
)

# List models from the AI Hub model zoo with specific filtering criteria
model_list = dg.list_models(
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token,
    device_type=list(supported_devices)
)

# Load the first model from the list with some optional parameters
model = dg.load_model(
    model_name = list(model_list)[0], 
    inference_host_address = your_host_address, 
    zoo_url = your_model_zoo, 
    token = your_token,
    overlay_show_probabilities = True,
    output_confidence_threshold = 0.5
)
print(model)
```

{% endcode %}

Example output:

{% code overflow="wrap" %}

```
<degirum.model._CloudServerModel object at 0x000002C079FEE270>
```

{% endcode %}

In this example, the code gets supported devices, lists models, then loads a model based on the filtered list with a specified parameter.

#### Minimum Code Example

Once you understand the steps above, you can streamline the code into a few lines.

{% code overflow="wrap" %}

```python
import degirum as dg

# Loading a model
model = dg.load_model(
    model_name = "yolov8n_relu6_coco--640x640_quant_rknn_rk3588_1",
    inference_host_address = "@cloud",
    zoo_url = "degirum/public",
    token = your_token 
    # optional parameters, such as overlay_show_probabilities = True
)
print(model)
```

{% endcode %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.degirum.com/pysdk/user-guide-pysdk/loading-an-ai-model.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.