# Inference with local models

*Estimated read time: 3 minutes*

This setup runs inference on local hardware using DeGirum AI Server. The model is stored in a local folder on the machine where the server runs.

You can set the server address via `inference_host_address="localhost"` or `inference_host_address="<host_ip>:<port>"`.

## Client and AI Server on the same host

In this case, both the AI Server and your client application run on the same machine.

First, download the model to a local folder using the `degirum download-zoo` command:

{% code overflow="wrap" %}

```bash
ZOO="$HOME/degirum_model_zoo"
mkdir -p "$ZOO"

degirum download-zoo \
  --path "$ZOO" \
  --url https://hub.degirum.com/degirum/hailo \
  --model_family yolov8n_coco--640x640_quant_hailort_multidevice_1
```

{% endcode %}

Then launch the AI Server with the following command:

{% code overflow="wrap" %}

```bash
degirum server --zoo "$ZOO"
```

{% endcode %}

You should see output like:

{% code overflow="wrap" %}

```bash
DeGirum asio server is started at TCP port 8778
Local model zoo is served from '/home/degirum/degirum_model_zoo' directory.
Press Enter to stop the server
```

{% endcode %}

{% hint style="info" %}
Zoo path may differ.
{% endhint %}

The server runs until you press `ENTER`. By default, it listens on TCP port 8778. To change the port, use the `--port` argument:

{% code overflow="wrap" %}

```
degirum server --port <your_port> --zoo "$ZOO"
```

{% endcode %}

## Example ModelSpec

This example configures `ModelSpec` to use the AI Server and load the model from the local zoo:

<pre class="language-python" data-overflow="wrap"><code class="lang-python"># Example ModelSpec
model_spec = ModelSpec(
    model_name="yolov8n_coco--640x640_quant_hailort_multidevice_1",
    <a data-footnote-ref href="#user-content-fn-1">zoo_url="aiserver://"</a>,
    <a data-footnote-ref href="#user-content-fn-2">inference_host_address="localhost"</a>,
    model_properties={"device_type": ["HAILORT/HAILO8L", "HAILORT/HAILO8"]}
)
</code></pre>

* `zoo_url="aiserver://"`: load the model from the zoo path given when launching the AI Server.
* `inference_host_address="localhost"`: run inference using the Hailo device managed by the local server.

## Client and AI Server on different hosts

To run inference from a separate client machine, first use `degirum download-zoo` to download the model to the host that will run the AI Server:

{% code overflow="wrap" %}

```bash
ZOO="$HOME/degirum_model_zoo"
mkdir -p "$ZOO"

degirum download-zoo \
  --path "$ZOO" \
  --url https://hub.degirum.com/degirum/hailo \
  --model_family yolov8n_coco--640x640_quant_hailort_multidevice_1
```

{% endcode %}

Star the AI Server on the remote host:

{% code overflow="wrap" %}

```bash
degirum server --zoo "$ZOO"
```

{% endcode %}

You should see output like:

{% code overflow="wrap" %}

```bash
DeGirum asio server is started at TCP port 8778
Local model zoo is served from '/home/degirum/degirum_model_zoo' directory.
Press Enter to stop the server
```

{% endcode %}

As before, the server runs until you press `ENTER`. By default, it listens on TCP port 8778. To change the port, use the `--port` argument:

{% code overflow="wrap" %}

```
degirum server --port <your_port> --zoo "$ZOO"
```

{% endcode %}

## Example ModelSpec

This time, the client points to a remote host:

<pre class="language-python" data-overflow="wrap"><code class="lang-python"># Example ModelSpec
model_spec = ModelSpec(
    model_name="yolov8n_coco--640x640_quant_hailort_multidevice_1",
    <a data-footnote-ref href="#user-content-fn-1">zoo_url="aiserver://"</a>,
    <a data-footnote-ref href="#user-content-fn-3">inference_host_address="&#x3C;host_ip>:&#x3C;port>"</a>,
    model_properties={"device_type": ["HAILORT/HAILO8L", "HAILORT/HAILO8"]}
)
</code></pre>

* `zoo_url="aiserver://"`: still points to the AI Server zoo.
* `inference_host_address="<host_ip>:<port>"`: runs inference using the Hailo device on the remote AI Server.

[^1]: Indicates that the model will be loaded from a model zoo path specified when the AI Server is run.

[^2]: Indicates that the model will be run using Hailo devices managed by the AI Server listening on `"localhost:8778"`.

[^3]: Indicates that the model will be run using Hailo devices managed by the AI Server available at `"<host_ip>:<port>"`.