Skip to content

Configuring and Launching AI Server

PySDK can be used to configure and launch DeGirum AI server on hosts equipped with DeGirum Orca AI accelerator card(s). This allows you to run AI inferences on this AI server host initiated from remote clients.

You can start AI server process multiple ways:

  • running AI server from the shell directly on the host OS;
  • running AI server as Linux System Service;
  • running AI server inside the Docker container.

Please refer to AI Server Configuration Guide for detailed instructions.

Connecting to AI Server from Client Side

Now your AI server is up and running and you may connect to it from Python scripts using PySDK.

If you want to work with models from local AI server model zoo, you pass the AI server network hostname or its IP address to the degirum.connect PySDK function and do not specify other parameters:

import degirum as dg
model_zoo = dg.connect(host_address)

If you run your PySDK script on the same host as the AI server, you may use the "localhost" string as a network hostname.

In local Linux networks with standard mDNS configuration the network hostname is a concatenation of the local hostname as returned by hostname command and .local suffix, for example, if hostname command returns ai-host, then the network hostname will be ai-host.local

If your server listens to a TCP port other than default 8778, you append the TCP port number to the hostname separated by colon, for example, "localhost:9999".

If you want to work with models from some cloud model zoo of your choice, you pass a cloud zoo URL and cloud API access token as the second and the third parameter. The cloud model zoo URL is specified in the "https://cs.degirum.com/<zoo URL>" format (see more on that in Configuration for Cloud Inference section):

import degirum as dg
model_zoo = dg.connect( host_address, "https://cs.degirum.com/<zoo URL>", token = "<your cloud API access token>")

Updating AI Server Local Model Zoo

If you need to update AI Server local model zoo you need to perform the following steps:

  • Manage your model zoo directory:

  • Add new models by downloading them from the cloud zoo the way described in the beginning of this chapter.

  • Remove models by deleting model subdirectories.

  • If you started your AI server not in Docker container, then tell the AI server to rescan the local model zoo directory by executing the following command on the same host, where the AI server runs:

degirum server rescan-zoo
  • If you started your AI server in the Docker container, then restart the container by executing the following command (assuming that the container was started with --name aiserver option):
docker restart aiserver