Skip to content

AI Server Module

DeGirum AI server launcher and model downloader.

Note

The functionality of this module is now exposed via PySDK CLI.

The purpose of this module is to start DeGirum AI server: 

python -m degirum.server --zoo <local zoo path> [--port <server TCP port>]

Keyword Args:

--zoo (str): Path to a local model zoo directory, containing AI models you want your AI server to serve.

    - One possible way to fill local model zoo directory is to download models from a model zoo repo using
    `download_models()` function and provide the path to the local zoo directory as `--zoo` parameter.

--port (int): TCP port to bind server to.

    - Default is 8778.
  • When AI server is started, it runs indefinitely.
  • If you started the server from a terminal, you may press Enter to shut it down.
  • If you started the server headless (for example, as a service), then to shut down the server you need to kill the Python process which runs the server.

The module also exposes download_models() function which can be used to prepare local model zoo directory to be served by AI server:

  • You first download models from the model zoo repo of your choice into some local directory of your choice by calling degirum.server.download_models function.
  • Then you start the AI server providing the path to that local directory as --zoo parameter.

degirum.server.download_models(path, *, url=ZooManager._default_cloud_url, token='', **kwargs)

Download all models from a model zoo repo specified by the URL.

Args:

path: Local filesystem path to store models downloaded from a model zoo repo.

url: Zoo repo URL.

    - If not specified, then DeGirum public model zoo URL will be used.

token: Zoo repo authorization token.

Keyword Args:

model_family (str): Model family name filter.

    - When you pass a string, it will be used as search substring in the model name. For example, `"yolo"`, `"mobilenet"`.
    - You may also pass `re.Pattern`. In this case it will do regular expression pattern search.

device (str): Target inference device -- string or list of strings of device names.

    - If passed, only models targeting the specified device(s) will be downloaded.

precision (str): Model calculation precision -- string or list of strings of model precision labels.

    - Possible labels: `"quant"`, `"float"`.

pruned (str): Model density -- string or list of strings of model density labels.

    - Possible labels: `"dense"`, `"pruned"`.

runtime (str): Runtime agent type -- string or list of strings of runtime agent types.

    - Possible types: `"n2x"`, `"tflite"`, `"tensorrt"`, `"openvino"`.