LogoLogo
AI HubCommunityWebsite
  • Start Here
  • AI Hub
    • Overview
    • Quickstart
    • Spaces
    • Device Farm
    • Browser Inference
    • Model Zoo
      • Hailo
      • Intel
      • MemryX
      • BrainChip
      • Google
      • DeGirum
      • Rockchip
    • View and Create Model Zoos
    • Compiler
    • PySDK Integration
  • PySDK
    • Overview
    • Quickstart
    • Installation
    • Runtimes and Drivers
      • Hailo
      • OpenVINO
      • MemryX
      • BrainChip
      • Rockchip
      • ONNX
    • PySDK User Guide
      • Core Concepts
      • Organizing Models
      • Setting Up an AI Server
      • Loading an AI Model
      • Running AI Model Inference
      • Model JSON Structure
      • Command Line Interface
      • API Reference Guide
        • PySDK Package
        • Model Module
        • Zoo Manager Module
        • Postprocessor Module
        • AI Server Module
        • Miscellaneous Modules
      • Older PySDK User Guides
        • PySDK 0.16.1
        • PySDK 0.16.0
        • PySDK 0.15.2
        • PySDK 0.15.1
        • PySDK 0.15.0
        • PySDK 0.14.3
        • PySDK 0.14.2
        • PySDK 0.14.1
        • PySDK 0.14.0
        • PySDK 0.13.4
        • PySDK 0.13.3
        • PySDK 0.13.2
        • PySDK 0.13.1
        • PySDK 0.13.0
    • Release Notes
      • Retired Versions
    • EULA
  • DeGirum Tools
    • Overview
      • Streams
        • Streams Base
        • Streams Gizmos
      • Compound Models
      • Analyzers
        • Clip Saver
        • Event Detector
        • Line Count
        • Notifier
        • Object Selector
        • Object Tracker
        • Zone Count
      • Inference Support
      • Support Modules
        • Audio Support
        • Model Evaluation Support
        • Math Support
        • Object Storage Support
        • UI Support
        • Video Support
      • Environment Variables
  • DeGirumJS
    • Overview
    • Get Started
    • Understanding Results
    • Release Notes
    • API Reference Guides
      • DeGirumJS 0.1.3
      • DeGirumJS 0.1.2
      • DeGirumJS 0.1.1
      • DeGirumJS 0.1.0
      • DeGirumJS 0.0.9
      • DeGirumJS 0.0.8
      • DeGirumJS 0.0.7
      • DeGirumJS 0.0.6
      • DeGirumJS 0.0.5
      • DeGirumJS 0.0.4
      • DeGirumJS 0.0.3
      • DeGirumJS 0.0.2
      • DeGirumJS 0.0.1
  • Orca
    • Overview
    • Benchmarks
    • Unboxing and Installation
    • M.2 Setup
    • USB Setup
    • Thermal Management
    • Tools
  • Resources
    • External Links
Powered by GitBook

Get Started

  • AI Hub Quickstart
  • PySDK Quickstart
  • PySDK in Colab

Resources

  • AI Hub
  • Community
  • DeGirum Website

Social

  • LinkedIn
  • YouTube

Legal

  • PySDK EULA
  • Terms of Service
  • Privacy Policy

© 2025 DeGirum Corp.

On this page
  • Starting an AI Server
  • Terminal
  • Linux Service
  • Docker Container
  • Rescanning Model Zoos

Was this helpful?

  1. PySDK
  2. PySDK User Guide

Setting Up an AI Server

Read this page if you'll host an AI server or perform inference with a local server.

PreviousOrganizing ModelsNextLoading an AI Model

Last updated 1 day ago

Was this helpful?

Starting an AI Server

Use PySDK to configure and launch an AI server. The server runs on your host and processes inference requests from remote clients.

You can start the AI server in several ways:

  • running AI server from a terminal directly on the host OS.

  • running AI server as a Linux system service.

  • running AI server with a Docker container.

Terminal

To run the PySDK AI server from a terminal, perform the following steps:

1

Create or select a user with administrative rights

Choose a user with administrative rights on the host.

2

Set up a Python virtual environment

For convenience and future maintenance, install PySDK in a Python virtual environment. Ensure Python and PySDK are installed in the virtual environment.

3

Create a directory for the local model zoo

Create a directory for hosting a local model zoo.

mkdir /home/<your_username>/zoo
4

Download models to the local model zoo

Download the models from the DeGirum AI Hub Model Zoo to the directory created earlier:

degirum download-zoo --path /home/<your_username>/zoo --token "token string" [--url "model_zoo_url"]
  • "token string": Your AI Hub access token from the .

  • Optional model_zoo_url: The URL for the model zoo in the format "https://hub.degirum.com/<organization>/<zoo>". If omitted, the public model zoo will be used.

5

Start the AI server

Launch the server with the following command:\

degirum server --zoo /home/<your_username>/zoo

The server runs until you press ENTER in the terminal. By default, it listens on TCP port 8778. To specify a different port, use the --port argument:

degirum server --zoo /home/<your_username>/zoo --port <your_port>

Linux Service

To automatically start the server on boot, configure it as a Linux service:

1

Complete the terminal setup steps

Follow all steps in except for launching the server.

2

Create a systemd service configuration file

Create a file named degirum.service in the /etc/systemd/system directory. Use the following template:

[Unit]
Description=DeGirum AI Service

[Service]
WorkingDirectory=/home/<your_username>/
ExecStart=<path_to_python> -m degirum.server --zoo /home/<your_username>/zoo
Restart=always
RestartSec=10
SyslogIdentifier=degirum-ai-server
User=<your_username>

[Install]
WantedBy=multi-user.target
3

Start the service

Start the service using systemctl:

sudo systemctl start degirum.service
4

Check the service status

Check the service status using systemctl:

sudo systemctl status degirum.service
5

Enable the service on startup

Use systemctl to automatically enable the degirum service on startup:

sudo systemctl enable degirum.service

Docker Container

To run the AI server as a Docker container, follow these steps:

1

Ensure Docker is installed

2

Prepare the local model zoo

3

Run the Docker container

When hosting models locally:

docker run --name <your_server_name> -d -p 8778:8778 -v <your local model zoo>:/zoo --privileged degirum/<your_server_name>:<version>
   

When serving models only from AI Hub:

docker run --name <your_server_name> -d -p 8778:8778 --privileged degirum/<your_server_name>:<version>

Rescanning Model Zoos

If you started your AI server in a terminal or as a Linux service, you can tell the AI server to rescan the local model zoo directory by executing the following command on the same host: degirum server rescan-zoo

If you started your AI server in the Docker container, then you should rescan the model zoo directory by restarting the container: docker restart <your_server_name>

Refer to the for installation instructions

If hosting models locally, create and populate the model zoo directory as described in .

official Docker documentation
DeGirum AI Hub
Starting AI Server from Terminal
Starting AI Server from Terminal