Setting Up an AI Server
Read this page if you'll host an AI server or perform inference with a local server.
Starting an AI Server
Use PySDK to configure and launch an AI server. The server runs on your host and processes inference requests from remote clients.
You can start the AI server in several ways:
running AI server from a terminal directly on the host OS.
running AI server as a Linux system service.
running AI server with a Docker container.
Terminal
To run the PySDK AI server from a terminal, perform the following steps:
Install PySDK
Any device with PySDK installed can run an AI server. Refer to PySDK Installation for details on how to install PySDK.
Create a directory for the local model zoo
You'll need to create a directory to hold your models.
mkdir ~/degirum-model-zoo
Download or copy models to the local model zoo
After creating the model zoo directory, you'll need to populate it with models
To easily populate the model zoo, you can use the PySDK CLI tool for downloading models from the DeGirum AI Hub.
cd ~/degirum-model-zoo
degirum download-zoo
Start the AI server
Launch the server with the following command:
cd ~/degirum-model-zoo
degirum server
The server runs until you press ENTER
in the terminal. By default, it listens on TCP port 8778. To specify a different port, use the --port
argument:
degirum server --port <your_port>
Linux Service
To automatically start the server on boot, configure it as a Linux service:
Complete the terminal setup steps
Follow all steps in Starting AI Server from Terminal except for launching the server.
Create a systemd service configuration file
Create a file named degirum.service
in the /etc/systemd/system
directory. Use the following template:
[Unit]
Description=DeGirum AI Service
[Service]
WorkingDirectory=/home/<your_username>/
ExecStart=<path_to_python> -m degirum.server --zoo /home/<your_username>/zoo
Restart=always
RestartSec=10
SyslogIdentifier=degirum-ai-server
User=<your_username>
[Install]
WantedBy=multi-user.target
Start the service
Start the service using systemctl
:
sudo systemctl start degirum.service
Check the service status
Check the service status using systemctl
:
sudo systemctl status degirum.service
Enable the service on startup
Use systemctl
to automatically enable the degirum service on startup:
sudo systemctl enable degirum.service
Docker Container
To run the AI server as a Docker container, follow these steps:
Ensure Docker is installed
Refer to the official Docker documentation for installation instructions
Prepare the local model zoo
If hosting models locally, create and populate a model zoo directory:
mkdir -p ~/degirum-model-zoo
cd ~/degirum-model-zoo
degirum download-zoo
Run the Docker container
When hosting models locally:
docker run --name aiserver -d -p 8778:8778 -v ~/degirum-model-zoo:/zoo --privileged degirum/aiserver:latest
When serving models only from AI Hub:
docker run --name aiserver -d -p 8778:8778 --privileged degirum/aiserver:latest
Rescanning Model Zoos
If you started your AI server in a terminal or as a Linux service, you can tell the AI server to rescan the local model zoo directory by executing the following command on the same host: degirum server rescan-zoo
If you started your AI server in the Docker container, then you should rescan the model zoo directory by restarting the container: docker restart aiserver
Last updated
Was this helpful?