Quickstart

Quick Start Guide: Hosted Inference

This quick start guide demonstrates the Hosted Inference use case, where AI inferences are performed on the DeGirum AI Hub using DeGirum AI accelerator hardware deployed in the DeGirum Device Farm.

Basic Inference Example

import degirum as dg
model = zoo.load_model(
    model_name="mobilenet_v2_ssd_coco--300x300_quant_n2x_orca_1",
    inference_host_address="@cloud",
    zoo_url="degirum/public",
    token="<your ai hub token>",
    image_backend='pil'
)
result = model("https://docs.degirum.com/images/samples/TwoCats.jpg")
display(result.image_overlay)

Step-by-Step Instructions

1. Import the PySDK Package

   • Start by importing the degirum package.

2. Load an AI Model

   • Use the zoo.load_model method to load an AI model. The required parameters include:
     • model_name: The name of the model you wish to load (e.g., mobilenet_v2_ssd_coco--300x300_quant_n2x_orca_1).
     • inference_host_address: The host address, set to "@cloud" for AI Hub.
     • zoo_url: The URL of the model zoo (e.g., degirum/public).
     • token: Your AI Hub token for authentication.

3. Perform Inference

   • Call the loaded model with an input image. The input image can be provided in the following formats:

     Local FileURLPIL Image

     result = model("./images/TwoCats.jpg")
     

     result = model("https://docs.degirum.com/images/samples/TwoCats.jpg")
     

     from PIL import Image
     image = Image.open("./images/TwoCats.jpg")
     result = model(image)
     

4. Work with Results

   • The result object provides:
     • Numeric Inference Results
     • Graphical Inference Results
     • Original Image

   • Access the graphical results using the image_overlay property and display or save them as needed:

     result_image = result.image_overlay
     result_image.save("./images/TwoCatsResults.jpg")
     result_image.show()
     

Exploring Model Zoo and Running Examples

Listing Available Models

If you want to explore the available models in the model zoo, connect explicitly to the zoo and call list_models:

zoo = dg.connect(
    inference_host_address="@cloud",
    zoo_url="degirum/public",
    token="<your ai hub token>"
)
model_list = zoo.list_models()
print(model_list)

Running PySDK Examples

DeGirum maintains the PySDKExamples repository, which contains Jupyter notebooks illustrating how to build edge AI applications using PySDK. These examples demonstrate ML inferences on the following hosting options:

1. DeGirum AI Hub: Perform inferences via the hosted AI Hub.
2. AI Server: Use the DeGirum AI Server running on localhost or a machine within your LAN or VPN.
3. Local ORCA Accelerator: Run inferences directly on a DeGirum ORCA accelerator installed on your local machine.

To explore these options, simply uncomment the relevant line in the code cell labeled "Specify where do you want to run your inferences".

Go to PySDKExamples Repo