Device selection

Choose the device type (runtime + hardware) your model runs on. This guide covers supported types, discovery, deterministic vs. fallback selection, and pinning specific cards.

Estimated read time: 3 minutes

Pick where your model runs by selecting a device type (runtime + hardware) and optionally a device index. This guide focuses on device selection. For performance tuning (e.g., latency, throughput, etc.), see the advanced guides.

Model JSON info

Each model package defines a default and supported device list in its JSON file:

"DEVICE": [
  {
    "DeviceType": "HAILO8L",
    "RuntimeAgent": "HAILORT",
    "SupportedDeviceTypes": "HAILORT/HAILO8L, HAILORT/HAILO8"
  }
]

Interpretation:

• Default: HAILO8L and HAILORT — derived from DeviceType and RuntimeAgent
• Supported list: HAILORT/HAILO8L, HAILORT/HAILO8.

At runtime, this shows up as model.supported_device_types (read‑only).

Device discovery

1

Load the model

from degirum_tools import ModelSpec
model_spec = ModelSpec(
    model_name="…",
    zoo_url="degirum/hailo",
    inference_host_address="@local",
)
model = model_spec.load_model()

2

Check supported device types

print(
    "supported:", model.supported_device_types
)  # e.g., ['HAILORT/HAILO8L', 'HAILORT/HAILO8']

3

Check what's available on this machine

• CLI: degirum sys-info → lists discovered hardware per runtime
• Or, after choosing a type, read model.devices_available (read‑only indices for that type)

model.device_type = "HAILORT/HAILO8L"  # Pick a type to query
print("available indices:", model.devices_available)  # e.g., [0] or [0,1]

Choose a device type

You can set device_type to pass a string for a fixed type (deterministic) or as a list (to allow fallbacks). If a list is used, PySDK picks the first available.

Deterministic pick (fixed device type):

# Lock to one runtime/hardware type
model_spec = ModelSpec(
    ..., model_properties={"device_type": "HAILORT/HAILO8L"}
)
model = model_spec.load_model()

Graceful fallback (try multiple types in order):

# Prefer HAILO8, then HAILO8L, then CPU as a last resort
model_spec = ModelSpec(
    ...,
    model_properties={
        "device_type": ["HAILORT/HAILO8", "HAILORT/HAILO8L", "CPU"]
    },
)
model = model_spec.load_model()  # First available wins

Use exact strings from model.supported_device_types to avoid typos.

(Optional) Pin a specific device index

On multi‑card systems, you can set devices_selected to choose which device ID to use:

model.device_type = "HAILORT/HAILO8L"
print(model.devices_available)  # e.g., [0, 1]
model.devices_selected = [1]  # Choose card #1

• devices_available: read‑only list of indices for the current device_type
• devices_selected: writable list of indices to use

Set‑once vs. tweak‑later

• Set once in ModelSpec: Use device_type (single or list) to ensure consistent, team‑wide defaults.
• Tweak later on the model: Use devices_selected to pin a card, or switch device_type interactively while exploring.

Common pitfalls

• String mismatch: Use only values from model.supported_device_types.
• “No device found”: Check hardware presence using degirum sys-info. Consider fallback lists for device_type.
• Wrong device used: Set devices_selected = [index] after choosing device_type.