Device selection

Choose the device type (runtime + hardware) your model runs on. This guide covers supported types, discovery, deterministic vs. fallback selection, and pinning specific cards.

Estimated read time: 3 minutes

Pick where your model runs by selecting a device type (runtime + hardware) and optionally a device index. This guide focuses on device selection. For performance tuning (e.g., latency, throughput, etc.), see the advanced guides.

Model JSON info

Each model package defines a default and supported device list in its JSON file:

"DEVICE": [
  {
    "DeviceType": "METIS",
    "RuntimeAgent": "AXELERA",
    "SupportedDeviceTypes": "AXELERA/METIS"
  }
]

Interpretation:

• Default: METIS and AXELERA — derived from DeviceType and RuntimeAgent
• Supported list: AXELERA/METIS.

At runtime, this shows up as model.supported_device_types (read‑only).

Device discovery

1

Load the model

from degirum_tools import ModelSpec
model_spec = ModelSpec(
    model_name="…",
    zoo_url="degirum/axelera",
    inference_host_address="@local",
)
model = model_spec.load_model()

2

Check supported device types

print(
    "supported:", model.supported_device_types
)  # e.g., ['AXELERA/METIS']

3

Check what's available on this machine

• CLI: degirum sys-info → lists discovered hardware per runtime
• Or, after choosing a type, read model.devices_available (read‑only indices for that type)

model.device_type = "AXELERA/METIS"  # Pick a type to query
print("available indices:", model.devices_available)  # e.g., [0]

Choose a device type

You can set device_type to pass a string for a fixed type (deterministic) or as a list (to allow fallbacks). If a list is used, PySDK picks the first available.

Deterministic pick (fixed device type):

# Lock to one runtime/hardware type
model_spec = ModelSpec(
    ..., model_properties={"device_type": "AXELERA/METIS"}
)
model = model_spec.load_model()

Graceful fallback (try multiple types in order):

# Prefer Metis, then CPU as a last resort
model_spec = ModelSpec(
    ...,
    model_properties={
        "device_type": ["AXELERA/METIS", "CPU"]
    },
)
model = model_spec.load_model()  # First available wins

Use exact strings from model.supported_device_types to avoid typos.

(Optional) Pin a specific device index

On multi‑card systems, you can set devices_selected to choose which device ID to use:

model.device_type = "AXELERA/METIS"
print(model.devices_available)  # e.g., [0]
model.devices_selected = [0]  # Choose the first available card

• devices_available: read‑only list of indices for the current device_type
• devices_selected: writable list of indices to use

Set‑once vs. tweak‑later

• Set once in ModelSpec: Use device_type (single or list) to ensure consistent, team‑wide defaults.
• Tweak later on the model: Use devices_selected to pin a card, or switch device_type interactively while exploring.

Common pitfalls

• String mismatch: Use only values from model.supported_device_types.
• “No device found”: Check hardware presence using degirum sys-info. Consider fallback lists for device_type.
• Wrong device used: Set devices_selected = [index] after choosing device_type.