# Device Management for Inference

AI Models can be run on various hardware configurations, and the DeGirumJS provides a flexible way to manage device types chosen for inference. This is particularly useful when you want to switch between different hardware accelerators or runtimes without having to significantly change your code.

Both `AIServerModel` and `CloudServerModel` classes offer flexible ways to manage device types, allowing you to configure and switch between devices dynamically.

## Supported Device Types

Each model has a set of `SupportedDeviceTypes`, which indicates the runtime/device combinations that are compatible for inference. The format for device types is `"RUNTIME/DEVICE"`, where:

* **RUNTIME** refers to the AI engine or runtime used for inference (e.g., `TENSORRT`, `OPENVINO`).
* **DEVICE** refers to the hardware type (e.g., `CPU`, `GPU`, `NPU`).

## AIServerModel / CloudServerModel Device Management

In the `AIServerModel` and `CloudServerModel` classes, device management is integrated into both the initialization and runtime phases of the model lifecycle. Below are key scenarios and examples:

### Default Device Type Selection

When you load a model without specifying a device type, the default device type **specified in the model parameters** is selected.

{% code overflow="wrap" %}

```javascript
let model = await zoo.loadModel('your_model_name');
console.log(model.deviceType); // Outputs: "DefaultRuntime/DefaultAgent"
```

{% endcode %}

### Switching Device Types After Initialization

You can change the device type even after the model has been initialized. The model will validate the requested device type against the system’s supported device types.

{% code overflow="wrap" %}

```javascript
model.deviceType = 'RUNTIME2/CPU';
console.log(model.deviceType); // Outputs: "RUNTIME2/CPU"
```

{% endcode %}

If the requested device type is not valid, an error will be thrown.

### Specifying a Device Type During Initialization

You can specify a device type when loading the model. The model will start with the specified device type if it’s available.

{% code overflow="wrap" %}

```javascript
let model = await zoo.loadModel('your_model_name', { deviceType: 'RUNTIME2/CPU' });
console.log(model.deviceType); // Outputs: "RUNTIME2/CPU"
```

{% endcode %}

### Handling Multiple Device Types

The SDK allows you to provide a list of device types. The **first available** option in the list will be selected.

{% code overflow="wrap" %}

```javascript
model.deviceType = ['RUNTIME3/CPU', 'RUNTIME1/CPU'];
console.log(model.deviceType); // Outputs: "RUNTIME3/CPU" if available, otherwise "RUNTIME1/CPU"
```

{% endcode %}

### Fallback and Error Handling

If none of the specified device types are supported, the model will throw an error, ensuring that only valid configurations are used.

{% code overflow="wrap" %}

```javascript
try {
    model.deviceType = ['INVALID/DEVICE', 'ANOTHER_INVALID/DEVICE'];
} catch (e) {
    console.error('Error: Invalid device type selection');
}
```

{% endcode %}

### Supported Device Types

You can check the supported device types for a model using the `supportedDeviceTypes` property.

{% code overflow="wrap" %}

```javascript
console.log(model.supportedDeviceTypes); // Outputs: ["RUNTIME1/CPU", "RUNTIME2/CPU"]
```

{% endcode %}

### System Supported Device Types

You can check the system’s list of supported devices for inference using the `getSupportedDevices()` method of the `dg_sdk` class.

{% code overflow="wrap" %}

```javascript
let dg = new dg_sdk();
let aiserverDevices = dg.getSupportedDevices('targetAIServerIp');
console.log(aiserverDevices); // Outputs: ["RUNTIME1/CPU", "RUNTIME2/CPU", "RUNTIME3/CPU"]
let cloudDevices = dg.getSupportedDevices('cloud');
console.log(cloudDevices); // Outputs: ["RUNTIME1/CPU", "RUNTIME2/CPU", "RUNTIME3/CPU"]
```

{% endcode %}

Device management in both `AIServerModel` and `CloudServerModel` is designed to be flexible, allowing you to fine-tune the inference environment. You can easily switch between device types, handle fallbacks, and ensure that your models are always running on supported configurations.