Understanding Results

This page explains the structure and content of inference result objects returned by the AIServerModel and CloudServerModel classes.

The AIServerModel and CloudServerModel classes return a result object that contains the inference results from the predict and predict_batch functions.

Example:

let someResult = await someModel.predict(image);
console.log(someResult);

For example, the result can be structured like this:

{
    "result": [
        [
            { "category_id": 1, "label": "foo", "score": 0.2 },
            { "category_id": 0, "label": "bar", "score": 0.1 }
        ],
        "frame123"
    ],
    "imageFrame": imageBitmap
}

Accessing the Result Data

• Inference Results: Access the main results using someResult.result[0].

• Frame Info / Number: Get the frame information or frame number using someResult.result[1].

• Original Input Image: Access the original input image with someResult.imageFrame.

Inference Result Types

The inference results can be one of the following types:

• Detection

• Classification

• Pose Detection

• Segmentation

Example Results

1. Detection Result

Detection results include bounding boxes (bbox) along with category IDs, labels, and confidence scores for detected objects:

{
    "result": [
        [
            {
                "bbox": [101.98, 77.67, 175.04, 232.99],
                "category_id": 0,
                "label": "face",
                "score": 0.856
            },
            {
                "bbox": [314.91, 52.55, 397.32, 228.70],
                "category_id": 0,
                "label": "face",
                "score": 0.844
            }
        ],
        "frame_15897"
    ],
    "imageFrame": {}
}

In this example:

• bbox represents the coordinates for each detected object's bounding box.

• category_id is the numerical ID of the detected category.

• label is the label of the detected category.

• score represents the confidence of the detection.

1. Pose Detection Result

Pose detection results include landmarks, with each landmark having coordinates (x, y), labels, and confidence scores:

{
    "result": [
        [
            {
                "landmarks": [
                    { "category_id": 0, "label": "Nose", "landmark": [93.99, 115.81], "score": 0.9986 },
                    { "category_id": 1, "label": "LeftEye", "landmark": [110.31, 98.96], "score": 0.9988 }
                ],
                "score": 0.4663
            }
        ],
        "frame_18730"
    ],
    "imageFrame": {}
}

In this example:

• landmarks represent detected joints or points of interest with coordinates (x, y), a label, and a confidence score.

• The connect field (if present) indicates which landmarks should be connected in visualizations.

1. Classification Result

Classification results include category IDs, labels, and confidence scores but typically don’t have bounding boxes:

{
    "result": [
        [
            { "category_id": 401, "label": "academic gown, academic robe, judge's robe", "score": 0.8438 },
            { "category_id": 618, "label": "lab coat, laboratory coat", "score": 0.0352 }
        ],
        "frame_19744"
    ],
    "imageFrame": {}
}

In this example:

• The model classifies the image into categories with associated confidence scores.