# Model Evaluation Support

{% hint style="info" %}
This API Reference is based on DeGirum Tools version 1.2.0.
{% endhint %}

## Model Evaluation Support Module Overview <a href="#model-evaluation-support-module-overview" id="model-evaluation-support-module-overview"></a>

This module provides base classes and utilities for model evaluation, including performance metrics calculation, ground truth comparison, and evaluation result reporting. It supports various evaluation scenarios and metrics.

Key Features

* **Base Evaluator Class**: Abstract base class for model evaluators
* **YAML Configuration**: Support for evaluator configuration via YAML files
* **Flexible Evaluation**: Support for different evaluation metrics and scenarios
* **Result Reporting**: Standardized evaluation result reporting
* **Ground Truth Integration**: Support for comparing model outputs with ground truth

Typical Usage

1. Create a custom evaluator by subclassing ModelEvaluatorBase
2. Configure the evaluator using YAML or constructor parameters
3. Run evaluation on test datasets
4. Analyze and report evaluation results

Integration Notes

* Works with DeGirum PySDK models
* Supports standard evaluation metrics
* Handles various input formats
* Provides extensible evaluation framework

Key Classes

* `ModelEvaluatorBase`: Base class for model evaluators

Configuration Options

* Model parameters
* Evaluation metrics
* Dataset paths
* Ground truth format

## Classes <a href="#classes" id="classes"></a>

## ModelEvaluatorBase <a href="#modelevaluatorbase" id="modelevaluatorbase"></a>

`ModelEvaluatorBase`

Bases: `ABC`

Base class for model evaluators.

This abstract class initializes a model object, loads configuration parameters and defines the interface for performing evaluation.

Parameters:

| Name       | Type    | Description                                                                                                                                                                                    | Default    |
| ---------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| `model`    | `Model` | Model instance to evaluate.                                                                                                                                                                    | *required* |
| `**kwargs` | `Any`   | Arbitrary model or evaluator parameters. Keys matching the model attributes are applied directly to the model. Remaining keys are assigned to the evaluator instance if such attributes exist. | `{}`       |

Attributes:

| Name    | Type    | Description                |
| ------- | ------- | -------------------------- |
| `model` | `Model` | The model being evaluated. |

### ModelEvaluatorBase Methods <a href="#modelevaluatorbase-methods" id="modelevaluatorbase-methods"></a>

#### \_\_init\_\_(model, ...) <a href="#init" id="init"></a>

`__init__(model, **kwargs)`

Initialize the evaluator.

Parameters:

| Name       | Type    | Description                                                                                                             | Default    |
| ---------- | ------- | ----------------------------------------------------------------------------------------------------------------------- | ---------- |
| `model`    | `Model` | PySDK model object.                                                                                                     | *required* |
| `**kwargs` | `Any`   | Arbitrary model or evaluator parameters. Keys must either match model attributes or attributes of `ModelEvaluatorBase`. | `{}`       |

#### evaluate(image\_folder\_path, ...) <a href="#evaluate" id="evaluate"></a>

`evaluate(image_folder_path, ground_truth_annotations_path, max_images=0)`

`abstractmethod`

Evaluate the model on a dataset.

Parameters:

| Name                            | Type  | Description                                                                | Default    |
| ------------------------------- | ----- | -------------------------------------------------------------------------- | ---------- |
| `image_folder_path`             | `str` | Directory containing evaluation images.                                    | *required* |
| `ground_truth_annotations_path` | `str` | Path to the ground truth JSON file in COCO format.                         | *required* |
| `max_images`                    | `int` | Maximum number of images to process. `0` uses all images. Defaults to `0`. | `0`        |

Returns:

| Type   | Description                                 |
| ------ | ------------------------------------------- |
| `list` | Evaluation statistics (algorithm specific). |

#### init\_from\_yaml(model, ...) <a href="#init_from_yaml" id="init_from_yaml"></a>

`init_from_yaml(model, config_yaml)`

`classmethod`

Construct an evaluator from a YAML file.

Parameters:

| Name          | Type                     | Description                                                      | Default    |
| ------------- | ------------------------ | ---------------------------------------------------------------- | ---------- |
| `model`       | `Model`                  | PySDK model object.                                              | *required* |
| `config_yaml` | `Union[str, TextIOBase]` | Path or open stream with evaluator configuration in YAML format. | *required* |

Returns:

| Name   | Type   | Description                    |
| ------ | ------ | ------------------------------ |
| `Self` | `Self` | Instantiated evaluator object. |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.degirum.com/degirum-tools/support/eval_support.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.