Compound Models
Compound Models Module Overview
This module provides a toolkit for creating compound models using the DeGirum PySDK.
A compound model orchestrates multiple underlying models into a pipeline to enable complex inference scenarios. Common examples include:
Detecting objects and then classifying each detected object.
Running coarse detection first, then applying a refined detection model on specific regions.
Combining outputs from multiple independent models into a unified inference result.
Compound models run in a single thread and are intended primarily for simple usage scenarios. Compound models still provide efficient batch prediction pipelining using batch_predict() in non-blocking mode. For more performant applications requiring better scalability and more flexible connections, we recommend using Gizmos, which in multiple threads.
Key Concepts
Model Composition: Compound models sequentially (or concurrently) invoke multiple models. Typically, results from the first model (e.g., bounding boxes from detection) feed into subsequent models (classification or refined detection).
Pipeline Workflow: A typical workflow involves:
Run
model1
to identify regions of interest (ROIs).Crop these ROIs and run them through
model2
.Integrate or transform outputs from
model2
back into the original context.
Unified Model Interface: All compound models follow the same interface as regular models in DeGirum SDK, providing
.predict()
for single frames and.predict_batch()
for iterators of frames.
Included Compound Models
CombiningCompoundModel: Combines results from two models run concurrently on the same input.
CroppingCompoundModel: Crops regions identified by
model1
and feeds them intomodel2
.CroppingAndClassifyingCompoundModel: Specialized pipeline: object detection (
model1
) followed by classification (model2
) of each detected object.CroppingAndDetectingCompoundModel: Pipeline for refined detection: initial coarse detection (
model1
) followed by detailed detection (model2
) within each ROI.RegionExtractionPseudoModel: Extracts predefined regions of interest without actual inference, optionally filtering by motion detection.
Basic Usage Examples
Detection + Classification:
Detection + Detection:
See class-level documentation below for details on individual classes and additional configuration options.
Classes
ModelLike
ModelLike
Bases: ABC
A base class which provides a common interface for all models, similar to PySDK model class.
When calling predict_batch(data)
, each item in data
can be:
A single frame (image/array/etc.), or
A 2-element tuple in the form
(frame, frame_info)
.
The frame_info
object (of any type) then appears in the final InferenceResults.info
attribute, allowing you to carry custom metadata through the pipeline.
Functions
__call__(data)
__call__(data)
Perform a whole inference lifecycle on a single frame (callable alias to predict()
).
Parameters:
data
any
Inference input data, typically an image or array, or a tuple (frame, frame_info)
.
required
Returns:
InferenceResults or None
The combined inference result object, or None if no result.
predict(data)
predict(data)
Perform a whole inference lifecycle on a single frame.
Parameters:
data
any
Inference input data, typically an image or array, or a tuple (frame, frame_info)
.
required
Returns:
InferenceResults or None
The combined inference result object, or None if no result.
predict_batch(data)
predict_batch(data)abstractmethod
Perform a whole inference lifecycle for all objects in the given iterator object (for example, list
).
Each item in data
can be a single frame (any type acceptable to the model) or
a 2-element tuple (frame, frame_info)
. In the latter case, frame_info
is
carried through and placed in InferenceResults.info
for that frame.
Parameters:
data
iterator
Inference input data iterator object such as a list or a generator function. Each element returned by this iterator should be compatible with what regular PySDK models accept.
required
Returns:
Iterator[InferenceResults or None]
A generator or iterator over the inference result objects (or None in non-blocking mode). This allows you to use the result in for
loops.
FrameInfo
FrameInfo
Class to hold frame info.
By default, DeGirum PySDK allows you to pass any arbitrary object as 'frame info'
alongside each frame in predict_batch()
.
Attributes:
result1
any
The result object produced by the first model in a compound pipeline. For instance, an InferenceResults
object.
sub_result
int
The index of a sub-result within result1
(e.g., which bounding box led to this cropped image).
CompoundModelBase
CompoundModelBase
Compound model class which combines two models into one pipeline.
One model is considered primary (model1), and the other is nested (model2).
The primary model (model1
) processes the input frames. Its results
are then passed to the nested model (model2
).
Attributes
_custom_postprocessor
_custom_postprocessorproperty
writable
Custom postprocessor is not supported for compound models. This property will always return None.
Note
Attempting to set this property will raise an Exception, because compound models do not support custom postprocessors.
Returns:
None
Always returns None for compound models.
non_blocking_batch_predict
non_blocking_batch_predictproperty
writable
Flag controlling whether predict_batch()
operates in non-blocking mode
for model1. In non-blocking mode, predict_batch()
can yield None
when no results are immediately available.
Returns:
bool
True if non-blocking mode is enabled, False otherwise.
Classes
NonBlockingQueue
NonBlockingQueue
Bases: Queue
Specialized non-blocking queue which acts as an iterator to feed data to the nested model.
Functions
__iter__
__iter__()
Yield items from the queue until a None
sentinel is reached.
Yields:
any or None
The item from the queue, or None
if the queue is empty.
Functions
__init__(model1, ...)
__init__(model1, model2)
Constructor.
Parameters:
model1
Model to be used for the first step of the pipeline.
required
model2
Model to be used for the second step of the pipeline.
required
attach_analyzers(analyzers)
attach_analyzers(analyzers)
Attach analyzers to a model.
Parameters:
analyzers
Union[ResultAnalyzerBase, list[ResultAnalyzerBase], None]
A single analyzer, or a list of analyzer objects, or None
to detach all analyzers.
required
predict_batch(data)
predict_batch(data)
Perform a whole inference lifecycle for all objects in the given iterator object (for example, list
).
Works in a pipeline fashion
Pass input frames (or
(frame, frame_info)
tuples) tomodel1
.Use
queue_result1(result1)
to feedmodel2
.Collect
model2
results, transform them withtransform_result2(result2)
,Yield the final output.
Parameters:
data
iterator
Inference input data iterator object such as a list or a generator function. Each element returned should be compatible with model inference requirements.
required
Returns:
Iterator[InferenceResults or None]
Generator object which iterates over the combined inference result objects (or None in non-blocking mode). This allows you to use the result in for
loops.
queue_result1(result1)
queue_result1(result1)abstractmethod
Process the result of the first model and put it into the queue.
Parameters:
result1
InferenceResults
Prediction result of the first model.
required
transform_result2(result2)
transform_result2(result2)abstractmethod
Transform (or integrate) the result of the second model.
Parameters:
result2
InferenceResults
Prediction result of the second model.
required
Returns:
InferenceResults or None
Transformed/combined result to be returned by the compound model. If None, that means no result is produced at this iteration.
CombiningCompoundModel
CombiningCompoundModel
Compound model class which executes two models in parallel on the same input data and merges their results.
Restriction: both models should produce the same type of inference results (e.g., both detection).
Functions
queue_result1(result1)
queue_result1(result1)
Queues the original image from result1
and a new FrameInfo
instance
that references result1
. This (frame, frame_info)
tuple is then read by model2
.
Parameters:
result1
InferenceResults
Inference result from model1. We extract result1.image
as the frame, and create a FrameInfo
so we know which result1
this frame corresponds to.
required
transform_result2(result2)
transform_result2(result2)
Merges results from model2
into result1
that was stored in FrameInfo
.
This implementation appends the second model's inference results to the first model's result list.
Parameters:
result2
InferenceResults
Inference result of the second model, which has info
attribute containing the FrameInfo
.
required
Returns:
InferenceResults
The merged inference results (model1 + model2).
CroppingCompoundModel
CroppingCompoundModel
Compound model class which crops the original image according to results of the first model and then passes these cropped images to the second model.
Restriction: the first model should be of object detection type.
Functions
__init__(model1, ...)
__init__(model1, model2, crop_extent=0.0, crop_extent_option=CropExtentOptions.ASPECT_RATIO_NO_ADJUSTMENT)
Constructor.
Parameters:
model1
Object detection model that produces bounding boxes.
required
model2
Classification model that will process each cropped region.
required
crop_extent
float
Extent of cropping (in percent of bbox size) to expand the bbox.
0.0
crop_extent_option
CropExtentOptions
Method of applying extended crop to the input image for model2.
ASPECT_RATIO_NO_ADJUSTMENT
_adjust_bbox(bbox, ...)
_adjust_bbox(bbox, image_sz)
Inflate bbox coordinates to the crop extent according to the chosen approach and adjust to image size.
Parameters:
bbox
list[float]
Bbox coordinates in [x1, y1, x2, y2] format.
required
image_sz
tuple[int, int]
Image size (width, height) of the original image.
required
Returns:
list[float]
Adjusted bbox coordinates.
queue_result1(result1)
queue_result1(result1)
Put the original image into the queue, along with bounding boxes from the first model.
If no bounding boxes are detected, puts a small black image to keep the pipeline in sync.
Parameters:
result1
InferenceResults
Prediction result of the first (object detection) model.
required
CroppingAndClassifyingCompoundModel
CroppingAndClassifyingCompoundModel
Compound model class which
Runs an object detection (model1) to generate bounding boxes.
Crops each bounding box from the original image.
Runs a classification (model2) on each cropped image.
Patches the original detection results with the classification labels.
Restriction: first model must be object detection, second model must be classification.
Functions
__init__(model1, ...)
__init__(model1, model2, crop_extent=0.0, crop_extent_option=CropExtentOptions.ASPECT_RATIO_NO_ADJUSTMENT)
Constructor.
Parameters:
model1
An object detection model producing bounding boxes.
required
model2
A classification model to classify each cropped region.
required
crop_extent
float
Extent of cropping (in percent of bbox size).
0.0
crop_extent_option
CropExtentOptions
Specifies how to adjust the bounding box before cropping.
ASPECT_RATIO_NO_ADJUSTMENT
_finalize_current_result
_finalize_current_result()
Finalize the current results by returning a copy of the first model results patched with classification labels.
Returns:
InferenceResults or None
Final result for the previous frame, or None if there was no frame.
predict_batch(data)
predict_batch(data)
Perform the full inference lifecycle for all objects in the given iterator (for example, list
),
but patch model1 bounding box labels with classification results from model2.
Parameters:
data
iterator
Iterator of input frames for model1. Each element returned by this iterator should be compatible with regular PySDK models.
required
Returns:
Iterator[InferenceResults]
Yields the detection results with patched classification labels after each frame completes.
transform_result2(result2)
transform_result2(result2)
Transform (patch) the classification result into the original detection results.
Parameters:
result2
InferenceResults
Classification result of model2.
required
Returns:
InferenceResults or None
The detection result (from model1) patched with classification labels, or None if we haven't moved to a new frame yet.
CroppingAndDetectingCompoundModel
CroppingAndDetectingCompoundModel
Compound model class which
Uses an object detection model (model1) to generate bounding boxes (ROIs).
Crops each bounding box from the original image.
Uses another object detection model (model2) to further detect objects in each cropped region.
Combines the results of the second model from all cropped regions, mapping coords back to the original image.
Optionally, you can add model1 detections to the final result and/or apply NMS.
Restriction
First model should be object detection or pseudo-detection model like RegionExtractionPseudoModel
,
second model should be object detection.
Functions
__init__(model1, ...)
__init__(model1, model2, *, crop_extent=0.0, crop_extent_option=CropExtentOptions.ASPECT_RATIO_NO_ADJUSTMENT, add_model1_results=False, nms_options=None)
Constructor.
Parameters:
model1
Object detection model (or pseudo-detection).
required
model2
Object detection model.
required
crop_extent
float
Extent of cropping in percent of bbox size.
0.0
crop_extent_option
CropExtentOptions
Method of applying extended crop to the input image for model2.
ASPECT_RATIO_NO_ADJUSTMENT
add_model1_results
bool
If True, merges model1 detections into the final combined result.
False
nms_options
Optional[NmsOptions]
If provided, applies non-maximum suppression (NMS) to the combined result.
None
_finalize_current_result
_finalize_current_result()
Finalize the combined detection results from model2 (optionally merged with model1 results). Apply NMS if requested.
Returns:
InferenceResults or None
The final combined detection results for the previous frame, or None if no data.
predict_batch(data)
predict_batch(data)
Perform the full inference lifecycle for all objects in the given iterator object (for example, list
):
model1 detects or extracts bounding boxes (ROIs).
Each ROI is passed to model2 for detection.
model2 results for each ROI are merged and mapped back to original coordinates.
(Optional) NMS is applied and results from model1 can be included.
Parameters:
data
iterator
Iterator of input frames for model1. Each element returned by this iterator should be compatible with regular PySDK models.
required
Returns:
Iterator[InferenceResults]
Generator object which iterates over final detection results with possibly merged bounding boxes, adjusted to original image coordinates.
transform_result2(result2)
transform_result2(result2)
Combine detection results from model2 for each bbox from model1, translating coordinates back to the original image space.
Parameters:
result2
InferenceResults
Detection result of the second model.
required
Returns:
InferenceResults or None
The final detection results for the previous frame if a new frame started, or None otherwise.
RegionExtractionPseudoModel
RegionExtractionPseudoModel
Pseudo-model class which extracts regions from a given image according to given ROI boxes.
Attributes
custom_postprocessor
custom_postprocessorproperty
writable
Custom postprocessor class. Required for attaching analyzers to the pseudo-model.
When set, this replaces the default postprocessor with a user-defined postprocessor.
Returns:
Optional[type]
The user-defined postprocessor class, or None if not set.
non_blocking_batch_predict
non_blocking_batch_predictproperty
writable
Controls non-blocking mode for predict_batch()
.
Returns:
bool
True if non-blocking mode is enabled; otherwise False.
Functions
__init__(roi_list, ...)
__init__(roi_list, model2, *, motion_detect=None)
Constructor.
Parameters:
roi_list
Union[list, ndarray]
Can be: - list of ROI boxes in [x1, y1, x2, y2]
format, - 2D NumPy array of shape (N, 4), - 3D NumPy array of shape (K, M, 4), which will be flattened.
required
model2
Model
The second model in the pipeline.
required
motion_detect
Optional[MotionDetectOptions]
* When None, disabled motion detection. * When not None, applies motion detection before extracting ROI boxes. Boxes without motion are skipped.
None
predict_batch(data)
predict_batch(data)
Perform a pseudo-inference that outputs bounding boxes defined in roi_list
.
If motion detection is enabled, skip ROIs where motion is not detected.
Parameters:
data
iterator
Iterator over the input images or frames. Each element returned by this iterator should be compatible with regular PySDK models.
required
Returns:
Iterator[InferenceResults or None]
Yields pseudo-inference results containing ROIs as bounding boxes, or yields None in non-blocking mode when no data is available.
NmsOptions
NmsOptionsdataclass
Options for non-maximum suppression (NMS) algorithm.
Attributes:
threshold
float
IoU or IoS threshold for box clustering (range [0..1]).
use_iou
bool
If True, use IoU for box clustering, otherwise IoS.
box_select
NmsBoxSelectionPolicy
Box selection policy (e.g., keep the box with the highest probability).
MotionDetectOptions
MotionDetectOptionsdataclass
Options for motion detection algorithm.
Attributes:
threshold
float
Threshold for motion detection [0..1], representing fraction of changed pixels relative to frame size.
look_back
int
Number of frames to look back to detect motion.
Last updated
Was this helpful?