Skip to content

PySDK Retired Versions

Version 0.11.1 (3/13/2024)

New Features and Modifications

  1. NPU device support is implemented for OpenVINO runtime. To make a model for NPU device you specify "DeviceType": "NPU" in model JSON file. OpenVINO runtime version 2023.3.0 is required for NPU support.

  2. Python version 3.12 is initially supported by PySDK for Linux and Windows platforms.

  3. Improvements for "Tensor" input type (InputType model parameter equal to "Tensor"):

    • The following tensor element types are supported.

      • "DG_FLT"
      • "DG_UINT8"
      • "DG_INT8"
      • "DG_UINT16"
      • "DG_INT16"
      • "DG_INT32"
      • "DG_INT64"
      • "DG_DBL"
      • "DG_UINT32"
      • "DG_UINT64"

      These type strings you assign to InputRawDataType model parameter.
      In previous versions, only "DG_FLT" and "DG_UINT8" types are supported.

    • Variable tensor shapes are supported. Now you can specify any combination of InputN, InputH, InputW, and InputC model parameters. They will define the input tensor shape in that order. For example, if you specify InputN=1, InputC=77 and omit InputH and InputW, this will give 2-D tensor of shape [1,77]. In previous versions you have to specify all four of them, which always gives 4-D tensor shapes.

    • InputQuantEn model parameter is now ignored by tensor pre-processor: now you have to specify InputRawDataType to match actual model input tensor data type and provide tensor data already converted to this data type.

  4. InputN, InputH, InputW, and InputC model parameters are now not mandatory: you may specify any subset of them.

  5. PySDK InferenceResults.image_overlay() method now returns a copy of input image instead of raising an exception. This gives possibility to safely call this method in case on "None" postprocessor type (OutputPostprocessType model parameter is set to "None")

  6. ModelParams class __str__() operator now prints all model parameters including ones, which are not specified in model JSON file. For such parameters their default values are printed.

  7. If DG_MEMORY_LIMIT_BYTES environment variable is defined, its value is used for AI server in-memory model cache size limit. When it is not defined, one half of the physical memory is used as a cache size limit, as in previous versions. This feature is useful, when AI server is running in Docker container and you want to further limit AI server cache memory size.

Bug Fixes

  1. PostProcessorInputs model parameter presence is now checked only for detection post-processor types to avoid unnecessary errors for post-processor types, which do not use this parameter, such as "None".

Version 0.11.0 (2/10/2024)

New Features and Modifications

  1. Support for different OpenVINO versions is implemented. Now PySDK can work with the following OpenVINO versions:

    • 2022.1.1
    • 2023.2.0
    • 2023.3.0

    When two or more OpenVINO installations are present on a system, the newest version will be used.

  2. Results filtering by class labels and category IDs is implemented: new output_class_set property is added to degirum.model.Model class for this purpose.

    By default, all results are reported by the model predict methods. However, you may want to include only results which belong to certain categories: either having certain class labels or category IDs. To achieve that, you can specify a set of class labels (or, alternatively, category IDs) so only inference results, which class labels (or category IDs) are found in that set, are reported, and all other results are discarded. You assign such a set to degirum.model.Model.output_class_set property.

    For example, you may want to include only results with class labels "car" and "truck":

    # allow only results with "car" and "truck" class labels
model.output_class_set = {"car", "truck"}

    Or you may want to include only results with category IDs 1 and 3:

    # allow only results with 1 and 3 category IDs
model.output_class_set = {1, 3}

    This category filtering is applicable only to models which have "label" (or "category_id") keys in their result dictionaries. For all other models this category filter will be ignored.

Bug Fixes

  1. When two different models have two different Python postprocessor implementations saved into files with the same name, only the first Python postprocessor module gets loaded on AI server. This happens because it is loaded into Python global 'sys.modules` collection as a module named after the file name, and if two files have the same name, they collide.

  2. When an implementation of Python postprocessor in a model gets changed, and that model was already loaded on AI server, then the Python postprocessor module is not reloaded on the next model load. This is because once the Python module is loaded into Python interpreter, it is saved in 'sys.modules` collection, and any attempt to load it again just takes it from there.

  3. Performing inferences with ONNX runtime agent (degirum.model.Model.model_info.RuntimeAgent equal to "ONNX") may cause AI server to crash.

Version 0.10.4 (1/24/2024)

New Features and Modifications

The dependency on CoreClient PySDK module is made on-demand, meaning that CoreClient PySDK module is attempted to load only when local inference is invoked, or when local AI server is started from PySDK. This allows using cloud and AI server client functionality of PySDK on systems with missing CoreClient's module dependencies.

Bug Fixes

Fixed bug in YOLOv8 post-processor affecting models with non-square input tensors. Previously Y-coordinate (height) of all detections coming from YOLOv8 models with input image resolutions with width not equal to height would be misinterpreted; now the behavior is correct.

Version 0.10.3 (1/17/2024)

New Features and Modifications

  1. ORCA1 firmware version 1.1.9 is included in this release. This firmware implements measures to improve data integrity of DDR4 external memory when entering/leaving low-power mode.

  2. To avoid any possible future incompatibilities, the PySDK package requirements now explicitly limit upper versions for all dependencies to be one major revision more than corresponding lower version. For example: requests >= 2.30.0 becomes requests >= 2.30.0, < 3.0.

  3. AI annotations drawing performance is greatly improved for object detection annotations.

  4. Default value for alpha blending coefficient is set to 1.0: disable blending. This is performance-improvement measure.

  5. Color selection for different classes in case when a list of colors is assigned to degirum.model.Model.overlay_color property, is improved. It is performed based on the class ID, if the object class ID is in the model dictionary. Otherwise new unique color is assigned to the class and it is associated with the class label. This mechanism produces stable color-to-class assignment from frame to frame and also allows combining results of multiple different models on a single annotation, assigning different colors to classes which may have the same class IDs but different class labels.

  6. Printing scores on AI annotations is now performed with type-dependent format: if the score is of integer type, there will be no fractional part. This improves readability in case of regression models producing integer results.

  7. Quality of OpenCV font used for AI annotations is improved.

  8. Model statistics formatting now uses wider columns to accommodate long statistics.

Version 0.10.2 (12/1/2023)

Discontinued Functionality

The N2X compiler support for DeGirum Orca 1.0 devices is discontinued. Starting from this version, N2X compiler cannot compile models for Orca 1.0 devices: only Orca 1.1 devices are supported.

However, runtime operations for Orca 1.0 devices are still fully supported: you can continue to use Orca 1.0 devices with already compiled models.

Bug Fixes

degirum server rescan-zoo and degirum server shutdown CLI commands do not work with new HTTP AI servers protocols. An attempt to execute such commands for AI servers launched with HTTP protocol option causes error messages.

Version 0.10.1 (11/2/2023)

New Features and Modifications

  1. The HTTP+WebSocket AI server protocol is initially supported for DeGirum AI Server.

    Starting from PySDK version 0.10.0, AI server supports two protocols: asio and http. The asio protocol is DeGirum custom socket-based AI server protocol, supported by all previous PySDK versions. The http protocol is a new protocol, which is based on REST HTTP requests and WebSockets streaming. The http protocol allows to use AI server from any programming language, which supports HTTP requests and WebSockets, such as browser-based JavaScript, which does not support native sockets, thus precluding the use of asio protocol.

    When you start AI server by executing degirum server start command, you specify the protocol using --protocol parameter, which can be asio, http, or both.

    If you omit this parameter, asio protocol will be used by default to provide compatible behavior with previous PySDK verisions.

    You select the http protocol by specifying --protocol http.

    You may select both protocols by specifying --protocol both. In this case, AI server will listen to both protocols on two consecutive TCP ports: the first port is used for asio protocol, the second port is used for http protocol.

    For example: start AI server to serve models from ./my-zoo directory, use asio protocol on port 12345, and use http protocol on port 12346:

    degirum server start --zoo ./my-zoo --port 12345 --protocol both

    On a client side, when you connect to AI server with http protocol, you have to prefix AI server hostname with http:// prefix, for example:

    zoo = dg.connect("http://localhost")

    To connect to AI server with asio protocol you simply omit the protocol prefix.

  2. Now you may pass arbitrary model properties (properties of degirum.model.Model`` class) as keyword arguments todegirum.zoo_manager.ZooManager.load_model` method. In this case these properties will be assigned to the model object.

    For example:

    model = zoo.load_model(model_name, output_confidence_threshold=0.5, input_pad_method="letterbox")

  3. Multi-classifier (or multi-label) classification models are initially supported. The post-processor type string, which is assigned to OutputPostprocessType model parameter, is "MultiLabelClassification". Each inference result dictionary contains the following keys:

    • classifier: object class string.
    • results: list of class labels and its scores. Scores are optional.

    The results list element is a dictionary with the following keys:

    • label: class label string.
    • score: optional class label probability.

    Example: 

    [
    {
        'classifier': 'vehicle color',
        'results': [
            {'label': 'red', 'score': 0.99},
            {'label': 'blue', 'score': 0.01}
        ]
    },
    {
        'classifier': 'vehicle type',
        'results': [
            {'label': 'car', 'score': 0.99},
            {'label': 'truck', 'score': 0.01}   
        ]
    }   
]

Bug Fixes

  1. Unclear error message 'NoneType' object has no attribute 'shape' appears when supplying non-existing file for model inference.

  2. Local AI inference of a model with Python post-processor hangs on model destruction due to Python GIL deadlock.

  3. degirum sys-info command re-initializes DeGirum Orca AI accelerator hardware not in interprocess-safe way, disrupting operation of other processes using the same Orca accelerator hardware. The first attempt to fix this bug was in PySDK version 0.9.6, this release finally fixes this bug.

Version 0.9.6 (10/17/2023)

New Features and Modifications

  1. New CoreInputFrameSize_bytes statistic is added to the inference statistics collection reported by degirum.model.Model.time_stats() method. This statistic represents the size (in bytes) of input frames received by AI model for inference.

Bug Fixes

  1. degirum sys-info command re-initializes DeGirum Orca AI accelerator hardware not in interprocess-safe way, disrupting operation of other processes using the same Orca accelerator hardware. For example, AI server is running on a system equipped with Orca accelerator hardware, and degirum sys-info command is executed on the same system. In this case the following error appears: "RPC control is in inconsistent state. Software recovery is not possible: please restart device".

  2. When performing AI inference of CPU-based ONNX quantized models using OpenVINO runtime on host computers equipped with modern AMD CPUs such as AMD RYZEN, the following error message appears: "Check 'false' failed at src/inference/src/core.cpp:131: could not create a primitive ... [ERROR]<continued> Failed to compile".

  3. When performing AI inference of models with audio input frame type, the following error message appears: "There is no Image inputs in model".

Version 0.9.3 (10/02/2023)

New Features and Modifications

  1. Inference on Intel® Arc™ family of GPUs with OpenVINO™ runtime is initially supported.

  2. Pretty-printing is implemented for model inference statistics. If model is an instance of degirum.model.Model class, then print(model.time_stats()) will print inference statistics in tabulated form, similar to the text below:

    Statistic                     ,    Min,    Avg,    Max,    Cnt
PythonPreprocessDuration_ms   ,   5.00,   5.00,   5.00,      1
CoreInferenceDuration_ms      , 349.94, 349.94, 349.94,      1
CoreLoadResultDuration_ms     ,   0.02,   0.02,   0.02,      1
CorePostprocessDuration_ms    ,   0.09,   0.09,   0.09,      1
CorePreprocessDuration_ms     ,   2.78,   2.78,   2.78,      1
DeviceInferenceDuration_ms    ,   0.00,   0.00,   0.00,      1
FrameTotalDuration_ms         , 610.34, 610.34, 610.34,      1

    str(model.time_stats()) expression will return the same text.

Bug Fixes

  1. Python post-processor support was broken. An attempt to specify Python post-processor for a model led to the following error: Model postprocessor type is not known: Python. Starting from this version, the Python post-processor support is restored the following way. If you want to use Python post-processor, then:

    • you need to specify the name of the Python file with your Python post-processor implementation in the PythonFile parameter of the POST_PROCESS section;
    • you need to specify one of supported PySDK post-processor types in the OutputPostprocessType parameter of the POST_PROCESS section;
    • the result format generated by your Python post-processor must be compatible with the PySDK post-processor type specified in the OutputPostprocessType parameter.

    Currently supported post-processor types are:

    • "None"
    • "Classification"
    • "Detection"
    • "FaceDetection"
    • "PoseDetection"
    • "HandDetection"
    • "Segmentation"

    The corresponding result formats are described in PySDK User's Guide, degirum.postprocessor.InferenceResults.results attribute description.

    For security reasons, at the time of this release the DeGirum cloud platform does not allow uploading to cloud model zoos models with Python post-processor for regular accounts: only cloud platform administrators can do it.

  2. TFLite runtime plugin was missing in PySDK package for Windows.

Version 0.9.2 (09/06/2023)

New Features and Modifications

  1. Plugin for ONNX runtime is initially supported. This plugin allows performing inferences of ONNX AI models directly on AI server host CPU without any AI accelerator.

    The ONNX runtime delivers better performance compared to OpenVINO™ runtime on ARM64 platforms, while OpenVINO&trade runtime delivers better performance than ONNX runtime on x86-64 platforms.

  2. Default values for some model properties are changed. The following is the list of changes:

    • degirum.model.Model.input_image_format:

      Was: "JPEG" for cloud inference, "RAW" for all other inference types

      Now: "RAW" for local inference, "JPEG" for all other inference types

    • degirum.model.Model.input_numpy_colorspace:

      Was: "RGB"

      Now: "auto", meaning it will be "BGR" for OpenCV backend and "RGB" for PIL backend

  3. The meaning of "auto" selection for degirum.model.Model.image_backend property has changed:

    Was: try to use PIL first, and if not installed, use OpenCV

    Now: try to use OpenCV first, and if not installed, use PIL

  4. AI server protocol is improved for robustness. To deal with unreliable network connection, the following retries have been implemented:

    • Retry on a client side when connecting to a server

    • Retry on a server side when connecting to a cloud zoo

  5. In-memory cache size limiting and cache eviction mechanism is implemented for AI server model cache. This greatly improves AI server robustness in case when multiple models were requested for inference during AI server lifetime: loading too many different models caused host memory exhaustion and AI server crash.

Bug Fixes

  1. If a cloud model zoo has capital letters in its name, there was no possibility to load models from such zoo. The following error message appeared in such case:

    DegirumException: Model zoo 'myorg/ZooWithCaps' is not found. (cloud server response: 400 Client Error: Bad Request for url: https://hub.degirum.com/zoo/v1/public/models/myorg/ZooWithCaps)

  2. degirum trace list command entrypoint did not list all available traces. In particular all traces defined in runtime plugins were not included in the list.

  3. In AI server host computer has integrated GPU, then this GPU as well as discrete GPU(s) are used by OpenVINO runtime plugin for GPU-based inferences. Since integrated GPU typically has much lower performance compared to discrete GPUs, this led to significant performance degradation when an inference happens to be scheduled on integrated GPU. Now integrated GPU is ignored if discrete GPU is present in the system.

  4. Loading too many different TFLite models caused AI server host memory exhaustion and AI server crash. To mitigate this bug, in-memory cache size limiting and cache eviction mechanism is implemented.

Version 0.9.1 (08/04/2023)

IMPORTANT: This release has critical bug fixes and incompatible changes with version 0.9.0. It is strongly recommended to upgrade from version 0.9.0 to version 0.9.1.

New Features and Modifications

  1. AI models of YOLOv8 family are redesigned to improve performance.

    New YOLOv8 models in the DeGirum public cloud zoo are incompatible with PySDK version 0.9.0, so in order to use YOLOv8 models you need to upgrade to version 0.9.1.

  2. Inference timeout handling is improved.

    • HTTP transactions of AI server with the cloud zoo had infinite timeouts in previous PySDK versions. This can lead to arbitrary long delays in case of poor connection between AI server and cloud zoo. In new version all such HTTP transactions now have finite timeouts.

    • Model-specific inference timeout is now passed to the cloud server, so the cloud server uses it instead of generic 180 second timeout. This greatly improves the cloud inference responsiveness.

Bug Fixes

  1. The following error message appears when you use PySDK version 0.9.0 and start AI server on systems equipped with ORCA1 AI hardware accelerator: Firmware image file 'orca-1.1.fi' is invalid or corrupt.

    This effectively prevents any usage of ORCA1 AI hardware accelerator with PySDK version 0.9.0, so you need to upgrade to version 0.9.1.

Version 0.9.0 (07/25/2023)

IMPORTANT: This release has changes in PySDK API

New Features and Modifications

  1. AI models of YOLOv8 family are initially supported.

  2. The possibility to define and install custom post-processor is implemented: degirum.model.Model.custom_postprocessor property is added for that purpose.

    When you want to work with some new AI model and PySDK does not yet provide post-processor class to interpret model results, then you may want to implement that post-processing code yourself.

    Such code typically takes the AI model output tensor data and interprets that raw tensor data to produce some meaningful results like bounding boxes, probabilities, etc. Then it renders these results on a top of original image to produce so-called image overlay.

    Starting from version 0.9.0, PySDK provides a way to seamlessly integrate such custom post-processing code so it will behave exactly like built-in post-processors. To do so, you need to complete the following two steps:

    1. Implement your own custom post-processor class.
    2. Instruct AI model object to use your custom post-processing class instead of built-in post-processor by assigning your new custom post-processor class to degirum.model.Model.custom_postprocessor property.

    Please refer to PySDK User Guide 0.9.0 for more details.

  3. Maximum size limit for AI model runtime in-memory cache is implemented. When the total size of all loaded AI models exceeds this limit, least recently used models are evicted from that cache.

  4. PySDK model filtering functionality (used in degirum.zoo_manager.ZooManager.list_models method and download-zoo CLI command) is modified to deal with quantized models. Before, it analyzed ModelQuantEn model parameter. Now it looks for quant or float suffixes in the model name. This is done to address the problem, when models which are internally quantized, have floating-point input/output tensors, and ModelQuantEn model parameter for such models is set to false.

Bug Fixes

  1. Concurrent access to Orca accelerator devices from multiple processes results in segmentation fault.

    Steps to reproduce:

    1. Start AI server on a system equipped with Orca accelerator device
    2. Run Python script, which performs AI inference of any Orca AI model on AI server at localhost: zoo = dg.connect('localhost', ...)
    3. Then run Python script, which performs AI inference of any Orca AI model directly on hardware: zoo = dg.connect(dg.LOCAL, ...)
    4. Then repeat step b.

Version 0.8.4 (07/12/2023)

New Features and Modifications

  1. degirum sys-info CLI command output has changed. Now it reports the device names prefixed with runtime agent names, which support those devices. For example, ORCA1 device is now reported as N2X/ORCA1. This modification is important to disambiguate the CPU device, which is supported by multiple runtimes such as OPENVINO/CPU, N2X/CPU, and TFLITE/CPU.

Bug Fixes

  1. Cloud inferences of AI models designed for third-party runtimes such as OpenVINO or TFLite may fail with the error message Model requires 'CPU' device, but no such device is found in the system. This happens because of incorrect scheduling of a model to a cloud farm node, which does not have proper third-party runtime installed.

Version 0.8.3 (07/11/2023)

New Features and Modifications

  1. Plugin for OpenVINO™ runtime is initially supported. This plugin allows performing inferences of ONNX AI models on devices supported by OpenVINO runtime, including:

    • CPU
    • Intel® Movidius™ Myriad™

    The plugin supports just-in-time compilation of AI models in ONNX format to OpenVINO format. Compiled models are then saved in the local cache for reuse.

Bug Fixes

  1. TensorRT runtime and TensorRT-supported devices were not recognized by degirum.zoo_manager.ZooManager.list_models method so no model filtering was possible for TensorRT runtime and TensorRT-supported devices.

Version 0.8.2 (06/23/2023)

New Features and Modifications

  1. Production version of ORCA1 AI Accelerator is initially supported. ORCA1 production version differs from ORCA1 engineering version in that it has smaller PCIe BAR size.

  2. Plugin for NVIDIA® TensorRT™ runtime is initially supported. This plugin allows performing inferences of ONNX AI models on devices supported by TensorRT runtime, including:

    • CPU
    • NVIDIA GPU
    • NVIDIA DLA

    The plugin supports just-in-time compilation of AI models in ONNX format to TensorRT format. Compiled models are then saved in the local cache for reuse.

Version 0.8.1 (06/07/2023)

IMPORTANT: This release has critical bug fixes for ORCA1 AI accelerator device.

New Features and Modifications

PySDK User Guide documentation is redesigned for improved look and feel.

Critical Fixes

New firmware is released for ORCA1 AI accelerator device, which fixes critical bug related to on-board DRAM operation. That bug caused DRAM data corruption.

Bug Fixes

  1. Device temperature DeviceTemperature_C as returned by degirum.model.Model.time_stats() method, is reported incorrectly for ORCA1 AI accelerator device.

  2. The label text shown on the image overlay result of object detection is shifted down in respect to the label box. This is especially visible when overlay font scale (degirum.model.Model.verlay_font_scale property) is set greater than 1.

Version 0.8.0 (05/19/2023)

IMPORTANT: This is the first release which supports DeGirum Orca 1.1 AI Accelerator

New Features and Modifications

DeGirum Orca 1.1 AI Accelerator is initially supported.

Orca 1.1 is DeGirum first production AI accelerator IC. It has the same functionality as pre-production Orca 1.0 AI accelerator IC, but delivers up to 40% performance improvements.

The kernel driver installation for Orca 1.1 is the same as for Orca 1.0.

The PySDK and DeGirum Cloud Platform support for Orca 1.1 is the same as for Orca 1.0.

However, the AI models compiled for Orca 1.0 are not compatible with Orca 1.1.

DeGirum public cloud model zoo now contains separate set of models compiled for Orca 1.1.

The names of Orca 1.1-compatible models have orca1 suffix in their name, like mobilenet_v1_imagenet--224x224_quant_n2x_orca1_1 (as opposed to orca for Orca 1.0: compare with mobilenet_v1_imagenet--224x224_quant_n2x_orca_1).

For Orca 1.1, the DeviceType property of ModelParams class (as returned by degirum.model.Model.model_info method) has ORCA1 value (as opposed to ORCA for Orca 1.0).

The Cloud Zoo browsing page on DeGirum Cloud Platform web-site has ORCA1 device type filter, which you can use to filter-out Orca 1.1-compatible models.

Version 0.7.0 (04/26/2023)

IMPORTANT: This release has changes in PySDK and C++ SDK APIs.

New Features and Modifications

  1. Starting from ver. 0.7.0, PySDK releases are released to PyPI.org. Now, to install PySDK using pip it is enough to invoke pip install degirum command without specifying --extra-index-url parameter.

    Previous PySDK versions are still available from DeGirum index site by specifying --extra-index-url https://degirum.github.io/simple

  2. Starting from ver. 0.7.0, PySDK can be installed on Ubuntu Linux 22.04 LTS for x86-64 and ARM AArch64 architectures.

  3. Inference timeouts are implemented for all three inference types: cloud inferences, AI server inferences, and local inference. Now in case of inference hangs, disconnections, and other failures, the PySDK inference APIs will not hang indefinitely, but will raise inference timeout exceptions.

    To control the duration of the inference timeout, the inference_timeout_s property is added to the degirum.model.Model class. It specifies the maximum time in seconds to wait for the model inference result before rasing an exception.

    The default value for the inference_timeout_s depends on the AI hardware to be used for inferences. For inferences on AI accelerators (like ORCA) this timeout is set to 10 sec. For pure CPU inferences it is set to 100 sec.

  4. C++ SDK: new argument inference_timeout_ms is added to AIModel class. It specifies the maximum time in milliseconds to wait for inference result from the model inference on AI server.

  5. Error reporting is improved:

    • More meaningful error messages are now produced in case of cloud model loading failures.
    • Extended model name is added to all inference-related error messages.

Bug Fixes

  1. When a class label dictionary is updated for some model in some cloud zoo, and this model is then requested for an inference on some AI Server, which already performed an inference of that model some time ago, then the class label information reported by this AI server does not include recent changes made in the cloud zoo. This happens because the AI Server label dictionary cache is not properly updated.

  2. Model.EagerBatchSize parameter is now fixed to 8 for all cloud inferences to avoid scheduling favoritism for models with smaller batch size.

Version 0.6.0 (03/28/2023)

IMPORTANT: This release has changes in PySDK API

New Features and Modifications

  1. The function degirum.connect_model_zoo is deprecated and replaced with new function degirum.connect, which has more convenient interface.

    The degirum.connect_model_zoo function is still available in 0.6.0 release for backward compatibility but it is not recommended to use in new development.

  2. New function degirum.connect is introduced to simplify establishing model zoo connections. This is the main PySDK entry point: you start your work with PySDK by calling this function.

    The following use cases are supported: - You want to perform cloud inferences and take models from some cloud model zoo. - You want to perform inferences on some AI server and take models from some cloud model zoo. - You want to perform inferences on some AI server and take models from its local model zoo. - You want to perform inferences on local AI hardware and take models from some cloud model zoo. - You want to perform inferences on local AI hardware and use particular model from your local drive.

    Parameters:

    • inference_host_address: inference engine designator; it defines which inference engine to use.

      For AI Server-based inference it can be either the hostname or IP address of the AI Server host, optionally followed by the port number in the form :port.

      For DeGirum Cloud Platform-based inference it is the string "@cloud" or degirum.CLOUD constant.

      For local inference it is the string "@local" or degirum.LOCAL constant.

    • zoo_url: model zoo URL string which defines the model zoo to operate with.

      For a cloud model zoo, it is specified in the following format: <cloud server prefix>[/<zoo suffix>]. The <cloud server prefix> part is the cloud platform root URL, typically https://hub.degirum.com. The optional <zoo suffix> part is the cloud zoo URL suffix in the form <organization>/<model zoo name>. You can confirm zoo URL suffix by visiting your cloud user account and opening the model zoo management page. If <zoo suffix> is not specified, then DeGirum public model zoo degirum/public is used.

      For AI Server-based inferences, you may omit both zoo_url and token parameters. In this case locally-deployed model zoo of the AI Server will be used.

      For local AI hardware inferences, if you want to use particular AI model from your local drive, then you specify zoo_url parameter equal to the path to that model's .json configuration file. The token parameter is not needed in this case.

    • token: cloud API access token used to access the cloud zoo. To obtain this token you need to open a user account on DeGirum cloud platform. Please login to your account and go to the token generation page to generate an API access token.

    The function returns an instance of Model Zoo manager object configured to work with AI inference host and model zoo of your choice.

  3. InputNormEn and InputNormCoeff model parameters (properties of ModelParams class) are substituted with InputScaleEn and InputScaleCoeff respectively. This renaming is done to better reflect the semantic of these parameters: the input data is scaled with InputScaleCoeff coefficient, when InputScaleEn is set; then per-channel normalization is applied if InputNormMean and InputNormStd parameters are not empty, and these two actions are independent from one another. The previous names of these parameters InputNormEn and InputNormCoeff made confusion with per-channel normalization parameters InputNormMean and InputNormStd, which are independent from the input scaling.

    Old parameters InputNormEn and InputNormCoeff are deprecated but remain for backward compatibility.

  4. The diagnostics of incorrect input image formats has been improved in case when a numpy array with incorrect shape is supplied for the inference.

  5. The default public cloud model zoo URL suffix is changed from degirum_com/public to degirum/public.

Bug Fixes

  1. When PySDK inference methods are called concurrently from multiple threads the Python application may crash intermittently.

Version 0.5.2 (03/13/2023)

New Features and Modifications

  1. Hand palm detection models are initially supported in PySDK. Such models detect multiple palm key-points such as fingers. To process palm detection results the HandDetection post-processor is added to the PySDK.

    Each detected hand palm is represented by the following JSON object:

    JSON Field Name Description Data Type
    "score" Probability of detected hand floating point
    "handedness" Probability of right hand floating point
    "landmarks" hand description landmarks array of landmark JSON objects (see next table)

    The following is the structure of hand description landmark JSON object:

    JSON Field Name Description Data Type
    "label" Classified object class label string
    "category_id" Classified object class index integer
    "landmark" landmark point three-element JSON array [x, y, z]
    "world_landmark" metric world landmark point three-element JSON array [x, y, z]
    "connect" list of adjacent landmarks indices JSON array of integers

  2. Two new properties are added to the degirum.model.Model class:

    • eager_batch_size: it specifies the size of the batch to be used by the AI Server device scheduler when performing the inference of this model. The batch is the number of consecutive frames before this model is switched to another model during batch predict. This property accepts integer values in the range [1..80] range. The default value is 8.

    • frame_queue_depth: it specifies the depth of the model prediction queue. When the queue size reaches this value, the next prediction call will block until there will be a free space in the queue. This property accepts integer values in the range [1..160] range. The default value is 80 for cloud inferences and 8 for other cases.

    Using these two properties you may fine tune the inference latency and throughput.

    In order to reduce the inference latency you reduce the frame_queue_depth. The downside of reduced frame_queue_depth is the increased probability of empty model prediction queue, which degrades the inference throughput.

    If you simultaneously perform the inference of multiple models on the same AI Server, then to further reduce the inference latency you reduce the eager_batch_size. This causes AI Server scheduler to switch between different models more frequently. However, this degrades the inference throughput, since the model switching takes extra time.

    If you want to maximize the throughput for one particular model at the expense of other models, you increase the eager_batch_size value for that particular model. This causes AI Server scheduler to postpone switching to another model(s) until the number of frames equal to eager_batch_size value are processed by this model.

Bug Fixes

  1. When two (or more) AI Server clients simultaneously request the same model from the AI Server, and that model is already in the local cache, the first client takes it from the cache, while the second client always downloads it from the cloud.

  2. The validation of PySDK preprocessor-related model properties is broken in ver. 0.5.1. For example the input_pad_method property has the following set of possible values: "stretch", "letterbox", "crop-first", "crop-last". But it was possible to assign arbitrary string value to this property and get unexpected errors during inference.

  3. When a cloud model is loaded for inference on an AI Server and then this model is updated in the cloud zoo, then on the next inference request it is updated in the AI Server local cache (which is correct), but it is not updated in the AI accelerator cache and the outdated model is used for the inference.

  4. Image colors get inverted when running inference on OpenCV image of exact model size. Steps to reproduce:

    1. Use the model with RGB input colorspace.
    2. Read the image by OpenCV so it will be in BGR colorspace.
    3. Resize this image to the model input size.
    4. Perform the inference
    5. Show the overlay result: it will be with inverted colors

Version 0.5.1 (02/13/2023)

IMPORTANT: This release has new features in PySDK API

New Features and Modifications

  1. PySDK packages for Python 3.10 and 3.11 are now available for the following platforms:

    • Ubuntu Linux 20.04 LTS x86-64
    • Ubuntu Linux 20.04 LTS ARM AArch64
    • Windows 10/11 x86-64

  2. The image cropping feature is implemented.

    Two new choices are added for degirum.model.Model.input_pad_method property: "crop-first" and "crop-last".

    New property, degirum.model.Model.input_crop_percentage, is added to specify the percentage of image dimensions to crop around when "input_pad_method" is set to "crop-first" or "crop-last".

    When you select the "crop-first" method, the image is first cropped to match the AI model input tensor aspect ratio with respect to the degirum.model.Model.input_crop_percentage and then resized to match the AI model input tensor dimensions.

    For example: an input image with dimensions 640x480 going into a model with input tensor dimensions 224x224 with crop percentage of 0.875 will first be center cropped to 420x420 and then resized to 224x224.

    When you select "crop-last" method, if the AI model input tensor dimensions are equal, the image is resized with its smaller side equal to model dimension with respect to degirum.model.Model.input_crop_percentage. If the AI model input tensor dimensions are not equal (rectangle) the image is resized with stretching to the input tensor dimensions with respect to degirum.model.Model.input_crop_percentage. The image is then cropped to fit the AI model input tensor dimensions and aspect ratio.

    For example: an input image with dimensions 640x480 going into a model with input tensor dimensions 224x224 with crop percentage of 0.875 will first be resized to 341x256 and then center cropped to 224x224, alternatively an input image with dimensions 640x480 and a model with input tensor dimensions 280x224 will be resized to 320x256 (not preserving image aspect ratio) and then center cropped to 280x224.

  3. InputFmt and ModelFmt parameters of ModelInfo class are deprecated and superseded by InputTensorLayout parameter. In case when some older models do not define InputTensorLayout parameter, the software falls back to deprecated parameters in the following order: InputTensorLayout --> InputFmt --> ModelFmt.

  4. Checksum model JSON file parameter is now mandatory. Attempt to load a model without Checksum parameter now produces a runtime error.

  5. Logging functionality is implemented for PySDK methods and functions. When enabled, each call to every PySDK method or function is logged to the provided Python-style log handler. The logging.INFO logging level is used.

    The following example shows how to install a log handler of your choice and set a logging level:

    import logging

dglogger = logging.getLogger("degirum") # query 'degirum' logger
handler = logging.StreamHandler() # instantiate console printing handler 
# you can use whatever handler you see fit, for example FileHandler or SocketHandler
dglogger.addHandler(handler) # add the handler to the logger
dglogger.setLevel(logging.INFO) # set the logging level

    If you just want to enable PySDK logging to a console, you may use the degirum.enable_default_logger helper function. Essentially, it performs the code listed just above.

Bug Fixes

  1. In case of cloud inferences, frequent re-connections to the cloud server happen in case of slow Internet connection. This leads to slow performance of the cloud inference. The error was caused by websocket-client third-party library. Now PySDK requirements for websocket-client library are bumped to version 1.5.1, which has necessary fix.

  2. AI Server error handling is improved in case of sudden AI server disconnections. In previous versions the client software may hang in case of sudden AI server disconnection.

  3. AI Server error handling is improved in case of model loading failures such as request to load a model without checksum or when loading a model from a cloud zoo fails. In previous versions the client software may hang in case of model loading failures.

Version 0.5.0 (01/03/2023)

IMPORTANT: This release has changes in PySDK API

IMPORTANT: This release implements incompatible changes in AI Server protocol. If you work in client-server configuration and you upgraded the client-side PySDK, then you also need to upgrade the AI server software to version 0.5.0 and vice versa.

New Features and Modifications

  1. Access to cloud model zoos hosted on DeGirum Cloud Platform is supported in PySDK. Now you may use PySDK to download and request inferences of AI models hosted in the cloud.

    The new version of PySDK supports the following use cases: - cloud inference of a model in a cloud zoo; - AI server inference of a model in a cloud zoo; - AI server inference of a model in a local zoo; - local inference of a model in a cloud zoo; - local inference of a model in a local zoo.

    The particular use case is selected based on the URL string which you pass to degirum.connect_model_zoo function as the first argument. Please refer to the PySDK documentation for the detailed description of degirum.connect_model_zoo function arguments. Here we provide just short summary:

    Inference Type Model Zoo Type URL to Use
    Cloud inference Cloud zoo "dgcps://hub.degirum.com[/<zoo URL>]"
    AI server inference Cloud zoo ("<hostname>", "https://hub.degirum.com[/<zoo URL>]")
    AI server inference Local zoo "<hostname>"
    Local inference Cloud zoo "https://hub.degirum.com[/<zoo URL>]"
    Local inference Local file "/path/to/model.json"
    Local inference DeGirum Public cloud zoo ""

    Note: GitHub-hosted cloud model zoos are no longer supported.

    Please refer to the PySDK documentation for the detailed description of the cloud zoo operations.

  2. The following new attributes are added to the ModelParams class (as returned by degirum.model.Model.model_info method):

    Attribute Name Description Possible Values
    Pre-Processing-Related Attributes
    InputResizeMethod Interpolation algorithm used for image resizing during model training List of the following strings:
    "nearest", "bilinear", "area", "bicubic", "lanczos"
    InputPadMethod How input image is padded when resized during model training List of the following strings:
    "stretch", "letterbox"
    ImageBackend Graphical package used for image processing during model training List of the following strings:
    "pil", "opencv"

    These attributes may be specified in the model configuration JSON file to indicate how the training image set was pre-processed during model training.

    The values of these attributes are used as default values for corresponding degirum.model.Model properties: degirum.model.Model.resize_method, degirum.model.Model.pad_method, and degirum.model.Model.image_backend respectively.

  3. The AvailableDeviceMask attribute is deleted from the ModelParams class (as returned by degirum.model.Model.model_info method).

  4. Cloud inference connection and disconnection time is reduced by one second each. Now a single Model.predict() call takes less than one second, while in previous releases it took more than two seconds.

  5. All working/temporary files, which PySDK may create during operation are moved into DeGirum-specific application data directory:

    • For Linux it is $HOME/.local/share/DeGirum
    • For Windows it is %APPDATA%\DeGirum
    • For MacOS it is $HOME/Library/Application Support/DeGirum

Bug Fixes

  1. Semantic segmentation models do not work with cloud inference.

  2. degirum sys-info command incorrectly reports that Intel Myriad device is present in the system, when it is not.

Version 0.4.1 (11/16/2022)

New Features and Modifications

  1. Semantic segmentation models are now supported by PySDK.

    For segmentation models the overlay image accessible by InferenceResults.image_overlay property shows all detected segments, highlighted by some color (see more details for color assignments just below).

  2. The Model.overlay_color property now can also be specified as a list of RGB tuples. The overlay_color property is used to define the color to draw overlay details. In the case of a single RGB tuple, the corresponding color is used to draw all the overlay data: points, boxes, labels, segments, etc. In the case of a list of RGB tuples the behavior depends on the model type.

    • For classification models different colors from the list are used to draw labels of different classes.
    • For detection models different colors are used to draw labels and boxes of different classes.
    • For pose detection models different colors are used to draw keypoints of different persons.
    • For segmentation models different colors are used to highlight segments of different classes.

    If the list size is less than the number of classes of the model, then overlay_color values are used cyclically, for example, for three-element list it will be overlay_color[0], then overlay_color[1], overlay_color[2], and again overlay_color[0].

    The default value of overlay_color single RBG tuple of yellow color for all model types except segmentation models. For segmentation models it is the list of RGB tuples with the list size equal to the number of model classes. You can use Model.label_dictionary property to obtain a list of model classes. Each color is automatically assigned to look pretty and different from other colors in the list.

  3. More detailed error information is now reported in the case of cloud server inference failures. This information now includes details reported by the inference stack instead of generic HTTP errors.

  4. PCI device information is now reported by degirum sys-info command for Orca accelerator cards. This information is listed in Device Description entry and includes PCI bus index and device index on that bus.

Version 0.4.0 (09/19/2022)

IMPORTANT: This release has changes in PySDK API

IMPORTANT: This release implements backward-compatible extensions in AI Server protocol. If you work in client-server configuration and you upgraded the client-side PySDK, then you also need to upgrade the AI server software to version 0.4.0.

New Features and Modifications

  1. PySDK command-line interface (CLI) has been implemented. Using PySDK CLI you can:

    • download AI models from the model zoo repo;
    • start and stop AI server;
    • command AI server to rescan local model zoo;
    • obtain system information dump either from local or remote AI system;
    • manage AI server tracing facility.

    The PySDK CLI is implemented as the degirum executable console script. As a part of PySDK installation it is installed into the system path. This console script extends PySDK functionality through the mechanism of entry points.

    The PySDK CLI supports the following commands:

    Command Description
    download-zoo Command to download AI models from the model zoo GitHub repo
    server Command to control the operation of AI server
    sys-info Command to obtain system information dump
    trace Command to manage AI server tracing facility

    You invoke the console script passing the command from the table above as its first argument followed by command-specific parameters described in the following sections.

    Please refer to the PySDK documentation for the detailed description of all CLI commands and their parameters.

  2. The possibility to pass arbitrary frame information objects along with the frame data is implemented for the Model.predict_batch() method. Recall that the Model.predict_batch() method accepts an iterator object which returns the frame data, one frame at a time. Now this iterator object may also return tuples, where the first element of such tuple is traditional frame data and the second element is an object which contains frame information of your choice. When the inference of a frame is complete, the frame information object is stored in the corresponding inference result object: you can access it using the info property of the inference result object. The frame information object can be of any type you want.

    This feature is very handy when you need to associate inference results with particular frame attributes. For example, you have labeled data set with ground truth data for each frame, and you need to access this ground truth data in the same place where you have the inference results. The following code demonstrates how to do it:

    # connect to model zoo and load model as usual

# define iterator function to iterate over frames
def source():
    frame_data = ... # here should be the code to obtain frame data
    ground_truth = ... # here should be the code to obtain ground truth
    yield (frame_data, ground_truth) # return a tuple of frame data and frame info

# perform batch prediction
for result in model.predict_batch(source()):
    print(result.results) # here is your inference result
    print(result.info) # here is your ground truth data for that result

Version 0.3.5 (08/31/2022)

New Features and Modifications

  1. Support for floating-point models has been added to the audio pre-processor.

  2. PySDK OpenCV image preprocessor is optimized to improve timing performance.

  3. PySDK documentation and docstrings are updated: missing information has been added, some sections were rewritten for better clarity.

Version 0.3.4 (08/15/2022)

New Features and Modifications

C++ SDK documentation has been updated to reflect recent changes.

Bug Fixes

  1. An attempt to load non-existent model using AI client-server API causes client application crash.

  2. Object detection models may give slightly different bounding box results based on which OS is used to run the inference.

Version 0.3.3 (08/10/2022)

Bug Fixes

  1. ML inference performed on DeGirum Cloud Platform may hang intermittently; more often when running long batch predicts on slow Internet connections.

  2. MSVC compiler warnings are fixed for C++ SDK.

Version 0.3.2 (08/03/2022)

New Features and Modifications

  1. Retries are implemented in PySDK when attempting to connect to DeGirum Cloud Platform servers. This improves robustness in case of poor Internet connection.

  2. When working with DeGirum Cloud Platform, the Model object reconnects to the cloud server at the beginning of each predict() and predict_batch() method call and disconnects at the end. This behavior is implemented to avoid keeping open connections to the cloud server for indefinite time. On the other hand it greatly decreases performance of single-frame or small-batch inferences. In order to improve single-frame performance the Model class now implements context manager interface. On __enter__() it connects to the cloud server and keeps connection open until __exit__() is called. The following code snippet demonstrates this approach: 

     import degirum as dg

 zoo = dg.connect_model_zoo("dgcps://hub.degirum.com") # connect to cloud server
 model = zoo.load_model(zoo.list_models()[0]) # load a model
 with model: # key point: use model object as context manager in `with` statement
     # at this point connection to the server server is open 
     # and will be open for all consecutive inferences within `with` statement
     res1 = model.predict("https://degirum.github.io/images/samples/TwoCats.jpg")
     res2 = model.predict("https://degirum.github.io/images/samples/Car.jpg")
     # and so on...
 # and only at this point connection will be closed

  3. When performing long batch prediction using Model.predict_batch() method, the last portion of pipelined inference results was delivered at once. The previous implementation worked the following way: once the last frame is sent for prediction it just waited for completion of all posted frames and only then starts yielding results. This created a long delay before the last portion of results is delivered. The new implementation continues keeping track of ready results and continues delivering them in a timely manner even after the last frame is sent for prediction. This guarantees smooth delivery of results without any time gaps.

Bug Fixes

  1. ML inference performed on DeGirum Cloud Platform may hang intermittently.

  2. ML inference performed on DeGirum Cloud Platform may intermittently raise the following exceptions:

  3. "DegirumException: Connection to cloud server unexpectedly closed"
  4. "DegirumException: Unable to open connection to cloud server hub.degirum.com: Operation aborted."
  5. "TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond"
  6. "DegirumException: Unable to access server hub.degirum.com: HTTPSConnectionPool(host='hub.degirum.com', port=443): Read timed out. (read timeout=5)"

They appear more often in case of poor Internet connection.

  1. Rendering of overlay image fails for object detection models when Model.measure_time is set to True, and no objects are detected.

Version 0.3.1 (07/20/2022)

IMPORTANT: This release has critical bug fixes. It is strongly recommended to upgrade your PySDK from version 0.3.0 to version 0.3.1.

Critical Bug Fixes

  1. DeGirum Cloud Platform communication protocol has critical flaw. When an open connection to the cloud server gets interrupted for whatever reason, the client starts sending a stream of reconnection requests flooding the cloud server. When multiple clients start exhibiting such behavior, the cloud server may become not responsive.

  2. When connection to the cloud server gets interrupted during inference, the client script may hang with infinite timeout. Now the following error message will be raised in case of connection failures: "Connection to cloud server unexpectedly closed".

  3. When performing batch inference on the cloud server, the following error may occur intermittently: "Result frame numbers mismatch: expected <N>, processed <M>"", where N and M are some integers numbers.

Other Bug Fixes

  1. When image URL provide for inference is redirection URL (not direct URL to an image file), then PySDK does not follow redirection, and downloads redirection HTML instead. This problem affects, for example, all images from Wikipedia.

  2. Changing parameter(s) of one cloud model object affects all newly created model objects. This bug is similar to the bug fixed in the release 0.2.1 (see below), but affects only cloud models supported in release 0.3.0.

  3. Zoo accessor list_model() method does not support filtering for OpenVINO runtime and Intel Myriad device.

  4. An attempt to send graphical data to audio pre-processor causes not very informative error message. Now the error message says: "Input data format passed for the audio model input #0 is not supported: expected 1-D numpy tensor containing audio waveform of 15600 samples"

New Features and Modifications

  1. Set input image type to "JPEG" when the cloud server model is used to reduce network traffic and improve performance. For all other models "RAW" input image type is set by default to reduce computation load on both client and server to avoid JPEG encoding and decoding.

  2. The following packages are added to the list of PySDK requirements:

    • opencv-python
    • python-dotenv
    • pyaudio (only for Windows platform)

  3. To avoid compatibility problems minimum versions are now provided for all PySDK requirements.

Version 0.3.0 (07/14/2022)

IMPORTANT: This release has changes in PySDK API

IMPORTANT: This release has changes in AI Server protocol, which make it incompatible with previous PySDK versions. If you work in client-server configuration you need to simultaneously upgrade both client and AI server software to version 0.3.0.

New Features and Modifications

  1. The DeGirum Cloud Platform is initially supported in PySDK.

    DeGirum Cloud Platform solves the Edge AI development problem by providing the toolchain to design, run, and evaluate ML models across different hardware platforms in the cloud: - no hardware to buy, - no drivers to install, - no worries on software dependencies, - no getting locked to one particular hardware.

    DeGirum Cloud Platform includes the following components: - DeGirum Cloud Device Farm accesible through Cloud Application Server and PySDK - DeGirum DeGirum Cloud Portal Management site, hub.degirum.com

    The DeGirum Cloud Farm is a set of computing nodes with various AI hardware accelerators installed in those nodes, including: - DeGirum Orca - Google Edge TPU - Intel® Movidius™ Myriad™ VPU

    The farm nodes are hosted by DeGirum.

    The DeGirum Cloud Application Server provides web API to perform AI inference on the Cloud Farm devices. Starting from ver. 0.3.0 PySDK supports Application Server web API and provides the same level of functionality transparently to the end user: you may run exactly the same code on the Cloud Platform as it was designed for traditional use cases such as local inference or AI server inference.

    The DeGirum Cloud Portal Management site provides GUI to manage web API tokens, which are required to access Cloud Application Server through PySDK.

    Please visit DeGirum Developers Center for the quick start guide on how to use PySDK with DeGirum Cloud Platform.

  2. Intel® OpenVINO™ runtime is initially supported by DeGirum software stack. It opens possibilities to run OpenVINO™ models in PySDK. For models compiled for Intel® Movidius™ Myriad™ VPU you need to install that VPU on your system which runs DeGirum AI server. Models for OpenVINO™ runtime have openvino suffix in the model name. Models compiled for Myriad™ VPU have myriad suffix in the model name, for example: mobilenet_v2_imagenet--224x224_float_openvino_myriad_1.

  3. AI Server protocol has been updated to revision 2. If you work in client-server configuration you need to simultaneously upgrade both client and AI server software to version 0.3.0. Otherwise you may get the following error message when you try to connect to older AI server with newer 0.3.0 client: Protocol version '2' is out of supported version range [1, 1]. Please upgrade your client software to the most recent version. Or vice versa: Protocol version '1' is out of supported version range [2, 2]. Please upgrade your client software to the most recent version.

  4. When inference time collection is enabled (degirum.model.Model.measure_time property is assigned to True), then additional statistic items are reported for DeGirum Orca AI accelerator hardware:

    Key Description
    DeviceInferenceDuration_ms Duration of AI inference computations on AI accelerator IC excluding data transfers
    DeviceTemperature_C Internal temperature of AI accelerator IC in C
    DeviceFrequency_MHz Working frequency of AI accelerator IC in MHz

Bug Fixes

  1. When Python installation which is used to run PySDK has PIL library version less than 9.1.0, then following error happened when you try to use PIL graphical backend: AttributeError: module 'PIL.Image' has no attribute 'Resampling'. Now PySDK requirements include correct PIL library version of 9.1.0 and above.

Version 0.2.1 (06/23/2022)

IMPORTANT: This release has critical bug fixes.

New Features and Modifications

  1. License plate OCR detection model returned empty result when no license plates detected. Now it returns single result with empty label and zero score.

Critical Bug Fixes

  1. When multiple concurrent AI inferences of YOLOv5-based detection models are running on a single instance of AI server, incorrect inference results may be reported intermittently.

  2. Changing parameter(s) of one model object affects all newly created model objects. For example, consider the following code:

    import degirum as dg

zoo = dg.connect_model_zoo()
name = zoo.list_models()[0]

m1 = zoo.load_model(name)
m1.output_postprocess_type = 'None' # change some parameter

m2 = zoo.load_model(name) # create another model object of the same model

assert m2.output_postprocess_type != 'None'
# this assertion fails

    Setting output_postprocess_type for the first model affects output_postprocess_type of all models created after that.

Other Bug Fixes

  1. An attempt to read the degirum.model.Model.label_dictionary property of a model, which does not have the label dictionary, leads to runtime exception 'NoneType' object has no attribute 'items'.

  2. An attempt to run inference on a model after assigning non-existing device index to degirum.model.Model.devices_selected property causes AI server crash.

Version 0.2.0 (05/23/2022)

IMPORTANT: This release has changes in PySDK API.

New Features and Modifications

  1. PySDK for macOS 12 / x86-64 / Python 3.9 and macOS 12 / AArch64 / Python 3.9 is officially supported.

  2. PySDK for Ubuntu 20.04 / AArch64 / Python 3.9 is officially supported.

  3. Multi-input models are initially supported.

    As a result, all properties of ModelInfo class prefixed with Input prefix changed their type from scalar values to lists. The list size of each such property is equal to the number of model inputs.

    The following new properties are added to the ModelInfo class:

    Property Name Description Possible Values
    InputN Input frame dimension size 1
    Other sizes to be supported
    InputH Input height dimension size Integer number
    InputW Input width dimension size Integer number
    InputC Input color dimension size Integer number
    InputQuantEn Enable input frame quantization flag (set for quantized models) Boolean value
    InputQuantOffset Quantization offset for input image quantization List of float values
    InputQuantScale Quantization scale for input image quantization List of float values

    The following propertied of the ModelInfo class become deprecated, since they are superseded by complementary per-input properties, listed in the table above.

    Property Name Description Possible Values
    ModelInputN Model frame dimension size 1
    Other sizes to be supported
    ModelInputH Model height dimension size Integer number
    ModelInputW Model width dimension size Integer number
    ModelInputC Model color dimension size Integer number
    ModelQuantEn Enable input frame quantization flag (set for quantized models) Boolean value
    InputImgQuantOffset Quantization offset for input image quantization Float value
    InputImgQuantScale Quantization scale for input image quantization Float value

  4. Audio classification models are initially supported.

    For audio models, ModelInfo.InputType property returns Audio input type value.

    As a result, the following new properties are added to the ModelInfo class:

    Property Name Description Possible Values
    InputWaveformSize Input waveform size in samples for audio input types List of positive integer values
    InputSamplingRate Input waveform sampling rate in Hz for audio input types List of positive float values

    When dealing with model inputs of audio type (InputType is equal to "Audio"), PySDK does not perform any conversions of the input data: it expects numpy 1-D array with audio waveform samples of proper size and with proper sampling rate. The waveform size should be equal to InputWaveformSize model info property. The waveform sampling rate should be equal to InputSamplingRate model info property. And finally the data element type should be equal to the data type specified by the InputRawDataType model info property.

  5. Non-blocking mode of batch predict is implemented. You turn on this mode by assigning True to new degirum.model.Model.non_blocking_batch_predict property. When non-blocking mode is enabled, the generator object returned by predict_batch() method accepts None from the input iterable object. This allows you to design non-blocking frame data source iterators: when no data is available, such iterator just yields None without waiting for the next frame. If None is returned from the input iterator, the model predict step is simply skipped for this iteration. Also in non-blocking mode when no inference results are available in the result queue at some iteration, the generator yields None result. This allows to continue execution of the code which operates with another model. In order to operate in non-blocking mode you need to modify your code the following way:

    • Modify frame data source iterator to return None if no frame is available yet, instead of waiting for the next frame.
    • Modify inference loop body to deal with None results by simply skipping them.

  6. Inference device selection is supported. To get the information about available devices you query degirum.model.Model.devices_available property of your model object. It returns the list of zero-based device indices of all available devices of the type this model is designed for. In certain cases you may want to limit the model inference to particular subset of available devices. To do so you need to assign degirum.model.Model.devices_selected property of your model object to contain the list of device indices you want your model to run on.

  7. degirum.model.Model.input_numpy_colorspace property default value is set to RGB for all graphical backends.

Bug Fixes

  1. degirum.zoo_manager.ZooManager.list_models method ignores pruned parameter.

  2. Any run-time error which happens during JPEG image decoding causes quiet process termination.

  3. Any change to degirum.model.Model.output_pose_threshold property is ignored in case when AI server models are used (it works OK when cloud or local models are used).

  4. License Plate OCR model returns result with empty label and score value equal to 1 in case of no results.

Version 0.1.0 (04/19/2022)

IMPORTANT: This release has critical bug fixes.

IMPORTANT: This release has changes in PySDK API.

New Features and Modifications

  1. Python version 3.9 is supported.

  2. The model filtering is introduced for cloud model zoo connections. Now the list of models available for local inference include only models supported by your local hardware configuration. For example, if the local host does not have DeGirum Orca AI accelerator hardware installed, all AI models designed for Orca AI accelerator will not be listed.

  3. Communication protocol version compatibility check is introduced for DeGirum AI server. Now if you try to connect new PySDK client software to older version of DeGirum AI server you will get an error: "AI server: Protocol version 'xx' is out of supported version range [yy,zz]. Please upgrade your client software to the most recent version.".

  4. Access to the model label dictionary is introduced. New method degirum.model.Model.label_dictionary is added to access the model label dictionary. It returns a dictionary where each element is a key-value pair, where the key is the class ID number and the value is the class label string.

  5. New property degirum.model.Model.output_pose_threshold is added. Use this property to control aggregated pose confidence threshold for pose detection models (as opposed to the individual body parts confidence threshold specified by degirum.model.Model.output_confidence_threshold).

  6. New non-positional arguments are added to degirum.server.download_models method. Using these new arguments you may download only specific models from the clound model zoo. The set of new arguments repeats the arguments of degirum.zoo_manager.ZooManager.list_models method:

    • model_family: model family name filter. When you pass a string, it will be used as search substring in the model name. For example, "yolo", "mobilenet". You may also pass re.Pattern object: in this case it will do regular expression pattern search.

    • device: target inference device - string or list of strings of device names. Possible names: "orca", "cpu", "edgetpu"

    • precision: model calculation precision - string or list of strings of model precision labels. Possible labels: "quant", "float"

    • pruned: model density - string or list of strings of model density labels. Possible labels: "dense", "pruned"

    • runtime: runtime agent type - string or list of strings of runtime agent types: Possible types: "n2x", "tflite"

  7. PySDK inference timing collection API is introduced. It includes the following new features:

    • degirum.model.Model.measure_time property. Set it to True to enable inference timing collection. Default value is False. When inference timing collection is enabled, the durations of individual steps for each frame prediction are accumulated in internal statistic accumulators.

    • degirum.model.Model.reset_time_stats method. Call it to reset internal statistic accumulators.

    • degirum.model.Model.time_stats method. When measure_time property is set to True, the time_stats method returns a dictionary with time statistic objects. Each time statistic object accumulates time statistics for particular inference step over all frame predictions happened since the timing collection was enabled or reset. The statistics includes minimum, maximum, average, and count. Inference steps correspond to dictionary keys. The following dictionary keys are supported:

      Key Description
      "FrameTotalDuration_ms" Frame total inference duration
      "PythonPreprocessDuration_ms" Duration of Python pre-processing step
      "CorePreprocessDuration_ms" Duration of low-level pre-processing step
      "CoreInferenceDuration_ms" Duration of actual AI inference step
      "CoreLoadResultDuration_ms" Duration of data movement step
      "CorePostprocessDuration_ms" Duration of low-level post-processing step

  8. degirum.zoo_manager.ZooManager.rescan_zoo() method is removed from PySDK API.

Critical Bug Fixes

  1. When a model is renamed or removed from a cloud zoo an attempt to connect to such model zoo by calling degirum.connect_model_zoo() produces non-recoverable error KeyError.

  2. Establishing connection to DeGirum AI server specified by host name (as opposed to IP address) takes 2 sec. on Windows OS.

Version 0.0.1 (03/24/2022)

IMPORTANT: This is the first official alpha release of DeGirum PySDK.