Calibration module¶

calibrate(model, dataset, method='MinMax', batches_count_for_calibration=128)¶

Calibrates model using data in dataset.

dataset should be an iterable object which will feed the model with samples during the calibration process. The result of the calibration process is a calibration table. This table is used in OrtTensorrtInt8Backend backend to implement 8-bit integers quantization and speed up inference. It supports 2 calibration methods: MinMax and Entropy. Now, we support calibration only for image-like datasets and require all samples in your dataset to implement shape method.

Parameters

model (filename or serialized ONNX or ORT format model in a byte string.) – Model for calibration.
dataset (Iterable) – Iterable object with samples which will be used for calibration. Samples should implement shape method. For example, it can be a Pytorch Dataloader object.
method (str) – Calibration algorithm: MinMax or Entropy. Default algorithm is MinMax.
batches_count_for_calibration (int) – Number of batches which will be used for calibration.

Returns

Table with calibrated parameters which will be used for inference with int8 quantization. Pass it to OrtTensorrtInt8Backend or export to FlatBuffers (CalibrationTableTensorrt.to_flatbuffers/to_file_flatbuffers) to save it for later use.

Return type

CalibrationTableTensorrt

Examples

>>> from enot_lite.backend import OrtTensorrtInt8Backend
>>> from enot_lite.calibration import calibrate
>>> calibration_table = calibrate('model.onnx', iterable_dataset)
>>> backend = OrtTensorrtInt8Backend('model.onnx', calibration_table)
>>> input_name = backend.get_inputs()[0].name
>>> backend.run(None, {input_name: sample})

class CalibrationTableTensorrt(table)¶

Representation of TensorRT calibration table.

The table consists of two columns: name and threshold. Calibration table can be created from raw dict by ctor, or loaded from file with the help of the following functions:

from_file_json

from_file_flatbuffers

from_file_tensorrt

Also, you can export calibration table to file or to bytearray:

to_file_json

to_json

to_file_flatbuffers

to_flatbuffers

to_file_tensorrt

Examples

Creation of calibration table from a raw dict:

>>> raw_table = {
...     'name_0': 0.1,
...     'name_1': 0.2,
... }
>>> table = CalibrationTableTensorrt(raw_table)

Export calibration table to FlatBuffers file:

>>> table.to_file_flatbuffers('table.flatbuffers')

Create calibration table from FlatBuffers file:

>>> table = CalibrationTableTensorrt.from_file_flatbuffers('table.flatbuffers')

Pretty print calibration table (requires tabulate package):

>>> print(table)

Parameters: table (Dict[str, float]) –

__init__(table)¶

Parameters: table (Dict[str, float]) – Table in format name : value, where value is a threshold value.

classmethod from_calibration_cache(calibration_cache)¶

Creates CalibrationTableTensorrt from calibration cache.

Return type: CalibrationTableTensorrt

classmethod from_file_flatbuffers(filename)¶

Loads CalibrationTableTensorrt from FlatBuffers file.

Parameters: filename (Union[str, Path]) – Name of FlatBuffers file with names and threshold values.
Returns
Return type: CalibrationTableTensorrt
Raises: RuntimeError: – When it is not possible to restore calibration table from file.

classmethod from_file_json(filename)¶

Loads CalibrationTableTensorrt from JSON file.

Parameters: filename (Union[str, Path]) – Name of JSON file with names and threshold values.
Returns
Return type: CalibrationTableTensorrt
Raises: RuntimeError: – When it is not possible to restore calibration table from file.

classmethod from_file_tensorrt(filename)¶

Creates CalibrationTableTensorrt from native TensortRT calibration table format.

Parameters: filename (str or Path) – Name of TensortRT calibration file.
Returns
Return type: CalibrationTableTensorrt

property table: Dict[str, float]¶

Returns inner representation (raw dict) of CalibrationTableTensorrt.

Return type: Dict[str, float]

to_file_flatbuffers(filename)¶

Saves CalibrationTableTensorrt to file in FlatBuffers format.

Parameters: filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.
Return type: None

to_file_json(filename)¶

Saves CalibrationTableTensorrt to file in JSON format.

Parameters: filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.
Return type: None

to_file_tensorrt(filename, header='TRT-7000-ENOT-Lite-Calibration')¶

Saves CalibrationTableTensorrt to file in native TensorRT format.

Parameters

filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.
header (str) – Header of native TensorRT table.

Return type

None

to_flatbuffers()¶

Converts CalibrationTableTensorrt to FlatBuffers representation.

Returns: Binary representation of CalibrationTableTensorrt.
Return type: bytearray

to_json()¶

Converts CalibrationTableTensorrt to JSON format.

Returns: JSON representation of CalibrationTableTensorrt.
Return type: str