Calibration module

calibrate(model, dataset, method='MinMax', batches_count_for_calibration=128)

Calibrates model using data in dataset.

dataset should be an iterable object which will feed the model with samples during the calibration process. The result of the calibration process is a calibration table. This table is used in TensorRT to implement 8-bit integers quantization and speed up inference. It supports 2 calibration methods: MinMax and Entropy. Now, we support calibration only for image-like datasets and require all samples in your dataset to implement shape method.

Parameters

  • model (filename or serialized ONNX format model in a byte string.) – Model for calibration.

  • dataset (Iterable) – Iterable object with samples which will be used for calibration. Samples should implement shape method. For example, it can be a Pytorch Dataloader object.

  • method (str) – Calibration algorithm: MinMax or Entropy. Default algorithm is MinMax.

  • batches_count_for_calibration (int) – Number of batches which will be used for calibration.

Returns

Table with calibrated parameters which will be used for inference with int8 quantization. Pass it to backend factory with TensorRT backend type or export to FlatBuffers (CalibrationTableTensorrt.to_flatbuffers/to_file_flatbuffers) to save it for later use.

Return type

CalibrationTableTensorrt

Examples

>>> from enot_lite.backend import BackendFactory
>>> from enot_lite.type import BackendType
>>> from enot_lite.calibration import calibrate
>>> calibration_table = calibrate('model.onnx', iterable_dataset)
>>> backend = BackendFactory().create('model.onnx', BackendType.ORT_TENSORRT, calibration_table=calibration_table)
>>> backend.run(sample)
class CalibrationTableTensorrt(table)

Representation of TensorRT calibration table.

The table consists of two columns: name and threshold. Calibration table can be created from raw dict by ctor, or loaded from file with the help of the following functions:

  • from_file_json

  • from_file_flatbuffers

  • from_file_tensorrt

Also, you can export calibration table to file or to bytearray:

  • to_file_json

  • to_json

  • to_file_flatbuffers

  • to_flatbuffers

  • to_file_tensorrt

Examples

Creation of calibration table from a raw dict:

>>> raw_table = {
...     'name_0': 0.1,
...     'name_1': 0.2,
... }
>>> table = CalibrationTableTensorrt(raw_table)

Export calibration table to FlatBuffers file:

>>> table.to_file_flatbuffers('table.flatbuffers')

Create calibration table from FlatBuffers file:

>>> table = CalibrationTableTensorrt.from_file_flatbuffers('table.flatbuffers')

Pretty print calibration table (requires tabulate package):

>>> print(table)
Parameters

table (Dict[str, float]) –

__init__(table)
Parameters

table (Dict[str, float]) – Table in format name : value, where value is a threshold value.

classmethod from_calibration_cache(calibration_cache)

Creates CalibrationTableTensorrt from calibration cache.

Return type

CalibrationTableTensorrt

classmethod from_file_flatbuffers(filename)

Loads CalibrationTableTensorrt from FlatBuffers file.

Parameters

filename (Union[str, Path]) – Name of FlatBuffers file with names and threshold values.

Returns

Return type

CalibrationTableTensorrt

Raises

RuntimeError: – When it is not possible to restore calibration table from file.

classmethod from_file_json(filename)

Loads CalibrationTableTensorrt from JSON file.

Parameters

filename (Union[str, Path]) – Name of JSON file with names and threshold values.

Returns

Return type

CalibrationTableTensorrt

Raises

RuntimeError: – When it is not possible to restore calibration table from file.

classmethod from_file_tensorrt(filename)

Creates CalibrationTableTensorrt from native TensortRT calibration table format.

Parameters

filename (str or Path) – Name of TensortRT calibration file.

Returns

Return type

CalibrationTableTensorrt

property table: Dict[str, float]

Returns inner representation (raw dict) of CalibrationTableTensorrt.

Return type

Dict[str, float]

to_file_flatbuffers(filename)

Saves CalibrationTableTensorrt to file in FlatBuffers format.

Parameters

filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.

Return type

None

to_file_json(filename)

Saves CalibrationTableTensorrt to file in JSON format.

Parameters

filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.

Return type

None

to_file_tensorrt(filename, header='TRT-8000-EntropyCalibration2')

Saves CalibrationTableTensorrt to file in native TensorRT format.

Parameters

  • filename (Union[str, Path]) – Filename of path to file, where CalibrationTableTensorrt should be stored.

  • header (str) – Header of native TensorRT table.

Return type

None

to_flatbuffers()

Converts CalibrationTableTensorrt to FlatBuffers representation.

Returns

Binary representation of CalibrationTableTensorrt.

Return type

bytearray

to_json()

Converts CalibrationTableTensorrt to JSON format.

Returns

JSON representation of CalibrationTableTensorrt.

Return type

str