Calibration module¶
- calibrate(model, dataset, method='MinMax', batches_count_for_calibration=128)¶
Calibrates
model
using data indataset
.dataset
should be an iterable object which will feed the model with samples during the calibration process. The result of the calibration process is a calibration table. This table is used inOrtTensorrtInt8Backend
backend to implement 8-bit integers quantization and speed up inference. It supports 2 calibration methods:MinMax
andEntropy
. Now, we support calibration only for image-like datasets and require all samples in yourdataset
to implementshape
method.- Parameters
model (filename or serialized
ONNX
format model in a byte string.) – Model for calibration.dataset (Iterable) – Iterable object with samples which will be used for calibration. Samples should implement
shape
method. For example, it can be aPytorch Dataloader
object.method (str) – Calibration algorithm:
MinMax
orEntropy
. Default algorithm isMinMax
.batches_count_for_calibration (int) – Number of batches which will be used for calibration.
- Returns
Table with calibrated parameters which will be used for inference with int8 quantization. Pass it to OrtTensorrtInt8Backend or export to
FlatBuffers
(CalibrationTableTensorrt.to_flatbuffers/to_file_flatbuffers
) to save it for later use.- Return type
Examples
>>> from enot_lite.backend import OrtTensorrtInt8Backend >>> from enot_lite.calibration import calibrate >>> calibration_table = calibrate('model.onnx', iterable_dataset) >>> backend = OrtTensorrtInt8Backend('model.onnx', calibration_table) >>> backend.run(sample)
- class CalibrationTableTensorrt(table)¶
Representation of
TensorRT
calibration table.The table consists of two columns:
name
andthreshold
. Calibration table can be created from raw dict by ctor, or loaded from file with the help of the following functions:from_file_json
from_file_flatbuffers
from_file_tensorrt
Also, you can export calibration table to file or to bytearray:
to_file_json
to_json
to_file_flatbuffers
to_flatbuffers
to_file_tensorrt
Examples
Creation of calibration table from a raw dict:
>>> raw_table = { ... 'name_0': 0.1, ... 'name_1': 0.2, ... } >>> table = CalibrationTableTensorrt(raw_table)
Export calibration table to
FlatBuffers
file:>>> table.to_file_flatbuffers('table.flatbuffers')
Create calibration table from
FlatBuffers
file:>>> table = CalibrationTableTensorrt.from_file_flatbuffers('table.flatbuffers')
Pretty print calibration table (requires
tabulate
package):>>> print(table)
- __init__(table)¶
- classmethod from_calibration_cache(calibration_cache)¶
Creates CalibrationTableTensorrt from calibration cache.
- Return type
- classmethod from_file_flatbuffers(filename)¶
Loads CalibrationTableTensorrt from
FlatBuffers
file.- Parameters
filename (Union[str, Path]) – Name of
FlatBuffers
file with names and threshold values.- Returns
- Return type
- Raises
RuntimeError: – When it is not possible to restore calibration table from file.
- classmethod from_file_json(filename)¶
Loads CalibrationTableTensorrt from
JSON
file.- Parameters
filename (Union[str, Path]) – Name of
JSON
file with names and threshold values.- Returns
- Return type
- Raises
RuntimeError: – When it is not possible to restore calibration table from file.
- classmethod from_file_tensorrt(filename)¶
Creates CalibrationTableTensorrt from native
TensortRT
calibration table format.- Parameters
filename (str or Path) – Name of
TensortRT
calibration file.- Returns
- Return type
- property table: Dict[str, float]¶
Returns inner representation (raw dict) of CalibrationTableTensorrt.
- to_file_flatbuffers(filename)¶
Saves CalibrationTableTensorrt to file in
FlatBuffers
format.
- to_file_json(filename)¶
Saves CalibrationTableTensorrt to file in
JSON
format.
- to_file_tensorrt(filename, header='TRT-8000-EntropyCalibration2')¶
Saves CalibrationTableTensorrt to file in native
TensorRT
format.
- to_flatbuffers()¶
Converts CalibrationTableTensorrt to
FlatBuffers
representation.- Returns
Binary representation of CalibrationTableTensorrt.
- Return type