from onnxruntime import SessionOptions

from onnxtr.models import ocr_predictor, EngineConfig

general_options = SessionOptions()  # For configuartion options see: https://onnxruntime.ai/docs/api/python/api_summary.html#sessionoptions
general_options.enable_cpu_mem_arena = False

# NOTE: The following would force to run only on the GPU if no GPU is available it will raise an error
# List of strings e.g. ["CUDAExecutionProvider", "CPUExecutionProvider"] or a list of tuples with the provider and its options e.g.
# [("CUDAExecutionProvider", {"device_id": 0}), ("CPUExecutionProvider", {"arena_extend_strategy": "kSameAsRequested"})]
providers = [("CUDAExecutionProvider", {"device_id": 0})]  # For available providers see: https://onnxruntime.ai/docs/execution-providers/

engine_config = EngineConfig(
    session_options=general_options,
    providers=providers
)
# We use the default predictor with the custom engine configuration
# NOTE: You can define different engine configurations for detection, recognition and classification depending on your needs
predictor = ocr_predictor(
    det_engine_cfg=engine_config,
    reco_engine_cfg=engine_config,
    clf_engine_cfg=engine_config
)

Full Changelog: v0.2.0...v0.3.0

Assets 2

13 May 10:40

felixdittrich92

v0.2.0

162a10d

v0.2.0

What's Changed

Added 8-Bit quantized models
Added Dockerfile and CI for CPU/GPU Usage

8-Bit quantized models

8-Bit quantized variants of all models was added (expect: the FAST models - which are already reparameterized)

from onnxtr.models import ocr_predictor, detection_predictor, recognition_predictor

predictor = ocr_predictor(det_arch="db_resnet50", reco_arch="crnn_vgg16_bn", load_in_8_bit=True)

det_predictor = detection_predictor("db_resnet50", load_in_8_bit=True)
reco_predictor = recognition_predictor("parseq", load_in_8_bit=True)

CPU benchmarks:

Library	FUNSD (199 pages)	CORD (900 pages)
docTR (CPU) - v0.8.1	~1.29s / Page	~0.60s / Page
OnnxTR (CPU) - v0.1.2	~0.57s / Page	~0.25s / Page
OnnxTR (CPU) 8-bit - v0.1.2	~0.38s / Page	~0.14s / Page
EasyOCR (CPU) - v1.7.1	~1.96s / Page	~1.75s / Page
PyTesseract (CPU) - v0.3.10	~0.50s / Page	~0.52s / Page
Surya (line) (CPU) - v0.4.4	~48.76s / Page	~35.49s / Page