Graph optimization removes significant casts even if all optimizations are disabled

[cbourjau]

Describe the issue

Consider the following model

where a is a float64 input, the first cast is to float32 and the second one is back to float64. The model is stored in original_model.onnx below.

Running

    import onnxruntime as ort

    opts = ort.SessionOptions()
    opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
    opts.optimized_model_filepath = "model_disable_all_optimizations.onnx"

    ort.InferenceSession("original_model.onnx", opts)

yields the following model:

The cast are significant. Removing them changes the output of the model. Furthermore, I would expect no optimizations at all to take place if opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL.

To reproduce

The respective models are attached.

models.zip

Urgency

The (disabled) optimization produces wrong results in surprising ways. Users may rely on the truncations from the cast. We found this bug because the lack of these casts produced wrong results in a subsequent TreeEnsembleRegressor.

Platform

Mac

OS Version

12.3.1

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.1

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[wangyems]

there is a way to disable certain optimizers: e.g

onnxruntime/onnxruntime/python/tools/transformers/optimizer.py

Line 138 in efd416b

kwargs["disabled_optimizers"] = disabled_optimizers

[tianleiwu]

The InsertCastTransformer cannot be disabled right now:

onnxruntime/onnxruntime/core/session/inference_session.cc

Line 1066 in c969237

InsertCastTransformer insert_cast_transformer{"CastFloat16Transformer", cpu_regs};

There are some similar issues reported related to this: #8787.

Solution is to update the logic for precision loss detection here:

onnxruntime/onnxruntime/core/optimizer/insert_cast_transformer.cc

Line 303 in c969237

if (src_type_group > dst_type_group) {

and/or disable RemoveDuplicateCastTransforme when all optimization are disabled.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[cbourjau]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

The issue is not resolved, albeit it might be a duplicate.