About the difficulties of exporting onnx

[nullhd2]

Hello Denis,
Thanks a lot for your proposed CFlow method!
Do you have any plans to export the model to 'onnx' format? In my attempts to export, I had to use 'torch.jit.script' as an intermediate method to implement 'onnx' due to the presence of the loop in the forward code. But this seems to be difficult and the Freia library always fails to export.
Sincerely looking forward to your reply!

[Tekno-H]

Hello @nullhd2 ,
I am also trying to export the CFlow model to onnx format, but failing to do so...
I think that this is a good starting point to reach our goal.
But I am not sure how I should proceed.
I stumbled upon this comment on a stack-overflow issue:

• Unfortunately, you can't create a single combined model from encoder and decoder, even if you do it you won't be able to use it while inferencing. because the encoder runs only once and the decoder runs several times.

Maybe we should create separate models for the encoder and decoder?
Please update if you have any progress.

[gudovskiy]

@nullhd2 @Tekno-H did you check https://github.com/openvinotoolkit/anomalib ?

[Tekno-H]

@gudovskiy Yes I already checked that repository.
The problem is that they use a different structure for the Cflow model.
In order for the export to work, one must create a base class "model" to feed the torch.onnx.export function.
And although your paper clearly depicts how the model is structured, I was not able to build that class.
I have spent some time trying to do the export, I think if the base model is available the export can be easily done.
Any help would be appreciated.

[gudovskiy]

@Tekno-H I will be on the trip next week. May be I will be able to look at this problem after that

[Tekno-H]

@Tekno-H I will be on the trip next week. May be I will be able to look at this problem after that

Thanks, in the mean time I will try my best to come up with a solution.
Have a safe trip.

[Tekno-H]

Hello @gudovskiy
Where you able to give this issue a thought ?
This is an issue I opened to get some assistance.

[gudovskiy]

Hi @Tekno-H, could you pull request your export to ONNX script from this issue such that I can run it?

[Tekno-H]

@gudovskiy
I made the pull request, my scripts aren't beautiful.
If you have any issues I am ready to help.
I believe the source of the problem is from the FreIA library, maybe some operator can't be converted ?
I was able to isolate the error at line 248 in the all_in_one_block.py file, I am currently trying to understand why it fails there.

[dygcjlu]

Hi @Tekno-H ， Have you found a solution for this issue. I got the following error message when i tried to convert the model.

/home/hu/anaconda3/envs/mmdetection/bin/python /home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py

LR schedule: [12, 18, 22]
/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torchvision/models/mobilenetv2.py:23: FutureWarning: The ConvBNReLU/ConvBNActivation classes are deprecated since 0.12 and will be removed in 0.14. Use torchvision.ops.misc.Conv2dNormActivation instead.
warnings.warn(
CNF coder: 40
CNF coder: 112
CNF coder: 160
Loading weights from /home/hu/dyg/anomaly_detection/model/mvtec_mobilenet_v3_large_freia-cflow_pl3_cb8_inp512_run0_metal_nut.pt
Loaded !
/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py:80: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
FIB = E // self.N + int(E % self.N > 0) # number of fiber batches
/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py:80: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
FIB = E // self.N + int(E % self.N > 0) # number of fiber batches
/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if f < (FIB - 1):
Traceback (most recent call last):
File "/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py", line 210, in
main(c)
File "/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py", line 198, in main
torch.onnx.export(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/init.py", line 350, in export
return utils.export(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/utils.py", line 163, in export
_export(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/utils.py", line 1074, in _export
graph, params_dict, torch_out = _model_to_graph(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/utils.py", line 727, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/utils.py", line 602, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/onnx/utils.py", line 517, in _trace_and_get_graph_from_model
trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/jit/_trace.py", line 1175, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/jit/_trace.py", line 127, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/jit/_trace.py", line 118, in wrapper
outs.append(self.inner(*trace_inputs))
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py", line 110, in forward
height, width, test_dist = self.Decoder_module(self.pool_layers)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/dyg/anomaly_detection/cflow-ad/cflow-to-onnx2.py", line 91, in forward
z, log_jac_det = decoder(e_p, [c_p, ])
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/FrEIA/framework/sequence_inn.py", line 106, in forward
x_or_z, j = self.module_list[i](x_or_z, c=[c[self.conditions[i]]],
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/FrEIA/modules/all_in_one_block.py", line 249, in forward
a1 = self.subnet(x1c)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/dyg/anomaly_detection/cflow-ad/model.py", line 48, in forward
return self.subnet(input)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/hu/anaconda3/envs/mmdetection/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient
Tensor:
Columns 1 to 6-5.7972e-02 -5.9264e-02 6.5910e-02 6.2591e-02 7.7508e-02 -2.9350e-02
-1.3733e-02 4.6010e-03 2.8477e-02 5.3804e-03 -6.9055e-02 -5.5563e-02.....

[l1onog]

Hi, @dygcjlu Have you found a solution for this issue. I got the same error message with you when i tried to convert the model.