Add output_names argument for ONNX export with dynamic axes

[SamSamhuns]

Pull request regarding bug #3444 to fix the dynamic output shape issue.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhanced ONNX export functionality in YOLOv5.

📊 Key Changes

• 🛠 Refactored ONNX export arguments, separating input_names and output_names.
• 📐 Updated dynamic_axes definitions for inputs and outputs during ONNX export.

🎯 Purpose & Impact

• 🎨 Provides clearer code structure around names of inputs and outputs, easing understanding and maintenance.
• 🌉 Ensures better compatibility and flexibility with ONNX by explicitly naming inputs and outputs and adjusting dynamic axes, facilitating easier integration with ONNX-compatible tools.
• 💼 This may benefit users requiring custom input/output handling when exporting their models to ONNX, improving the model's interoperability and deployment potential.

[github-actions]

👋 Hello @SamSamhuns, thank you for submitting a 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify your PR is up-to-date with origin/master. If your PR is behind origin/master an automatic GitHub actions rebase may be attempted by including the /rebase command in a comment body, or by running the following code, replacing 'feature' with the name of your local branch:

git remote add upstream https://github.com/ultralytics/yolov5.git
git fetch upstream
git checkout feature  # <----- replace 'feature' with local branch name
git rebase upstream/develop
git push -u origin -f

• ✅ Verify all Continuous Integration (CI) checks are passing.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." -Bruce Lee

[glenn-jocher]

@SamSamhuns thanks for the PR! It looks like L102 serves no purpose?

[SamSamhuns]

If you remove L102, the outputs will no longer have dynamic shapes. You can try commenting out L102 and exporting with --dynamic. The output_names parameter is matching the names given in the dynamic_axes parameter.

Output of python3 -c "import onnx; m = onnx.load('yolov5s.onnx'); print(m.graph.input); print(); print(m.graph.output)" after commenting out L102

[name: "images"
type {
  tensor_type {
    elem_type: 1
    shape {
      dim {
        dim_param: "batch"
      }
      dim {

        dim_value: 3
      }
      dim {
        dim_param: "height"
      }
      dim {
        dim_param: "width"
      }
    }
  }
}
]

[name: "672"
type {
  tensor_type {
    elem_type: 1
    shape {
      dim {
        dim_value: 1
      }
      dim {
        dim_value: 25200
      }
      dim {
        dim_value: 85
      }
    }
  }
} ..... so on for other outputs

[glenn-jocher]

@SamSamhuns ah perfect, now I understand! I didn't see they were arguments since I was only looking at the diff before.

[glenn-jocher]

@SamSamhuns if we pass the y and x keys in the output dictionaries does that break things?

{0: 'batch', 2: 'y', 3: 'x'}

[SamSamhuns]

Unfortunately, if you pass in {0: 'batch', 2: 'y', 3: 'x'} for let's say the first output, you'll get a weird output shape of ["batch", 25200, "y"] for the first output.

As I see it, the non-batch axes of the output (all axes after the first one) need not be stated in the dynamic_axes parameter of the torch.onnx.export. You could use {0: 'batch', 1: 'y', 2: 'x'} for the first output which would give an output shape of ["batch", 'y, "x"] but this seems unnecessary and it will again not match the pattern for the other outputs which have 5 axes (one more than the first).

I'd expect the shapes should be the following for the respective export commands:

1. python models/export.py

    INPUT = [1,3,640,640]
    OUTPUT = [[1, 25200, 85], [1,3,80,80,85], [1,3,40,40,85], [1,3,20,20,85]]

2. python models/export.py --dynamic

    INPUT = [batch,3,height,width]
    OUTPUT = [[batch, 25200, 85], [batch,3,80,80,85], [batch,3,40,40,85], [batch,3,20,20,85]]

So if we add {0: 'batch', 2: 'y', 3: 'x'}, the shape would be OUTPUT = [[batch, 25200, y], [batch,3,80,80,85], [batch,3,40,40,85], [batch,3,20,20,85]]

And if we add {0: 'batch', 1: 'y', 2: 'x'}, the shape would be OUTPUT = [[batch, y, x], [batch,3,80,80,85], [batch,3,40,40,85], [batch,3,20,20,85]]

If you really need the x and y names for the axes of the shape somewhere later on, you can still use this

torch.onnx.export(model, img, f, verbose=False, opset_version=opt.opset_version, input_names=['images'],
                              training=torch.onnx.TrainingMode.TRAINING if opt.train else torch.onnx.TrainingMode.EVAL,
                              do_constant_folding=not opt.train,
                              output_names=['output0', 'output1', 'output2', 'output3'],
                              dynamic_axes={'images': {0: 'batch', 2: 'height', 3: 'width'},  # size(1,3,640,640)
                                            'output0': {0: 'batch', 1: 'y', 2: 'x'},
                                            'output1': {0: 'batch'},
                                            'output2': {0: 'batch'},
                                            'output3': {0: 'batch'}} if opt.dynamic else None)

[glenn-jocher]

@SamSamhuns Ok I see. I think the reason that the height, width, y, x axes are labelled as dynamic is because some workflows involve users running inference at different input image size (in addition to different batch sizes). I think maybe the reason the first output is the only one labelled as such is because this is the only one that should be used, it is the 3 concatenated outputs with grid applied. Except now that I look at it the y, x don't make any sense to apply to the first output, the only axes that should be dynamic on input shape change are dim 0 and 1, or batch and anchors.

For example:

    INPUT = [1,3,640,640]
    OUTPUT = [[1, 25200, 85], [1,3,80,80,85], [1,3,40,40,85], [1,3,20,20,85]]

    INPUT = [1,3,320,480]
    OUTPUT = [[1, 9450, 85], [1,3,40,60,85], [1,3,20,30,85], [1,3,10,15,85]]

[glenn-jocher]

@SamSamhuns could we do this to allow for dynamic batch and image height width?

torch.onnx.export(model, img, f, verbose=False, opset_version=opt.opset_version, input_names=['images'],
                              training=torch.onnx.TrainingMode.TRAINING if opt.train else torch.onnx.TrainingMode.EVAL,
                              do_constant_folding=not opt.train,
                              output_names=['output0', 'output1', 'output2', 'output3'],
                              dynamic_axes={'images': {0: 'batch', 2: 'height', 3: 'width'},  # shape(1,3,640,640)
                                            'output0': {0: 'batch', 1: 'anchors'},  # shape(1,25200,85)
                                            'output1': {0: 'batch', 2: 'y', 3: 'x'},  # shape(1,3,80,80,85)
                                            'output2': {0: 'batch', 2: 'y', 3: 'x'},  # shape(1,3,40,40,85) ... etc.
                                            'output3': {0: 'batch', 2: 'y', 3: 'x'},} if opt.dynamic else None)

[SamSamhuns]

Hmm, I guess if you want users to only use the first output then the export can only contain the dynamic_axes for the first output

torch.onnx.export(model, img, f, verbose=False, opset_version=opt.opset_version, input_names=['images'],
                              training=torch.onnx.TrainingMode.TRAINING if opt.train else torch.onnx.TrainingMode.EVAL,
                              do_constant_folding=not opt.train,
                              output_names=['output'],
                              dynamic_axes={'images': {0: 'batch', 2: 'height', 3: 'width'},  # size(1,3,640,640)
                                            'output': {0: 'batch', 1: 'x'}} if opt.dynamic else None)

the x should be there though since the 2nd dim of the first output changes with the input shapes as you show it, so now the shapes for dynamic onnx model will be: (The 3 other outputs that do not have dynamic output shapes)

# for img-size 640 640

 INPUT = [batch,3,height,width]
 OUTPUT = [[batch, x, 85], [1,3,80,80,85], [1,3,40,40,85], [1,3,20,20,85]]

[glenn-jocher]

@SamSamhuns hey perfect! I'll go ahead and update your PR then with this then. Typically we call this 1D vector of output points 'anchors', as in 'the model outputs 25k anchors'.

            torch.onnx.export(model, img, f, verbose=False, opset_version=opt.opset_version,
                              training=torch.onnx.TrainingMode.TRAINING if opt.train else torch.onnx.TrainingMode.EVAL,
                              do_constant_folding=not opt.train,
                              input_names=['images'],
                              output_names=['output'],
                              dynamic_axes={'images': {0: 'batch', 2: 'height', 3: 'width'},  # shape(1,3,640,640)
                                            'output': {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
                                            } if opt.dynamic else None)

[glenn-jocher]

@SamSamhuns all CI tests show ONNX export working correctly, and locally on MacOS I was able to use --dynamic also with no problems. Result looks like this.

[glenn-jocher]

@SamSamhuns PR is merged! Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!