Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nn.SiLU() export support #1713

Merged
merged 1 commit into from
Dec 17, 2020
Merged

nn.SiLU() export support #1713

merged 1 commit into from
Dec 17, 2020

Conversation

glenn-jocher
Copy link
Member

@glenn-jocher glenn-jocher commented Dec 17, 2020

nn.SiLU() export support.

πŸ› οΈ PR Summary

Made with ❀️ by Ultralytics Actions

🌟 Summary

Enhancement of model export capabilities with improved activation functions.

πŸ“Š Key Changes

  • πŸ”„ Modified export.py to handle a new activation function.
  • ✨ Introduced compatibility for the SiLU activation function in export operations.
  • 🧹 Cleaned up the activation imports by adding SiLU to the list.

🎯 Purpose & Impact

  • πŸ€– Helps ensure that models using SiLU activation functions can be exported efficiently.
  • πŸ”§ Increases the flexibility of the model export process to support latest neural network advancements.
  • πŸš€ Potentially improves model performance and compatibility with different deployment environments, benefiting users who integrate YOLOv5 models in applications requiring export functionality.

@glenn-jocher glenn-jocher merged commit 6bd5e8b into master Dec 17, 2020
@glenn-jocher glenn-jocher deleted the SiLU branch December 17, 2020 01:55
@hendrycks
Copy link

The nn.SiLU() function available in PyTorch is more optimized than x*sigmoid(x). pytorch/pytorch#42976
It might make sense to use the PyTorch native version, as it's available in 1.7.0.
Also, the arXiv link is wrong (https://arxiv.org/pdf/1905.02244.pdf is a 2019 paper), as the SiLU was initially introduced in 2016 in this paper: https://arxiv.org/pdf/1606.08415.pdf

@glenn-jocher
Copy link
Member Author

@hendrycks ah thanks for the citation correction. PyTorch ops all use the native nn.SiLU() module. Export typically has to accommodate various compatibility issues (i.e. CoreML has no SiLU() op), hence the replacement with an export friendly version.

glenn-jocher added a commit that referenced this pull request Jan 7, 2021
@glenn-jocher
Copy link
Member Author

@hendrycks citation corrected in fea9c9b

KMint1819 pushed a commit to KMint1819/yolov5 that referenced this pull request May 12, 2021
KMint1819 pushed a commit to KMint1819/yolov5 that referenced this pull request May 12, 2021
taicaile pushed a commit to taicaile/yolov5 that referenced this pull request Oct 12, 2021
taicaile pushed a commit to taicaile/yolov5 that referenced this pull request Oct 12, 2021
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants