Dataset processing time.

[UnkoC0wbOy]

How long does it take the hub to finish process datasets after uploading?

[kalenmike]

@UnkoC0wbOy The dataset processing will depend on how many images are in the uploaded dataset. COCO2017 for example took 3 hours to finish processing. We are working to improve the speed of processing datasets. The current implementation creates optimised copies of all the images and this along with the data transfer is what is causing the delay. These copies are only used in the preview on the Ultralytics HUB. The original images are used for training.

[UnkoC0wbOy]

Thank you! Is there anything that can be done to optimize our datasets for faster processing times? I have two relatively small datasets that have been processing for close to 36 hours.

Also is there a way to change how many epochs the dataset trains for? I've noticed that the default is set at 300 epochs.

[glenn-jocher]

@UnkoC0wbOy you should be able to edit the epochs field here before training starts:

I just tried this myself though and the edit field disappears when I place my cursor on it. @kalenmike I'll raise a new issue for this.

EDIT: raised bug in #10

[glenn-jocher]

@UnkoC0wbOy if your datasets are relatively small, i.e. ~5k images they should process in less than an hour, maybe only a few minutes depending on the image sizes and labels count. It's likely there may have been some problem in dataset processing. Can you delete your two uploads and try again, and if your times still exceed an hour please raise a new bug report with a link to your *.zip file and copying your system info shown below. Thank you!