You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When there are multiple test dataloaders and test_percent_check is specified. The estimated total batches are incorrect and progress bar doesn't show properly.
For example, when I specify two dataloaders each of which has 100 batches and test_percent_check=0.1. The expected total batches are 200*0.1=20. But actually, 40 batches are run.
Return multiple dataloaders from test_dataloaders()
Specify test_percent_check.
Run trainer.test()
Observe expected_batches * num_loaders be run. The progress bar also fails to show progress after expected_batches as it exceeds its specified total steps.
Just had a look at this. The problem is in the trainer as you say, not the progress bar.
There are two loops, the outer runs through the number of dataloaders and the inner loop runs through each.
so the max batches should be the number of batches to run in each dataloader, not totally.
We can easily fix this.
There should really be a test. There seems to be no test that checks that *_percent_check works with the correct amount of data. we should definitely have these tests.
🐛 Bug
When there are multiple test dataloaders and
test_percent_check
is specified. The estimated total batches are incorrect and progress bar doesn't show properly.For example, when I specify two dataloaders each of which has 100 batches and
test_percent_check=0.1
. The expected total batches are200*0.1=20
. But actually,40
batches are run.At this line,
num_batches
is the global number of batches and will be assigned toself.num_test_batches
. https://github.com/PyTorchLightning/pytorch-lightning/blob/3459a546672303204a4ae6efcc2613a90f003903/pytorch_lightning/trainer/data_loading.py#L243while in the evaluation loop,
max_batches
is regarded as the number of batches for one data loader.https://github.com/PyTorchLightning/pytorch-lightning/blob/3459a546672303204a4ae6efcc2613a90f003903/pytorch_lightning/trainer/evaluation_loop.py#L262
To Reproduce
Steps to reproduce the behavior:
test_dataloaders()
test_percent_check
.trainer.test()
expected_batches * num_loaders
be run. The progress bar also fails to show progress afterexpected_batches
as it exceeds its specified total steps.Expected behavior
Run correct number of batches.
Environment
The text was updated successfully, but these errors were encountered: