Pending job count not correct on batch with failures allowed

[bilfeldt]

• Horizon Version: 5.7.11
• Laravel Version: 8.56.0
• PHP Version: 7.4
• Redis Driver & Version: predis/phpredis: Unknown
• Database Driver & Version: MySQL 8.0

Description:

When using batches with failures allowed then the Batch information about pending jobs shown in Horizon is misleading.

Example (can be found in github repo below):

\Illuminate\Support\Facades\Bus::batch([
        new \App\Jobs\TestJob(1), // Will succeed on first attempt
        new \App\Jobs\TestJob(2), // Will succeed on second attempt
        new \App\Jobs\TestJob(3), // Will succeed on third attempt
        new \App\Jobs\TestJob(4), // Will succeed on fourth attempt (which will never happen since $maxExceptions = 3)
        new \App\Jobs\TestJob(0), // Will never succeed
    ])->allowFailures()->dispatch();

This batch should have a total of 5 jobs and 3 of them should eventually succeed (60%). Now the end result in Horizon shows 2 pending jobs + 2 failed jobs:

I think this is a bug since only jobs not marked as "failed" or "completed" should appear as pending (since they will be processed soon). This is also confirmed by no pending jobs appearing:

While there are in fact three completed jobs:

and two failed:

Steps To Reproduce:

A Github repository showcasing the error can be seen here: https://github.com/bilfeldt/bug-report

1. Start Horizon (remember to use redis as queue driver): php artisan horizon
2. Go to /batch which will dispatch the batch of jobs
3. Go to the horizon dashboard and see the images also attached above

[themsaid]

This is the expected behaviour. Pending jobs are one that haven't finished. In your case, these are the failed jobs.

[driesvints]

I tried to improve the UI here: #1064

[bilfeldt]

Thanks for the feedback @themsaid. It makes sense that those failed jobs are in fact also pending jobs since they have not completed. That said I would expect that they are somewhere stored as being processed since they are not going to be handled by any worker from now on? Cannot seem to find this information from the batch itself though.

I like your #1064 contribution @driesvints, much appreciated.

@themsaid, @driesvints is there a way we could indicate that all jobs has been handled and no further processing will occur? It can be tricky to know when exactly the batch has no further jobs that will be processed - especially if you also use the "Retry Failed Jobs" feature, as the "Pending" jobs will consist of both jobs being retried and jobs that will no longer be retried. Maybe we should not remove the "Pending" badge as soon as there are "Failures" but instead show both badges?

I have sketched the process below, which indicates that it can be tricky to know the meaning of "pending" jobs and determining if the batch has completed running (simply added a sleep(5) to the job from the original issue above).

Process using retry

No jobs yet processed - 5 pending jobs that will be processed

First job processed (success) - 4 pending jobs that will be processed

Second job processed (success) - 3 pending jobs that will be processed

Third job processed (success) - 2 pending jobs that will be processed

Fourth job processed (failed) - 2 pending jobs but only one that will be processed

Fifth job processed (failed) - 2 pending jobs but none that will be processed

Retry jobs

Now clicking the "Retry Failed Jobs" button.

Fourth job retried (failed) - 2 pending jobs but one that will be processed

Fifth job retried (failed) - 2 pending jobs but none that will be processed

[driesvints]

Hey @bilfeldt, I think that all looks good to be honest. It's probably best that if you want to improve something here that you attempt a PR.