Heroku / dokku and "ValueError: There's already an active RQ scheduler"

[darkpixel]

I'm running dokku, which is a "Docker powered mini-Heroku in around 100 lines of Bash".

I switched to RQ a few weeks ago after celery kept locking up workers due to a deep multi-threading issue in python. RQ is working perfectly except for one small issue...

When I 'git push' my code up to my server, dokku creates a new container with the updated code, then it starts all the services in the NEW container, makes sure they are running, switches traffic to the new container, then shuts down the old container.

The only problem is that rq-scheduler checks to see if there is an active instance of the scheduler running. Unfortunately there is--in the old container that is about to get killed.

In my case, I don't care if the about-to-die-scheduler queues up a bunch of jobs in an instant before the new scheduler comes up and queues the same jobs up.

Would a command-line flag to ignore an already running scheduler, be useful?

Another option might be to allow multiple schedulers to run, but only the one that holds the lock can submit new tasks. If it dies, and the lock expires, another scheduler could grab it and start submitting...

I'd be willing to try my hand at writing a patch if you approve of either idea.

[jmmills]

Related issues: #62 and #70

On Aug 17, 2015, at 9:56 PM, Aaron C. de Bruyn notifications@github.com wrote:

I'm running dokku, which is a "Docker powered mini-Heroku in around 100 lines of Bash".

I switched to RQ a few weeks ago after celery kept locking up workers due to a deep multi-threading issue in python. RQ is working perfectly except for one small issue...

When I 'git push' my code up to my server, dokku creates a new container with the updated code, then it starts all the services in the NEW container, makes sure they are running, switches traffic to the new container, then shuts down the old container.

The only problem is that rq-scheduler checks to see if there is an active instance of the scheduler running. Unfortunately there is--in the old container that is about to get killed.

In my case, I don't care if the about-to-die-scheduler queues up a bunch of jobs in an instant before the new scheduler comes up and queues the same jobs up.

Would a command-line flag to ignore an already running scheduler, be useful?

Another option might be to allow multiple schedulers to run, but only the one that holds the lock can submit new tasks. If it dies, and the lock expires, another scheduler could grab it and start submitting...

I'd be willing to try my hand at writing a patch if you approve of either idea.

—
Reply to this email directly or view it on GitHub.

[selwin]

I like this idea, please see my comment here: #70 (comment)

[Fajkowsky]

Any suggestions how to fix that?

[isuruperera97]

remove all keys in redis if your in development