Finer Control Over Which Worker Does What

[fm3]

The Worker postgres table now has new columns to control job scheduling:

maxParallelHighPriorityJobs INT NOT NULL DEFAULT 1,
maxParallelLowPriorityJobs INT NOT NULL DEFAULT 1,
supportedJobCommands VARCHAR(256)[] NOT NULL DEFAULT array[]::varchar(256)[]

This means that in a multi-worker setup, you can select which worker should run which jobs. Also, you can allow low-pri jobs to be scheduled with less parallelity than high-pri jobs.

Steps to test:

• Compare worker setup instructions at https://github.com/scalableminds/voxelytics/tree/master/voxelytics/worker
• With the new schema, adapt your postgres workers table to support different job commands
• Should be distributed to your workers accordingly

TODOs:

• support parallelity for high and low priority jobs
• support list of supported job commands per worker
• add possibility for frontend to query which job commands are available for a datastore
• sql migration

Follow-Up

• Jobs that are not available for a particular datastore should already be disabled in the frontend #7485

Issues:

• fixes Finer-grained maxparalleljobs #7458

---

• Updated changelog
• Updated migration guide if applicable

[frcroth]

This is general and should not be configurable?

[fm3]

I’d say for now, yes

[hotzenklotz]

@fm3 If I see this correctly then the PR only adds the jobsSupportedByAvailableWorkers info to the endpoint /api/datasets/<organization>/<dataset_name>.

What about the /api/features route? Some worker jobs can only be triggered in admin menus, e.g. dataset upload/conversion or finding the max segment ID.

[fm3]

Good point!

I now pushed a commit on your branch of PR #7591

In there I moved both fields jobsEnabled and jobsSupportedByAvailableWorkers from the dataset json to the dataStore json.

I cannot change the /api/features route content since that is directly from the application.conf, whereas this information is worker (and thus datastore) specific.

Note that the dataset json does contain a full copy of the datastore json (dataset.dataStore) so all points that previously accessed dataset.jobsEnabled can now, as far as I understand, instead access dataset.dataStore.jobsEnabled.

In dataset upload, you can use the info from the selected datastore.

When finding the largestSegmentId, I would have expected that you can use the dataset-specific route, as this concerns a specific dataset.

Hope this helps!