feat: deterministic tasking #85
Conversation
Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
| export async function getTaskKey (task, randomness) { | ||
| assertEquals(typeof task, 'object', 'task must be an object') | ||
| assertEquals(typeof task.cid, 'string', 'task.cid must be a string') | ||
| assertEquals(typeof task.minerId, 'string', 'task.minerId must be a string') | ||
| assertEquals(typeof randomness, 'string', 'randomness must be a string') | ||
|
|
||
| const data = [task.cid, task.minerId, randomness].join('\n') | ||
| const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(data)) | ||
| return BigInt('0x' + encodeHex(hash)) | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} stationId | ||
| */ | ||
| export async function getStationKey (stationId) { | ||
| assertEquals(typeof stationId, 'string', 'stationId must be a string') | ||
|
|
||
| const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(stationId)) | ||
| return BigInt('0x' + encodeHex(hash)) | ||
| } | ||
|
|
||
| /** | ||
| * @param {object} args | ||
| * @param {Task[]} args.tasks | ||
| * @param {string} args.stationId | ||
| * @param {string} args.randomness | ||
| * @param {number} args.maxTasksPerRound | ||
| * @returns {Promise<Task[]>} | ||
| */ | ||
| export async function pickTasksForNode ({ tasks, stationId, randomness, maxTasksPerRound }) { | ||
| assertInstanceOf(tasks, Array, 'tasks must be an array') | ||
| assertEquals(typeof stationId, 'string', 'stationId must be a string') | ||
| assertEquals(typeof randomness, 'string', 'randomness must be a string') | ||
| assertEquals(typeof maxTasksPerRound, 'number', 'maxTasksPerRound must be a number') | ||
|
|
||
| const keyedTasks = await Promise.all(tasks.map( | ||
| async (t) => ({ ...t, key: await getTaskKey(t, randomness) }) | ||
| )) | ||
| const stationKey = await getStationKey(stationId) | ||
|
|
||
| /** | ||
| * @param {{key: bigint}} a | ||
| * @param {{key: bigint}} b | ||
| * @returns {number} | ||
| */ | ||
| const comparator = (a, b) => { | ||
| const ad = a.key ^ stationKey | ||
| const bd = b.key ^ stationKey | ||
| return ad > bd ? 1 : ad < bd ? -1 : 0 | ||
| } | ||
|
|
||
| keyedTasks.sort(comparator) | ||
| keyedTasks.splice(maxTasksPerRound) | ||
|
|
||
| return keyedTasks.map(({ key, ...t }) => (t)) | ||
| } |
There was a problem hiding this comment.
This code will need to stay in sync with the fraud-detection code in spark-evaluate. At the same time, I don't expect it to change often. When we do change it, we need to carefully design a migration/deployment path how to update both spark-evaluate and the checker network at the same time. Considering that, I think it's ok to keep this code duplicated between the two GitHub repos (spark and spark-evaluate).
Also in spark-evaluate, I'll need to use a more efficient version using k-closest module instead of Array.prototype.sort. If we wanted to share that code, I would need to place it into a standalone npm package or else inline k-closest module into our codebase, otherwise we won't be able to vendor it here.
There was a problem hiding this comment.
Also in spark-evaluate, I'll need to use a more efficient version using k-closest module instead of Array.prototype.sort
Does this method produce different results?
There was a problem hiding this comment.
It must produce the same results, otherwise the checkers will pick tasks that will not be accepted by spark-evaluate's fraud detection.
I verified this in my PoC by running Array.prototype.sort alongside k-closest and comparing the results. See here:
https://github.com/filecoin-station/spark-evaluate/pull/287/files#diff-0f9e17445925f01057ff062d7d52d2862b081121d6e722cdeda8a2b1ad106ebcR275-R283
There was a problem hiding this comment.
Ok, and have you checked if Array#sort is too slow for spark-evaluate?
There was a problem hiding this comment.
If we can get dependabot to work for this repo, then I think it would be nicest to share this code in a dependency. If not, inlining like this should be the simplest
There was a problem hiding this comment.
Ok, and have you checked if
Array#sortis too slow forspark-evaluate?
Yes, of course. I don't remember the exact number, though. It was an order of magnitude slower.
If we can get dependabot to work for this repo, then I think it would be nicest to share this code in a dependency. If not, inlining like this should be the simplest
I don't see any easy way how to enable dependabot here, let's keep this inlined for now.
If not, can we add comments to spark-evaluate to also update this?
Sure. Would you like me to add a code comment to this file too?
There was a problem hiding this comment.
Yes, of course. I don't remember the exact number, though. It was an order of magnitude slower.
Order of magnitude slower doesn't mean too slow though. Eg if it's 1ms vs 10ms or even 100ms it's probably still not be worth the extra complexity.
Sure. Would you like me to add a code comment to this file too?
I don't think it's necessary since we're going to evolve the algorithm from spark-evaluate and not this repo
There was a problem hiding this comment.
The k-closest version takes 3-4 seconds to finish. The sort-based version took 1-2 minutes IIRC.
There was a problem hiding this comment.
I re-ran the PoC using the Array sort() version. It takes 27.38 seconds to build the list of valid tasks.
Evaluating round 10788n of contract 0x8460766edc62b525fc1fa4d628fc79229dc73031
(...)
EVALUATE ROUND 10788n: using randomness f427ee008087a37142c9c22f6b227a1f915a495648bf3a3494c555c097f0880c
EVALUATE ROUND 10788n: built per-node task lists in 27380ms [Tasks=1000;TN=15;Nodes=61095]
The k-closest version takes 4 seconds to complete.
EVALUATE ROUND 10788n: using randomness f427ee008087a37142c9c22f6b227a1f915a495648bf3a3494c555c097f0880c
EVALUATE ROUND 10788n: built per-node task lists in 4040ms [Tasks=1000;TN=15;Nodes=61095]
For perspective: using the current spark-evaluate main, the entire dry-run evaluation of a single round takes 4.86 seconds.
I thought that increasing the time to evaluate a round from 5 seconds to 33 seconds is not acceptable. However, if you think it's fine, then I am happy to use the Array sort() version in spark-evaluate too.
There was a problem hiding this comment.
Blocking the event loop for 33 seconds is super long, and the service will be unresponsive and not receive events. I agree k-closest is worth it here :)
Co-authored-by: Julian Gruber <julian@juliangruber.com>
Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
| export async function getTaskKey (task, randomness) { | ||
| assertEquals(typeof task, 'object', 'task must be an object') | ||
| assertEquals(typeof task.cid, 'string', 'task.cid must be a string') | ||
| assertEquals(typeof task.minerId, 'string', 'task.minerId must be a string') | ||
| assertEquals(typeof randomness, 'string', 'randomness must be a string') | ||
|
|
||
| const data = [task.cid, task.minerId, randomness].join('\n') | ||
| const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(data)) | ||
| return BigInt('0x' + encodeHex(hash)) | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} stationId | ||
| */ | ||
| export async function getStationKey (stationId) { | ||
| assertEquals(typeof stationId, 'string', 'stationId must be a string') | ||
|
|
||
| const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(stationId)) | ||
| return BigInt('0x' + encodeHex(hash)) | ||
| } | ||
|
|
||
| /** | ||
| * @param {object} args | ||
| * @param {Task[]} args.tasks | ||
| * @param {string} args.stationId | ||
| * @param {string} args.randomness | ||
| * @param {number} args.maxTasksPerRound | ||
| * @returns {Promise<Task[]>} | ||
| */ | ||
| export async function pickTasksForNode ({ tasks, stationId, randomness, maxTasksPerRound }) { | ||
| assertInstanceOf(tasks, Array, 'tasks must be an array') | ||
| assertEquals(typeof stationId, 'string', 'stationId must be a string') | ||
| assertEquals(typeof randomness, 'string', 'randomness must be a string') | ||
| assertEquals(typeof maxTasksPerRound, 'number', 'maxTasksPerRound must be a number') | ||
|
|
||
| const keyedTasks = await Promise.all(tasks.map( | ||
| async (t) => ({ ...t, key: await getTaskKey(t, randomness) }) | ||
| )) | ||
| const stationKey = await getStationKey(stationId) | ||
|
|
||
| /** | ||
| * @param {{key: bigint}} a | ||
| * @param {{key: bigint}} b | ||
| * @returns {number} | ||
| */ | ||
| const comparator = (a, b) => { | ||
| const ad = a.key ^ stationKey | ||
| const bd = b.key ^ stationKey | ||
| return ad > bd ? 1 : ad < bd ? -1 : 0 | ||
| } | ||
|
|
||
| keyedTasks.sort(comparator) | ||
| keyedTasks.splice(maxTasksPerRound) | ||
|
|
||
| return keyedTasks.map(({ key, ...t }) => (t)) | ||
| } |
There was a problem hiding this comment.
Ok, and have you checked if Array#sort is too slow for spark-evaluate?
Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
Suspect IssuesThis pull request was deployed and Sentry observed the following issues:
Did you find this useful? React with a 👍 or 👎 |
Rework the current random task selection algorithm to a deterministic algorithm using DRAND randomness and distance between the hash of the task and the hash of the node to sort the tasks in a deterministic way that's unique to each round and each node.
Design doc:
Proof of Concept:
Links: