SN1-195: add inference task #344

Hollyqui · 2024-08-21T11:29:52Z

Added Dynamic Model Loading/unloading to allow validators to run multiple reference models
Added inference task to allow us to force miners to generate with a specific model (layer to be used for ensembling)
Made scoring loop run asynchronously from task generation

Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>

…macrocosm-os/prompting into SN1-110-move-from-pip-to-poetry

* initial push on openai miner * deprecates langchain and hf miner, adds openai miner * brings back pm2 autorun command * Prevent multi-turn on translation and sentiment * updates gitignore * bumps bittensor version * adds block to logged wandb data * updates version * Add Unit Tests * Improve multi-turn prompting * Reiterate need for pronouns * Stress importance of pronouns * SN1-12 * drops block from log data * Switch last_update_block to local var instead of metagraph attribute * Update README.md * update logging to use python rather than loguru refs #200 * Reduce timout to 15 seconds * Change event logging * Remove loguru requirement * Increment Version Number * v2.5.2 (#287) * Point to macrocosmos entity * Adjust project name * Increase spec version to 2.5.2 --------- Co-authored-by: bkb2135 <bkb2135@columbia.edu> Co-authored-by: p-ferreira <38992619+p-ferreira@users.noreply.github.com> Co-authored-by: p-ferreira <pe_drojunior@hotmail.com> Co-authored-by: bkb2135 <98138173+bkb2135@users.noreply.github.com> Co-authored-by: Chengzhe <chengzhe8919@gmail.com>

Merge Main into pre-staging

- Added pytest and cleaned poetry.lock - Make vllm optional via validator extra - Made `angle-emb` optional

… typing

…-task

…ng probability sum

dbobrenko

Make sure to change PR request to pre-staging branch, and merge with v2.7.1 changes

dbobrenko · 2024-08-29T18:42:52Z

README.md

+
+# Compute Requirements
+
+1. To run a **validator**, you will need at least 70GB of VRAM.


We have vali/miner requirements and instructions defined in assets/validator.md and assets/miner.md
Make sure to merge with v2.7.0+

I believe this should already be merged with v2.7.1, no?

So it is merged with 2.7.1 but pre-staging has a different commit history, so github doesn't realise. I would suggest force pushing this into pre-staging later to get the correct commit history which is then also aligned with staging/main

We moved miner/validator info into separate markdown files, main README is currently short and concise with links to detailed instructions
Looks like you've restored the previous version of README

neurons/forward.py

neurons/miners/inference_miner/miner.py

neurons/validator.py

prompting/tasks/task_registry.py

prompting/utils/timer.py

dbobrenko · 2024-08-29T20:46:06Z

prompting/utils/uids.py

@@ -45,7 +42,7 @@ def check_uid_availability(
    return True


-def get_random_uids(self: BaseNeuron, k: int, exclude: list[int] = None) -> np.ndarray:
+def get_random_uids(k: int | None = 10**6, exclude: list[int] = None, own_uid: int | None = None) -> np.ndarray:


We should make k mandatory, it's bounded to ~1000

Suggested change

def get_random_uids(k: int | None = 10**6, exclude: list[int] = None, own_uid: int | None = None) -> np.ndarray:

def get_random_uids(k: int, exclude: list[int] = None, own_uid: int | None = None) -> np.ndarray:

Why is that needed - there's only 1k miners in the network, right? So passing k=None simply means you sample all miners

Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>

- Adding dataset to retrieve random website, which feeds the inference task

dbobrenko

LGTM

dbobrenko · 2024-09-02T13:10:35Z

README.md

+
+# Compute Requirements
+
+1. To run a **validator**, you will need at least 70GB of VRAM.


We moved miner/validator info into separate markdown files, main README is currently short and concise with links to detailed instructions
Looks like you've restored the previous version of README

prompting/llms/vllm_llm.py

…nce-task

Hollyqui and others added 30 commits July 15, 2024 14:53

Moving from pip to poetry

aab1837

SN1-14: Adding additional packages

7bdef43

SN1-14: Fixing issues with pyproject.toml file

78a6ce7

SN1-14: Improving README file

c24e2b8

Adding ruff, pre-commit hooks & black config

93e4ed6

SN1-110: Adding poetry env activation to github actions workflow

a892488

SN1-110: Fixing poetry env activation

8df0e9d

SN1-110: Run pytest with poetry

dc1bb55

Removing unnecessary newline

4076ee4

Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>

SN1-110: Moving line length to 120

a742b9f

Merge branch 'SN1-110-move-from-pip-to-poetry' of https://github.com/…

7e8b7f1

…macrocosm-os/prompting into SN1-110-move-from-pip-to-poetry

SN1-110: Attempting to fix CI/CD

40b2cee

SN1-110: Adding debugging to CI/CD

b4f712b

Merge pull request #305 from macrocosm-os/main

40f5a6e

Merge Main into pre-staging

SN1-110 Rebased

8e25639

- Added pytest and cleaned poetry.lock - Make vllm optional via validator extra - Made `angle-emb` optional

SN1-110 Make vllm optional via validator extra

777c9cf

SN1-110 made angle-emb optional install

4f8a77c

SN1-110 Github actions to install extras so can run tests

7fdd464

SN1-110 moved import for flake8 linter

b2f221f

SN1-110 testing pre-commit hook

3a3a85a

SN1-110 removed dup code

1d6a7f3

SN1-110 Merge staging

66b9c57

SN1-152: Fixing styling & minor refactors

511d7e5

SN1-152: Continuing refactor: eliminating configs in favour of proper…

c1f7001

… typing

SN1-110 fix tests

d69e17e

SN1-110 Fix float/integer error with math.py in tests

ab4e9f8

SN1-110 revert python version on test and updated readme

abb194b

SN1-152: Continuing refactors; adding demo notebook

3452cb5

SN1-110 upgrade btcli to 7.3.1

e41a68a

Hollyqui requested review from dbobrenko and bkb2135 and removed request for dbobrenko August 28, 2024 18:10

Hollyqui changed the base branch from SN1-150-remove-low-value-subcompetitions to release/v2.7.1 August 28, 2024 18:13

Hollyqui changed the base branch from release/v2.7.1 to pre-staging August 28, 2024 18:14

Hollyqui added 3 commits August 29, 2024 11:53

SN1-195: Removing issues with namespaces

5b14c48

SN1-195: Fixing issue with reward calculation

ded44b5

Merge remote-tracking branch 'origin/main' into SN1-195-add-inference…

15ae88c

…-task

Hollyqui changed the base branch from pre-staging to main August 29, 2024 15:48

Hollyqui added 4 commits August 29, 2024 16:34

SN1-195: Removing tokenizer from stream processing

55a7eed

SN1-195: Fixing miner issues introduced by merge

f918b13

SN1-195: Making sure task prbabilities add up to 1

ef0bd12

SN1-195: Ensuring to round to avoid floating point issues when checki…

008a40b

…ng probability sum

dbobrenko requested changes Aug 29, 2024

View reviewed changes

Hollyqui changed the base branch from main to pre-staging August 30, 2024 10:13

Hollyqui and others added 2 commits August 30, 2024 11:14

SN1-195: Timer to use perf_counter

15949d7

Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>

SN1-195: Moving to perf_counter

e5e8e9e

Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>

Hollyqui changed the base branch from pre-staging to release/v2.7.1 August 30, 2024 10:20

Hollyqui and others added 5 commits August 30, 2024 10:34

SN1-195: Addressing various comments

dd4d8cd

SN1-195: Minor fixes/improvements

4e38c11

SN1-195: Moving loops into __name__ == __main__

4d944da

SN1-195: Setting ram based on .env file

bdd5230

SN1-224: Adding random website dataset (#359)

1f061fc

- Adding dataset to retrieve random website, which feeds the inference task

Hollyqui changed the base branch from release/v2.7.1 to staging September 2, 2024 12:05

dbobrenko approved these changes Sep 2, 2024

View reviewed changes

Hollyqui and others added 3 commits September 3, 2024 12:04

Merge remote-tracking branch 'origin/staging' into SN1-195-add-infere…

6496fc5

…nce-task

SN1-212: Fixing readme

117a688

Merge branch 'staging' into SN1-195-add-inference-task

df9c5fb

Hollyqui merged commit e766486 into staging Sep 4, 2024
0 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SN1-195: add inference task #344

SN1-195: add inference task #344

Hollyqui commented Aug 21, 2024

dbobrenko left a comment •

edited

Loading

dbobrenko Aug 29, 2024

Hollyqui Aug 30, 2024

Hollyqui Sep 2, 2024

dbobrenko Sep 2, 2024

dbobrenko Aug 29, 2024

Hollyqui Aug 30, 2024

dbobrenko left a comment

dbobrenko Sep 2, 2024


		# Compute Requirements

		1. To run a validator, you will need at least 70GB of VRAM.

	def get_random_uids(k: int \| None = 10**6, exclude: list[int] = None, own_uid: int \| None = None) -> np.ndarray:
	def get_random_uids(k: int, exclude: list[int] = None, own_uid: int \| None = None) -> np.ndarray:

SN1-195: add inference task #344

SN1-195: add inference task #344

Conversation

Hollyqui commented Aug 21, 2024

dbobrenko left a comment • edited Loading

Choose a reason for hiding this comment

dbobrenko Aug 29, 2024

Choose a reason for hiding this comment

Hollyqui Aug 30, 2024

Choose a reason for hiding this comment

Hollyqui Sep 2, 2024

Choose a reason for hiding this comment

dbobrenko Sep 2, 2024

Choose a reason for hiding this comment

dbobrenko Aug 29, 2024

Choose a reason for hiding this comment

Hollyqui Aug 30, 2024

Choose a reason for hiding this comment

dbobrenko left a comment

Choose a reason for hiding this comment

dbobrenko Sep 2, 2024

Choose a reason for hiding this comment

dbobrenko left a comment •

edited

Loading