New `TaskV2` and `TaskGroupV2` attributes #1856

ychiucco · 2024-10-07T09:29:47Z

closes #1828
closes #1829

Checklist before merging

I added an appropriate entry to CHANGELOG.md
I added logging to new code - if appropriate.
I merged main into the current branch.

github-actions · 2024-10-07T09:34:23Z

Benchmark comparison

GET /api/alive/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`
vanilla	1.0	0.8	1.25
power	1.0	0.8	1.25
dataset	1.0	0.8	1.25
project	1.0	0.8	1.25
job	1.0	0.8	1.25

GET /api/v2/dataset/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	10.3	10.5	0.98	0.6	0.6
power	387.9	408.6	0.95	389.0	389.0
dataset	151.6	154.4	0.98	145.6	145.6
project	47.8	50.5	0.95	50.9	50.9
job	27.6	29.0	0.95	35.3	35.3

GET /api/v2/job/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	6.5	7.1	0.92	2.2	2.2
power	137.0	139.5	0.98	652.4	652.4
dataset	138.1	143.5	0.96	905.6	905.6
project	47.0	47.0	1.00	283.9	283.9
job	36.8	40.7	0.90	225.0	225.0

GET /api/v2/project/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	7.8	8.2	0.95	0.1	0.1
power	7.6	7.8	0.97	0.1	0.1
dataset	7.8	7.8	1.00	0.1	0.1
project	9.5	9.7	0.98	2.3	2.3
job	7.7	8.0	0.96	0.1	0.1

GET /api/v2/task/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	6.3	6.4	0.98	0.8	0.7
power	6.2	6.4	0.97	0.8	0.7
dataset	6.6	6.7	0.99	0.8	0.7
project	6.4	6.3	1.02	0.8	0.7
job	6.5	6.3	1.03	0.8	0.7

GET /api/v2/workflow/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	11.8	12.2	0.97	0.8	0.8
power	25.2	24.8	1.02	29.6	27.3
dataset	19.5	19.1	1.02	17.0	15.9
project	22.6	23.6	0.96	21.4	19.9
job	13.7	12.1	1.13	0.8	0.8

POST /api/v2/project/3/dataset/407/images/query/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
dataset	145.6	151.6	0.96	275.1	275.1

GET /auth/current-user/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
vanilla	5.0	5.2	0.96	0.2	0.2
power	4.9	4.9	1.00	0.2	0.2
dataset	4.9	5.1	0.96	0.2	0.2
project	5.0	5.1	0.98	0.2	0.2
job	5.0	5.0	1.00	0.1	0.1

POST /auth/token/login/

User	Time current (ms)	Time `main` (ms)	Ratio current/`main`	Size current (Kb)	Size `main` (Kb)
dataset	263.1	264.8	0.99	0.2	0.2

github-actions · 2024-10-07T09:41:45Z

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
fractal_server/app/models/v2
task.py	$This PR adds 11 to the number of statements in fractal_server/app/models/v2/task.py, taking it from 37 to 48.$	$This PR doesn't change the number of statements missing coverage in fractal_server/app/models/v2/task.py, which is 0.$	$This PR doesn't change the coverage rate in fractal_server/app/models/v2/task.py, which is 100% (48/48).$	$In this PR, 14 new statements are added to fractal_server/app/models/v2/task.py, 14 of which are covered (100%).$
fractal_server/app/routes/api/v2
task.py	$This PR adds 7 to the number of statements in fractal_server/app/routes/api/v2/task.py, taking it from 96 to 103.$	$This PR adds 3 to the number of statements missing coverage in fractal_server/app/routes/api/v2/task.py, taking it from 1 to 4.$	$This PR removes 3.90 percentage points from the coverage rate in fractal_server/app/routes/api/v2/task.py, taking it from 98.52% (95/96) to 94.63% (99/103).$	$In this PR, 7 new statements are added to fractal_server/app/routes/api/v2/task.py, 4 of which are covered (57.14%).$	65, 67, 70
task_collection_custom.py	$This PR adds 1 to the number of statements in fractal_server/app/routes/api/v2/task_collection_custom.py, taking it from 76 to 77.$	$This PR doesn't change the number of statements missing coverage in fractal_server/app/routes/api/v2/task_collection_custom.py, which is 2.$	$This PR adds 0.05 percentage points to the coverage rate in fractal_server/app/routes/api/v2/task_collection_custom.py, taking it from 94.89% (74/76) to 94.94% (75/77).$	$In this PR, 1 new statements are added to fractal_server/app/routes/api/v2/task_collection_custom.py, 1 of which are covered (100%).$
fractal_server/app/schemas/v2
manifest.py	$This PR adds 6 to the number of statements in fractal_server/app/schemas/v2/manifest.py, taking it from 63 to 69.$	$This PR doesn't change the number of statements missing coverage in fractal_server/app/schemas/v2/manifest.py, which is 0.$	$This PR doesn't change the coverage rate in fractal_server/app/schemas/v2/manifest.py, which is 100% (69/69).$	$In this PR, 7 new statements are added to fractal_server/app/schemas/v2/manifest.py, 7 of which are covered (100%).$
task.py	$This PR adds 27 to the number of statements in fractal_server/app/schemas/v2/task.py, taking it from 92 to 119.$	$This PR adds 4 to the number of statements missing coverage in fractal_server/app/schemas/v2/task.py, taking it from 0 to 4.$	$This PR removes 5.04 percentage points from the coverage rate in fractal_server/app/schemas/v2/task.py, taking it from 100% (92/92) to 94.96% (115/119).$	$In this PR, 52 new statements are added to fractal_server/app/schemas/v2/task.py, 48 of which are covered (92.3%).$	105, 183-185
task_group.py	$This PR adds 17 to the number of statements in fractal_server/app/schemas/v2/task_group.py, taking it from 20 to 37.$	$This PR doesn't change the number of statements missing coverage in fractal_server/app/schemas/v2/task_group.py, which is 0.$	$This PR doesn't change the coverage rate in fractal_server/app/schemas/v2/task_group.py, which is 100% (37/37).$	$In this PR, 18 new statements are added to fractal_server/app/schemas/v2/task_group.py, 18 of which are covered (100%).$
fractal_server/tasks/v2
background_operations.py	$This PR adds 4 to the number of statements in fractal_server/tasks/v2/background_operations.py, taking it from 129 to 133.$	$This PR doesn't change the number of statements missing coverage in fractal_server/tasks/v2/background_operations.py, which is 0.$	$This PR doesn't change the coverage rate in fractal_server/tasks/v2/background_operations.py, which is 100% (133/133).$	$In this PR, 4 new statements are added to fractal_server/tasks/v2/background_operations.py, 4 of which are covered (100%).$
background_operations_ssh.py	$This PR adds 4 to the number of statements in fractal_server/tasks/v2/background_operations_ssh.py, taking it from 138 to 142.$	$This PR adds 1 to the number of statements missing coverage in fractal_server/tasks/v2/background_operations_ssh.py, taking it from 8 to 9.$	$This PR removes 0.88 percentage points from the coverage rate in fractal_server/tasks/v2/background_operations_ssh.py, taking it from 92.26% (130/138) to 91.37% (133/142).$	$In this PR, 4 new statements are added to fractal_server/tasks/v2/background_operations_ssh.py, 3 of which are covered (75%).$	325
Project Total

_{This report was generated by python-coverage-comment-action}

…taskv2-attributes' into 1828-introduce-new-taskv2-attributes

fractal_server/app/schemas/v2/task.py

fractal_server/app/routes/api/v2/task.py

tcompa · 2024-10-07T14:11:06Z

fractal_server/app/routes/api/v2/task.py

+                for author in authors
+            )
+        ]
+    if tags is not None:


Let's postpone implementing this query, so that we can take our time to learn aboute sqlalchemy/sqlite/postgres support for JSON queries.
(please open a new issue)

…taskv2-attributes' into 1828-introduce-new-taskv2-attributes

ychiucco added 2 commits October 7, 2024 11:12

new columns to TaskV2

77f4d09

schemas

6947f8e

ychiucco linked an issue Oct 7, 2024 that may be closed by this pull request

Introduce new TaskV2 attributes #1828

Open

9 tasks

tcompa and others added 5 commits October 7, 2024 13:27

Merge branch 'main' into 1828-introduce-new-taskv2-attributes

6e84461

update manifest v2

ae5ef95

remove TaskV2.authors

480a369

taskgroup new model and schemas

c73526b

Merge remote-tracking branch 'refs/remotes/origin/1828-introduce-new-…

92bc3f3

…taskv2-attributes' into 1828-introduce-new-taskv2-attributes

tcompa reviewed Oct 7, 2024

View reviewed changes

fractal_server/app/schemas/v2/task.py Outdated Show resolved Hide resolved

tcompa reviewed Oct 7, 2024

View reviewed changes

fractal_server/app/schemas/v2/task.py Outdated Show resolved Hide resolved

tcompa reviewed Oct 7, 2024

View reviewed changes

fractal_server/app/schemas/v2/task.py Outdated Show resolved Hide resolved

ychiucco added 5 commits October 7, 2024 15:19

add authors to ManifestV2

5cdcad2

fix validators

41a206d

authors not nullable

39501ee

fix typo [skip ci]

c072245

query params in GET task

1b0cf70

tcompa reviewed Oct 7, 2024

View reviewed changes

fractal_server/app/routes/api/v2/task.py Outdated Show resolved Hide resolved

tcompa reviewed Oct 7, 2024

View reviewed changes

tcompa and others added 10 commits October 7, 2024 16:20

Merge branch 'main' into 1828-introduce-new-taskv2-attributes

30dc2bc

Merge branch 'main' into 1828-introduce-new-taskv2-attributes

39de8b1

Merge remote-tracking branch 'refs/remotes/origin/1828-introduce-new-…

04343c2

…taskv2-attributes' into 1828-introduce-new-taskv2-attributes

authors as string

31f8c9d

authors as string always

da8cec5

authors is Optional[str]

48bffe4

origin and package name for single task collection + fixture

a4358fb

Set more attributes during standard task collection

fb6cf2b

Update manifest_v2.json

d5c548a

Setup some task-group attributes in custom task collection

e1e6a3d

tcompa and others added 2 commits October 7, 2024 17:42

Fix TaskGroupCreateV2

57b8935

origin and pkg_name to mock

01e54d1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New `TaskV2` and `TaskGroupV2` attributes #1856

New `TaskV2` and `TaskGroupV2` attributes #1856

ychiucco commented Oct 7, 2024

github-actions bot commented Oct 7, 2024 •

edited

Loading

github-actions bot commented Oct 7, 2024 •

edited

Loading

tcompa Oct 7, 2024

New TaskV2 and TaskGroupV2 attributes #1856

Are you sure you want to change the base?

New TaskV2 and TaskGroupV2 attributes #1856

Conversation

ychiucco commented Oct 7, 2024

Checklist before merging

github-actions bot commented Oct 7, 2024 • edited Loading

Benchmark comparison

GET /api/alive/

GET /api/v2/dataset/

GET /api/v2/job/

GET /api/v2/project/

GET /api/v2/task/

GET /api/v2/workflow/

POST /api/v2/project/3/dataset/407/images/query/

GET /auth/current-user/

POST /auth/token/login/

github-actions bot commented Oct 7, 2024 • edited Loading

Coverage report

tcompa Oct 7, 2024

Choose a reason for hiding this comment

New `TaskV2` and `TaskGroupV2` attributes #1856

New `TaskV2` and `TaskGroupV2` attributes #1856

github-actions bot commented Oct 7, 2024 •

edited

Loading

github-actions bot commented Oct 7, 2024 •

edited

Loading