Session Level Pheno

[surchs]

Context

We have a good amount of data in our nodes and are starting to also host richer data. Many of the richer data have at least some longitudinal information on phenotypic data - and we cannot model this because we model phenotypic information at the subject level. This will allow us to provide much richer queries in the near future and is, from the tool and data model side, a relatively easy thing to do.

There is also BEP36 making its way towards completion, and we want to both have some ideas on that and be able to model BEP36 information once it merges.

The overall design ideas live here: https://miro.com/app/board/uXjVNIzFV9w=/

IMportant stuff

• right now the CLI can handle multiple sessions (rows about the same subject), but it just picks the first value

Why

• Part of Grow the amount and usefulness of data #11
• Follows Deploy on Release #64
• We want to model longitudinal data so we can represent richer information and query them
• So we can get more value from our own (PD) datasets locally
• To get ready for such modeling challenges as QPN where imaging and pheno sessions can be completely separate

Outcomes

• our data model schema supports age, sex, diagnosis, assessment at the session level
• all (of our own) graphs are running up to date data models (no more subject level pheno)
• I can search for a subject with 2 phenotypic sessions and no imaging
• I can annotate (annotation tool) and process (CLI) a table with session ID columns

What

CLI

• check schema compatibility for data dictionary with the Session ID bagel-cli#257
• Model phenotypic information at session level bagel-cli#225
• figure out if "session id" should be optional or not

Annotation tool

• can annotate "Session ID" as phenotypic session (but can be empty if only for the purpose of listing the imaging session) annotation_tool#663

"Work"

• [ ] update the data dictionaries for everything in OpenNeuro (because that schema has changed)
• [ ] [ENH] Make text associated with checkbox clickable query-tool#58
• Reprocess all Neurobagel-hosted datasets #107

API

• update query template to work with new session-level phenotypic data model api#249
• Decide whether "session_type" in API response should be a URI or human-readable api#262
• Update num_{imaging,phenotypic}_sessions to be num_matching_{imaging,phenotypic}_sessions api#271

Federation API

• Update query model with phenotypic/imaging session distinction federation-api#60

Query tool

• add a tooltip to explain how we "match" sessions #86
• Enable pheno level session for response and query query-tool#78

Documentation-related

• Update example participant-level-results.tsv following phenotypic + imaging session separation neurobagel_examples#26
• Update expected columns in participant-level-results.tsv based on new session-level data model documentation#157

What it is not

• we do not deal with session timing information (i.e. imaging and pheno happening within X days of each other)
• we do not support multiple session naming systems, if you have an imaging session, it has to be named in the big input .tsv
• allowing multiple columns "about" age or "sex" (different values for different sessions have to still be encoded longform)
• any information on ordering of sessions or "baseline" sessions

To close it all out

• Make coordinated releases across all tools #135

[rmanaem]

Since there were no issues open on the annotation tool and query tool related to this roadmap, I'm turning the checkboxes into issues for both tools.

[alyssadai]

Closing as all issues have been addressed! 🥳