Slurp pipeline: Initial design

[matentzn]

Updates

commit e5575265457308e40626870dbdee9de791cdcac0

    Slurp pipeline
    - Update: direct_owned_parents methods: Temporarily commented out, pending OAK fixes. Currently doing this in batch using SPARQL for now.
    - Add: Param --onto-exclusions-path: Now we no longer consider exluded terms to be possible slurp candidates.
    - Bugfix: Labels: Now they are successfully being fetched using OAK.
    - Add: New columns to output

    Exclusions
    - Bugfix: Sometimes, when exclude_children was left empty in input config/%_exclusions.tsv, yielded empty generated exclusion tables.
    - Update: Updated exclusion tables for: DOID, OMIM. These had problematic source files with no values for exclude_children, and needed to be re-run after bugfix.

    Misc
    - Update: run.sh: Reverted back to what it was before, undoing deletion of --rf, which was done to address pip installs not being persistent within the ODK docker container between runs.

commit 5fd00c4d3eaa5741b1d2c229166a686ac4000076

    Slurp pipeline
    - Delete: Initialized a variables for 'all relationships' and 'term parent map', but did a more OAK-reliant refactor and removed these.
    - Add: Term class
    - Update: Big refactor to utilize Term class
    - Add: Slurp output TSV files
    - Add: utils.py, which includes Term class
    - Add: New param: max_id

    Misc
    - Update: run.sh: This change will allow any Python package updates to be retained.
    - Add: SPARQL jinja query to get parents. Used by slurp pipeline.

commit d3e1c743e15f0e76a060793fc075e799041561cf

Slurp pipeline
- Delete: Initialized a variables for 'all relationships' and 'term parent map', but did a more OAK-reliant refactor and removed these.

commit f50d5809e71cb0bb02147e1568ab443bb0ee7438

Slurp pipeline
- Delete: `mondo-ingest.Makefile`: (i) `python-install-dependencies` phony target. This has been removed in favor of expediently updating the dependencies in ODK directly. (ii) Removed unused, commented out mondo component target.
- Update: `.gitignore`: Simplified to ignoring whole dir `src/ontology/components/*`
- Update: Python requirements: Removed `semsql` w/ specific version, due to some recent dependency fixes in OAK.

Term exclusions
- Update: Rename: reports/term_exclusions.txt -> reports/excluded_terms.txt

commit e5fab07c1638f8055a5f73396aa38f575675980a

Some fixes to make files

1. Never depend on a PHONY goal, because they force the whole pipeline to be rebuild - just add them to each goal, installing a few python dependencies that are already installed is fast enough.
2. PHONY goals cannot have wildcard dependencies: https://stackoverflow.com/questions/7887343/makefile-wildcard-static-rule-with-phony
3. Indentation is critical with Make targets, your `# TODO: Temporary debugging:` was wrongly intented to the level of the target name, not the body.

commit 1e7929bdef740e55223b4b32cb116ca5cbe800d4
Slurp pipeline
- Add: makefile: Missing make goals for dependencies for slurp goal.
- Add: Python: CLI
- Update: makefile: Slurp goal: (i) named keys/vals for all params, (ii) standardization in file/path params.

(before above commits)
- Add: makefile goals
- Add: initial python script

Related to

• Aim 2.1.2 Maintain high-quality equivalence mappings, working with the community to resolve upstream issues. phenomics_first_resource#24

[joeflack4]

@matentzn This afternoon, did: (1) adding of genes to OMIM exclusions, re-ran exclusions pipeline and slurp, (2) re-ran slurp for all ontologies, following our addition/changes to columns.

As you suggested, gonna merge this now. I imagine I'll be making continuous tweaks to the slurp pipeline very soon, but I'll make new (hopefully a lot smaller) PRs for that.