Create cluster environment with micromamba
The procedure should work on most of Linux distributions (but not on Mac).
- Create a folder, download the latest version of micromamba, configure and install:
mkdir -p micromamba/root
cd micromamba
wget -qO- https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
export MAMBA_ROOT_PREFIX=${PWD}/root
eval "$(./bin/micromamba shell hook -s posix)"
micromamba activate
micromamba create -n mix3r \
-c conda-forge \
python=3.11 \
conda-pack=0.7.1 \
numpy=1.26.0 \
scipy=1.11.3 \
numba=0.58.1 \
pandas=2.1.1 \
matplotlib-base=3.8.0 \
ipython=8.16.1 \
notebook=7.0.6 \
r=4.3 \
r-essentials=4.3 \
r-irkernel=1.3.2 \
r-eulerr=7.0.0 \
-c numba \
icc_rt=2020.2 \
--yes
- Activate and pack the environment:
micromamba activate mix3r
conda-pack -p root/envs/mix3r -o mix3r.tar.gz
- Then copy
mix3r.tar.gz
to cluster and unpack:
mkdir mix3r
tar -zxf mix3r.tar.gz -C mix3r/
source mix3r/bin/activate
conda-unpack
- Prepare config files (you may use
config_scz_adhd_mig_aug22_1.json
as template) for multiple (e.g. 16) runs with different variant subsets. You may do it by changing"rand_prune_seed"
parameter in the config file, e.g. set"rand_prune_seed" : i
for the i-th run (also don't forget to use different output files for all runs, e.g. for the i-th run set"out" : "my_analysis_run_i.json"
). - Submit jobs for all configs to cluster:
for i in {1..16}; do sbatch run_slurm_int.sh path/to/your_config_${i}.json; done
- When all runs are finished, produce a table with estimated model parameters for all runs:
python extract_p.py --input my_analysis_run_*.json --out my_analysis_all_runs
my_analysis_all_runs.parameters.csv
file will be created.
You can also produce a table for a selection of runs:
python extract_p.py --input my_analysis_run_1.json my_analysis_run_3.json my_analysis_run_7.json --out my_analysis_selected_runs
- Plot "median" Euler diagram:
Rscript make_euler.r my_analysis_all_runs.parameters.csv my_analysis.euler.png "Trait 1" "Trait 2" "Trait 3"
Where the first argument is a path to the file with model parameters produced with extract_p.py
script, the second argument is output file name and the last three arguments are trait labels to be used in the Euler diagram.