diff --git a/best-practices.md b/best-practices.md index 5d0e763fe..bd8bfe108 100644 --- a/best-practices.md +++ b/best-practices.md @@ -26,4 +26,4 @@ A group of researchers and software engineers put together a helpful guide for d ### [CISL Best Practices](https://www2.cisl.ucar.edu/user-support/cisl-best-practices) -If you are using NCAR Computational and Information Systems Lab (CISL) resources such as Casper/Cheyenne, this is a great place to start with learning more making the most of your computing and storage allocations. +If you are using NCAR Computational and Information Systems Lab (CISL) resources such as Casper/Cheyenne, this is a great place to start with learning to make the most of your computing and storage allocations. diff --git a/calendar.md b/calendar.md deleted file mode 100644 index 348bfbf13..000000000 --- a/calendar.md +++ /dev/null @@ -1,34 +0,0 @@ -# Calendar - -Currently, we have a few different ways to give involved with the ESDS effort. - -## ESDS Activities - -### ESDS Forum - -Every other Monday, we hold the ESDS Forum, which provides a venue for discussion, coordination and "Work In Progress" (WIP) talks. -Examples include: - -- Workflow demos -- Overview of some package -- Focused discussions on some new effort - -This is a great place to discuss progress on your project, and get feedback from the community! - -If you are interested in giving a talk, be sure to sign up using [this spreadsheet](https://docs.google.com/spreadsheets/d/1Tuv5vwGbjSKdJD5KLT0Zow8WCY0_zt35tTI6EIRGOk4/edit?usp=sharing)! - -### Office Hours - -Have a question about Python issues? Looking to get feedback on some project? [Schedule an office hours appointment](office-hours)! - -If you would like join the esds@ucar.edu Google group to be informed of updates and events about these meetings and question/answer opportunties, please do so by following [this link](https://groups.google.com/a/ucar.edu/g/esds/about?pli=1) and clicking "Join Group" next to the group name. - - - -## Python Training - -The NCAR Python Tutorial Series is a biweekly tutorial series, detailing various aspects of the Python ecosystem! Be sure to check out these tutorials! - -If you would like join the learnpython@ucar.edu Google group to be informed of updates and events in the seminar series, please do so by following [this link](https://groups.google.com/a/ucar.edu/g/learnpython/about?pli=1) and clicking "Join Group" next to the group name. - - diff --git a/communication.md b/communication.md index abdaacad2..52d846bfe 100644 --- a/communication.md +++ b/communication.md @@ -1,31 +1,26 @@ -# Communication +# Communication, Meetings, and Resources -ESDS uses several primary platforms for communication. Here is a list of communication platforms and learning resources if you'd like to be involved in the ESDS community: +ESDS has a few primary [communication platforms](#communication-platforms), [community meetings](#community-meetings), and [learning resources](#learning-resources) if you'd like to be involved in the ESDS community: -## [NSA Discourse](https://discourse-nsa.ucar.edu/) +## Communication Platforms -Discourse offers a community where users can ask questions and share progress on various projects. It is setup similarly to the [Pangeo discourse](https://discourse.pangeo.io/). You will need to register for an account, but once registered and logged on, be sure to follow the ESDS channel on this platform. +### [Email List](https://groups.google.com/a/ucar.edu/g/esds) -## [Email List](https://groups.google.com/a/ucar.edu/g/esds) +If you would like to join the ESDS Email List to be informed of community updates, we recommend you do so by joining the ESDS Google Group using [this link to the ESDS Group page](https://groups.google.com/a/ucar.edu/g/esds), then click "Join Group." -If you would like to join the ESDS Email List (Google Group) to be informed of updates about these meetings and question/answer opportunities, we recommend you join the ESDS Google Goup using [this link to the ESDS Group page](https://groups.google.com/a/ucar.edu/g/esds), then click "Join Group." +### [Zulip](https://zulip.ucar.edu/) -Joining the Google Group will automatically send you a Zulip invitation! +Zulip is an asynchronous messaging platform, run in-house at UCAR, which can be used for both private messages and messages separated by topic. When posting in a specific "channel", the user must add a topic, which makes it easier to track various conversations and refer back to communications at a later time. Channels include “ESDS”, “dask”, “python-questions”, “xarray”, “jupyterlab-hub”, and more. You can join with this [Zulip invite link](https://zulip2.cloud.ucar.edu/join/w5kv3jauplyepqndapyos7eh/). -## [Zulip](https://zulip.ucar.edu/) +### [ESDS GitHub Repository](https://github.com/ncar/esds) -Zulip is an asynchronous messaging platform, run in-house at UCAR, which -can be used for both private messages and messages seperated by topic. When posting in a specific "channel", the user must add a topic, which makes it easier to track various conversations and refer back to communications at a later time. Channels include “ESDS”, “dask”, “python-questions”, “python-dev”, “jupyterlab-hub” and more. +This repository and webpage is a space where we can post progress on this initiative, publish blog posts related to various workflows, document "best practices," in addition to serving as a resource/discussion spot via comments on pages and posts. Please feel free to create new 'issues' or make PRs to contribute to this repository. -Again, we recommend you join the ESDS Google Goup using [this link to the ESDS Group page](https://groups.google.com/a/ucar.edu/g/esds), then click "Join Group." New members to this Google Group will receive an invitation to join Zulip as well! Alternatively, this [Zulip invite link](https://zulip2.cloud.ucar.edu/join/w5kv3jauplyepqndapyos7eh/) does not expire. +## Community Meetings -## [ESDS Github Repository](https://github.com/ncar/esds) +### [ESDS Forum](https://docs.google.com/document/d/e/2PACX-1vQeHIGSSz_8A8gZVL87xDjYXEwqB4CkRk85yf0TACb-rVgubjb3ukiulEYuUwHZGVXhgYNpaRC2SNAt/pub) -This repository and webpage is a space where we can post progress on this initiative, publish blog posts related to various workflows, document "best practices," in addition to serving as a resource/discussion spot via comments on pages and posts. - -## [ESDS Forum](https://docs.google.com/document/d/e/2PACX-1vQeHIGSSz_8A8gZVL87xDjYXEwqB4CkRk85yf0TACb-rVgubjb3ukiulEYuUwHZGVXhgYNpaRC2SNAt/pub) - -Every other Monday from 2-3pm MT, we hold the ESDS Forum over Google Meet, which provides a venue for discussion, coordination and showcasing work in progress. Examples include: +Every other Monday from 2-3pm MT, we hold the ESDS Forum over [Google Meet](meet.google.com/uya-vqhi-twg), which provides a venue for discussion, coordination and showcasing work in progress. Examples include: - Workflow demos - Overview of packages and analysis tools @@ -38,14 +33,9 @@ If you are interested in giving a talk or leading a discussion, please fill out See the [ESDS Forum Meeting Notes](https://docs.google.com/document/d/e/2PACX-1vQeHIGSSz_8A8gZVL87xDjYXEwqB4CkRk85yf0TACb-rVgubjb3ukiulEYuUwHZGVXhgYNpaRC2SNAt/pub) for an overview of previous discussions. -## Analysis Workflow Special Interest Group (AWSIG) +### Analysis Workflow Special Interest Group (AWSIG) -The AWSIG provides a communication forum for the exchange of ideas -and experiences related to the operational aspects of CISL supported -**analysis** resources and services (e.g. Casper, JupyterHub, -GLADE, VirtualGL, etc.). This working group complements the more -modeling/simulation focused [NCAR HPC User Group -(NHUG)](https://arc.ucar.edu/knowledge_base/70549847). +The AWSIG provides a communication forum for the exchange of ideas and experiences related to the operational aspects of CISL supported **analysis** resources and services (e.g. Casper, JupyterHub, GLADE, VirtualGL, etc.). This working group complements the more modeling/simulation focused [NCAR HPC User Group (NHUG)](https://arc.ucar.edu/knowledge_base/70549847). The AWSIG Seeks to: @@ -56,45 +46,34 @@ The AWSIG Seeks to: - Suggest new research and technology directions in scientific computing, - Voice user concerns. -### Organization +#### Organization -The AWSIG will hold ~bi-monthly meetings during the ESDS Forum. The -CISL consulting group (CSG) will coordinate meetings with assistance -as needed from the ESDS leadership. Meetings will consist of free -format discussions, and informational presentations from CISL -technical staff and NCAR/UCAR scientific staff. +The AWSIG holds ~bi-monthly meetings during the ESDS Forum time slot. The CISL consulting group (CSG) will coordinate meetings with assistance as needed from the ESDS leadership. Meetings will consist of free format discussions, and informational presentations from CISL technical staff and NCAR/UCAR scientific staff. -CSG will rely on both ESDS leadership and the community to propose -topics of interest, and even to present emerging tools, best -practices, etc. A successful SIG will be driven by interested -stakeholders, as well as from the resource providers in CISL. CSG -will have items to directly contribute, but also do a lot of listening -and then implementing feedback. +CSG will rely on both ESDS leadership and the community to propose topics of interest, and even to present emerging tools, best practices, etc. A successful SIG will be driven by interested stakeholders, as well as from the resource providers in CISL. CSG will have items to directly contribute, but also do a lot of listening and then implementing feedback. -### Membership & Participation +#### Membership & Participation -The AWSIG is open to all users of NCAR’s HPC resources. Representatives -from CISL’s Consulting Group and relevant resource providers will -be in attendance. We desire that representatives from each scientific -lab attend as well. +The AWSIG is open to all users of NCAR’s HPC resources. Representatives from CISL’s Consulting Group and relevant resource providers will be in attendance. We desire that representatives from each scientific lab attend as well. -### Want to suggest a topic for the AWSIG? +### ESDS Activities Calendar -If you'd like to propose a discussion topic, or upvote a suggested topic, -please use the spreadsheet [here](https://docs.google.com/spreadsheets/d/14BT79YxmWOHFVm1kV8TDYMGQvNPjKZvQwq2OHx9zXGY/edit?usp=sharing) +You can add events to your calendar by clicking on an individual event and saving it to your Google Calendar. + + -## [Office Hours](office-hours) +## Learning Resources + +### [Office Hours](office-hours) Have a question about Python/data analysis issues? Looking to get feedback on some project? Set up an "office hours" appointment! -Several ESDS members are prepared to answer your Python questions via 1-on-1 appointments. You can now select the ESDS member with the specialization best equiped for your question, find a time that works for you, and generate an appointment. Please also provide how you'd like to meet (in-person or virtual) and a brief description of your question. +Several ESDS members are prepared to answer your Python questions via 1-on-1 appointments. You can now select the ESDS member with the specialization best equipped for your question, find a time that works for you, and generate an appointment. Please also provide how you'd like to meet (in-person or virtual) and a brief description of your question. -If your question is about an error message you are raising in your workflow, help us help you by being ready to provide a copy of your dataset and code so we can reproduce your error. Depending on your question, we may be able to walk you through a solution or send you documentation/resources that address your question. See the [Office Hour Notes](https://docs.google.com/document/d/1gIK0C-srceYmoYtgoODeLtuPQPL40iAHoI9DXWqfWP0/edit?usp=sharing) for a summary of previous sessions. +If your question is about an error message you are raising in your workflow, help us help you by being ready to provide a copy of your dataset and code so we can reproduce your error. Depending on your question, we may be able to walk you through a solution or send you documentation/resources that address your question. **NOTE:** Office hours are designed for NCAR/UCAR staff. -## [Project Pythia](https://projectpythia.org/) +### [Project Pythia](https://projectpythia.org/) Want to learn more about using Python, GitHub, or Jupyter Notebooks for Earth system data science? Project Pythia is a home for Python-centered learning resources that are open-source, community-owned, geoscience-focused, and high-quality. Check out [Project Pythia](https://projectpythia.org/) tutorials, examples, and other training materials here. Project Pythia shares several contributors with the ESDS community and is a resource we highly recommend. - - diff --git a/faq.md b/faq.md index ca10c8186..fa80396eb 100644 --- a/faq.md +++ b/faq.md @@ -2,11 +2,24 @@ This contains relevant questions and answers from common workflow issues and questions posted on Zulip. -**_This page is meant to be a list of FAQ regarding climate datasets, movivated by a variety of employees across UCAR/NCAR_** +**_This page is meant to be a list of FAQ regarding climate datasets, motivated by a variety of employees across UCAR/NCAR_** -## I need help with this! +## Help Topics: +[Where do I go for help?](#where-do-i-go-for-help) +[What do I do if my question is not answered on this page?](#what-do-i-do-if-my-question-is-not-answered-on-this-page) -### Where do I go for help? +[Someone must have written the function I want. Where do I look?](#someone-must-have-written-the-function-i-want-where-do-i-look) + +[How do I use conda environments?](#how-do-i-use-conda-environments) + +[Xarray and Dask](#xarray-and-dask) + +[Github](#github) + +[CESM Data](#cesm-data) + + +## Where do I go for help? Try one of the following resources. @@ -17,7 +30,7 @@ Try one of the following resources. Avoid personal emails and prefer a public forum. -### What do I do if my question is not answered on this page? +## What do I do if my question is not answered on this page? If your question is related to conda environments and you're affiliated with UCAR/NCAR, you can open a help ticket on the [NCAR Research Computing Helpdesk site](https://ithelp.ucar.edu/plugins/servlet/desk/site/rc). diff --git a/projects.md b/projects.md index 0f5717891..1604e281a 100644 --- a/projects.md +++ b/projects.md @@ -6,7 +6,7 @@ This page includes a list of ongoing projects related to the ESDS initiative! ### [GeoCAT-comp](https://geocat-comp.readthedocs.io/en/latest/) -GeoCAT-comp is computational component of the GeoCAT project. GeoCAT-comp provides implementations of computational functions for operating on geosciences data. Many of these functions originated in NCL and were translated into Python with the help of GeoCAT-comp; however, developers are welcome to come up with novel computational functions for geosciences data. +GeoCAT-comp is a computational component of the GeoCAT project. GeoCAT-comp provides implementations of computational functions for operating on geosciences data. Many of these functions originated in NCL and were translated into Python with the help of GeoCAT-comp; however, developers are welcome to come up with novel computational functions for geosciences data. - [Version 1.0 Roadmap](https://github.com/NCAR/geocat-comp/projects/1) - [Version 1.1 Roadmap](https://github.com/NCAR/geocat-comp/projects/2) @@ -21,7 +21,7 @@ This gallery contains visualization examples from many plotting categories of ge ## Diagnostic Efforts -### [AMP Diagnostics](https://github.com/NCAR/CAM_diagnostics) +### [Atmosphere Diagnostics Framework (ADF)](https://github.com/NCAR/CAM_diagnostics) Python-based diagnostics focused on evaluation of the atmospheric component of CESM. Produces time series and climatology files, seasonal and annual mean maps and zonal averages, global mean statistics, and an optional static website. @@ -33,14 +33,21 @@ A place to put sample workflows and tools that use CTSM model output. Tools to support analysis of POP2-CESM model solutions. +### [Modular Ocean Model (MOM6)-tools](https://github.com/NCAR/mom6-tools) +In development: Tools to support analysis of MOM6-CESM model solutions. + ### [Intake-esm](https://github.com/intake/intake-esm) -An intake plugin for parsing an Earth System Model (ESM) catalog and loading assets into xarray datasets. +An Intake plugin for parsing an Earth System Model (ESM) catalog and loading assets into xarray datasets. ### [Earth system model Catalog Generation Tools (ecgtools)](https://github.com/NCAR/ecgtools) -Tool which enables one to build intake-esm catalogs. +Tool which enables one to build intake-esm catalogs. Note that this is not currently managed but functional and still useful. + +### [Esm_catalog_utils](https://esm-catalog-utils.readthedocs.io/en/latest/) + +Utilities to build catalogs of ESM output that are usable by intake-esm. -### [Funnel](https://github.com/NCAR/esds-funnel) +### [Nbscuid](https://github.com/rmshkv/nbscuid) -Package which enables extensible diagnostic workflows, connecting intake-esm catalogs with operators and derived variables. +In development: package to enable running flexible, extensible notebook-based diagnostics workflows.