zach help

.github/workflows/ci.yml

@@ -17,17 +17,17 @@ jobs:
     strategy:
       matrix:
         monet: [cf, dev]
-      fail-fast: false  # just until cf versions are working
+      fail-fast: false  # always both
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python (micromamba)
-        uses: mamba-org/provision-with-micromamba@v15
+        uses: mamba-org/setup-micromamba@v1
         with:
           environment-file: docs/environment-docs.yml
-          cache-env: true
-          extra-specs: |
+          cache-environment: true
+          create-args: >-
             pytest
 
       - if: ${{ matrix.monet == 'dev' }}
@@ -53,8 +53,7 @@ jobs:
           cd docs/examples
           for f in *.ipynb; do
             if [ "$f" == 'idealized.ipynb' ]; then
-              jupytext --to py $f -o t.py
-              python t.py
+              jupytext --to py $f -o t.py && python t.py || exit 1
             fi
           done
           cd -
@@ -96,13 +95,13 @@ jobs:
         shell: bash -l {0}
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python (micromamba)
-        uses: mamba-org/provision-with-micromamba@v15
+        uses: mamba-org/setup-micromamba@v1
         with:
           environment-file: docs/environment-docs.yml
-          cache-env: true
+          cache-environment: true
 
       - name: linkcheck
         run: sphinx-build -b linkcheck docs docs/_build/linkcheck
@@ -125,7 +124,7 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Check that .py files have the license header
         run: python3 ci/check-for-license-header.py

docs/appendix/machine-specific-install.rst

@@ -1,11 +1,11 @@
 Machine-specific Install
 ========================
 
-NCAR HPC Cheyenne/Casper
+NCAR HPC Derecho/Casper
 ------------------------
 
 Below are specific recipes for getting started with MELODIES MONET
-on the NCAR HPC, Cheyenne/Casper.
+on the NCAR HPC, Derecho/Casper.
 
 **Official NCAR JupyterHub kernel**
 

docs/appendix/troubleshooting.rst

@@ -0,0 +1,18 @@
+Troubleshooting
+===============
+
+Installation problems
+---------------------
+* Conda installation fails:
+    * Often the problem is in the installation of wrf-python. Check that your computer does not have an Apple Silicon CPU (Apple Intel should be fine) and that the Python version is compatible with the wrf-python Conda package.
+
+Runtime issues
+--------------
+* Pairing (:meth:`~melodies_monet.driver.analysis.pair_models`) takes too long:
+    * ``analysis.pair_models()`` is one of the most expensive functions in MELODIES MONET, and you might be running out of memory. If you have access to more RAM, try it with that. A Time Chunking functionality is being developed to deal with this issue. Check :doc:`/users_guide/time_chunking`.
+* The plots are not produced. The error message references ``leg.legendHandles``:
+    * You are probably using Matplotlib 3.9+ with pandas 1.x. Downgrade Matplotlib to 3.8 (upgrading pandas should also work, but you might run into some incompatibilities for some specific functionalities, especially those related to downloading observational data with MONETIO. Check :doc:`/getting_started/installation`).
+* NaN does not exist, or pandas not detecting NaN values, etc:
+    * In NumPy 2.x, ``np.NaN`` has been completely replaced by ``np.nan``. ``np.NaN`` is required by some versions of MONETIO. There are two solutions: (i) downgrading NumPy to previous versions (``'numpy<2'``) or (ii) upgrading MONETIO (``'monetio>=0.2.7'``). If you go with option (ii), you might need to use the _develop_ branch of MONETIO (this should make it into _stable_ soon).
+* Failure downoloading maps:
+    * MELODIES-MONET uses Cartopy for mapping. Cartopy downloads shapefiles automatically to create the maps, and if you are offline or working on a machine with download restrictions, this might fail. Check :doc:`/appendix/machine-specific-install`. This is the situation for *NOAA HPC Hera*, and the solution discussed there should work.

docs/appendix/yaml.rst

@@ -185,7 +185,7 @@ Generalizing this to include other surface observations is under development.
 **filename:**  The file directory location and name. These observations need 
 to be preprocessed prior to incorporating them into MELODIES MONET.
 Shell variables prefixed with the ``$`` symbol, such as ``$HOME``, will be expanded.
-See :doc:`../tutorial/downloading_obs` for more details.
+See :doc:`../getting_started/downloading_obs` for more details.
 
 **obs_type:** The observation type. Options are: "pt_sfc" or point surface. Adding 
 options for Aircraft and Satellite observations are under development.
@@ -231,7 +231,7 @@ nan_values are set to NaN first and then the unit conversion is applied.
      (in %) is used to calculate the percentile (e.g., 5, 50, 95). Currently only
      used for "spatial_bias" plots. Will work with data as is and regulatory metrics.
    * **regulatory:** If false (default), use data as is. If set to true, the
-     regulatory metric is calculated as explained under :doc:`/background/supported_analyses`.
+     regulatory metric is calculated as explained under :doc:`/users_guide/supported_diagnostics`.
      Only works for "OZONE" and "PM2.5" variables.
    * **ylabel_reg_plot:** String to use as ylabel in plot for regulatory calculation.
      Useful for adding units or instrument information. Only used if regulatory = True.
@@ -276,7 +276,7 @@ Plots
 -----
 All input for each plotting group. A plotting group consists of one plotting 
 type. The plotting types are described in 
-:doc:`/background/supported_plots`. All model /
+:doc:`/users_guide/supported_plots`. All model /
 observational pairs and domains specified for the plotting group will be 
 included. You may include as many plotting groups as you like.
 
@@ -317,6 +317,10 @@ For example, ::
 **domain_type:** List of domain types to be plotted. These correspond with
 the columns in the observation file. (e.g., airnow: epa_region, state_name, 
 siteid, etc.).
+For automatic EPA or Giorgi region boxes (if they are not included
+with the columns in the observation file), choose ``auto-region:epa`` or
+``auto-region:giorgi``. Take into account that ``auto-region:epa`` is only a rough
+approximation, since it assumes perfect, rectangular lonlat boxes.
 
 **domain_name:** List of domain names to be plotted. If domain_type = all, all 
 data will be used and the domain_name is used only in the plot title. If 
@@ -397,7 +401,7 @@ Stats
 -----
 All input needed to calculate the statistics. The supported statistics available 
 in MELODIES MONET are described in 
-:doc:`/background/supported_stats`. All model /
+:doc:`/users_guide/supported_stats`. All model /
 observational pairs and domains specified will be included. You may include as 
 many statistics as you like. Note however that the calculation of the statistics 
 is relatively slow right now. Optimizing this code is under development.
@@ -407,7 +411,7 @@ use Kelvin. Wind direction has special calculations for AirNow if the observatio
 name is 'WD'. 
 
 **stat_list:** List of acronyms of statistics to calculate as defined in 
-:doc:`/background/supported_stats`. (e.g., ['MB', 'MdnB',
+:doc:`/users_guide/supported_stats`. (e.g., ['MB', 'MdnB',
 'NMB', 'NMdnB','R2', 'RMSE']). A dictionary of definitions is also included in 
 MELODIES-MONET/melodies_monet/stats/proc_stats.py. 
 

docs/conf.py

@@ -40,8 +40,8 @@
 ]
 
 extlinks = {
-    'issue': ('https://github.com/noaa-csl/melodies-monet/issues/%s', 'GH'),
-    'pull': ('https://github.com/noaa-csl/melodies-monet/pull/%s', 'PR'),
+    'issue': ('https://github.com/noaa-csl/melodies-monet/issues/%s', 'GH %s'),
+    'pull': ('https://github.com/noaa-csl/melodies-monet/pull/%s', 'PR %s'),
 }
 
 autosummary_generate = True  # default in Sphinx v4
@@ -88,7 +88,7 @@
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = 'en'
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.

docs/develop/contribute.rst

@@ -11,8 +11,8 @@ the contributions and support from you.
 - Please check out 
   `GitHub Projects <https://github.com/NOAA-CSL/MELODIES-MONET/projects>`__ 
   to learn about current development plans.
-- Join an :ref:`Upcoming Workshop <develop/workshops:Upcoming>`
-  or check out :ref:`Past Workshops <develop/workshops:Past>`.
+- Join an :ref:`Upcoming Workshop <develop/other_resources:Upcoming>`
+  or check out :ref:`Past Workshops <develop/other_resources:Past>`.
 - See our developers guide, to learn 
   :ref:`develop/developers_guide:How to incorporate updates to MELODIES MONET`.
 - Email melodies@ucar.edu with questions.

docs/develop/datasets.rst

@@ -21,3 +21,58 @@ While a part of the MONETIO repository,
 the private MELODIES MONET readers are designated with prefix ``_mm``.
 
 Support for additional models is also under developed.
+
+Standard variables are required to be computed in each model reader for each capability including surface, aircraft, and satellite as specified in the table below.
+
+.. list-table:: Required Variables for Model Readers
+   :widths: 10 30 30 30
+   :header-rows: 1
+
+   * - Capability
+     - | Variable Name 
+       | in Code
+     - Description
+     - Additional Requirements
+   * - Surface
+     - | ``time``
+       | ``latitude``
+       | ``longitude``
+     - | Time in ``datetime64[ns]`` format
+       | Latitude in degrees
+       | Longitude in degrees
+     - | Provide only surface model data 
+       | or if provide vertical model data, 
+       | first level must be the level 
+       | nearest to the surface.
+       | All gases are in ppb and 
+       | all aerosols are in µg/m3.
+   * - Aircraft
+     - | ``time``
+       | ``latitude``
+       | ``longitude``
+       | ``pres_pa_mid``
+       | ``temperature_k``
+     - | Time in ``datetime64[ns]`` format
+       | Latitude in degrees
+       | Longitude in degrees
+       | Mid-level pressure in pascals (Pa)
+       | Mid-level temperature in kelvin (K)
+     - | Provide vertical model data. 
+       | All gases are in ppb and 
+       | all aerosols are in µg/m3.
+   * - Satellites
+     - | ``time``
+       | ``latitude``
+       | ``longitude``
+       | ``pres_pa_mid``
+       | ``temperature_k``
+       | ``dz_m``
+       | ``surfpres_pa``
+     - | Time in ``datetime64[ns]`` format
+       | Latitude in degrees
+       | Longitude in degrees
+       | Mid-level pressure in pascals (Pa)
+       | Mid-level temperature in kelvin (K)
+       | Layer thickness in meters (m)
+       | Surface pressure in pascals (Pa)
+     - | Provide vertical model data.

docs/develop/developers_guide.rst

@@ -26,26 +26,6 @@ via pull requests.
 Forks of the central repository should follow the same
 branching conventions.
 
-develop_aircraft
-________________
-The *develop_aircraft* branch is the place
-for adding new features related to aircraft observations.
-
-develop_satellite
-_________________
-The *develop_satellite* branch is intended
-for adding support for reading and processing
-satellite observations.
-This includes both gridded satellite data products
-and swath or geostationary disk data products.
-Sub-branches are sometimes created off of *develop_satellite*,
-such as *develop_satellite_swath*.
-
-develop_testsuite
-_________________
-The *develop_testsuite* branch holds modules
-under development for unit tests and regression tests.
-
 .. _dev-install-instructions:
 
 Setting up your development environment
@@ -55,10 +35,10 @@ In order to prepare for developing MELODIES MONET,
 we clone the repositories and create a conda environment that references them.
 
 .. note::
-   If you are installing MELODIES MONET on NCAR Cheyenne or NOAA Hera
+   If you are installing MELODIES MONET on NCAR Casper or NOAA Hera
    follow these machine specific instructions instead.
 
-   - :ref:`NCAR Cheyenne <appendix/machine-specific-install:NCAR HPC Cheyenne/Casper>`
+   - :ref:`NCAR Casper <appendix/machine-specific-install:NCAR HPC Derecho/Casper>`
    - :ref:`NOAA Hera <appendix/machine-specific-install:NOAA HPC Hera>`
 
 .. important::
@@ -74,7 +54,7 @@ these instructions:
 
        $ conda create --name melodies-monet python=3.9
        $ conda activate melodies-monet
-       $ conda install -y -c conda-forge pyyaml pandas=1 monet monetio netcdf4 wrf-python typer rich pooch jupyterlab
+       $ conda install -y -c conda-forge pyyaml pandas=1 'matplotlib-base<3.9' monet monetio netcdf4 wrf-python typer rich pooch jupyterlab
 
 (b) Clone [#clone]_ and link the latest development versions of MONET and MONETIO from GitHub to
     your conda environment::
@@ -140,8 +120,8 @@ changes.
     changes. 
 
 
-User's Guide Development
-------------------------
+Contributions to the Docs
+-------------------------
 
 If you add a component to MELODIES MONET, please follow the instructions below 
 to update the readthedocs user guide. 
@@ -172,7 +152,7 @@ The generated HTML will be created in ``docs/_build/html``,
 with ``docs/_build/html/index.html`` the main page that can be
 viewed in any browser.
 
-Please see the `Documentation <https://github.com/NOAA-CSL/MELODIES-MONET/projects/2>`_ 
+Please see the `MELODIES-MONET Documentation <https://github.com/orgs/NOAA-CSL/projects/6>`_ 
 project on GitHub to learn about current and future development.
 
 

docs/develop/workshops.rst → docs/develop/other_resources.rst

@@ -1,5 +1,5 @@
-Tutorials & Workshops
-=====================
+Other Resources
+===============
 
 Check out our upcoming and past tutorials, workshops, surveys, etc. below.
 

docs/environment-docs.yml

@@ -7,6 +7,7 @@ dependencies:
   - python=3.9
   #
   # melodies_monet deps
+  - matplotlib-base<3.9  # for pandas v1 compat
   - monet
   - monetio
   - netcdf4