39 picontrol

[aradhakrishnanGFDL]

Addresses #39

Catalogs generated from this PR

/home/a1r/github/noaa-gfdl/catalogs/CM4.5v01_om5b06_piC_noBLING.json
/home/a1r/github/noaa-gfdl/catalogs/ESM4.5v01_om5b04_piC.json
/home/a1r/github/noaa-gfdl/catalogs/CM4_piControl_C.json

[aradhakrishnanGFDL]

you can expect to see msgs such as these..(TODO: keep or rm or set log verbose/warning as needed, but as I recollect workflow wanted all of the msgs in stdout)

Skipping river_instant_cubic.009601-010012.LWSr.tile5.nc
Skipping river_instant_cubic.009601-010012.FWSr.tile6.nc
Skipping river_instant_cubic.009601-010012.HSr.tile6.nc
Skipping river_instant_cubic.009601-010012.LWSr.tile6.nc
Skipping .fregrid_remap_file_288_by_180.nc
Skipping .fregrid_remap_file_288_by_180.nc

[aradhakrishnanGFDL]

This is how I tested:

From GFDL workstation

module load conda 
conda activate catalogbuilder 

(GFDL users, feel free use: conda activate /nbhome/Aparna.Radhakrishnan/conda/envs/catalogbuilder )

From the cloned catalogbuilder repository (use the PR branch if not merged into main) 

Go to scripts/

./gen_intake_gfdl.py --config ../../configs/config-CM4.5v01_om5b06_piC_noBLING.yaml      

./gen_intake_gfdl.py --config ../../configs/config-CM4_piControl_C.yaml 
./gen_intake_gfdl.py --config ../../configs/config-ESM4.5v01_om5b04_piC.yaml 

Note that config files are all in the catalogbuilder repository for these test cases.    

[aradhakrishnanGFDL]

supports NOAA-GFDL/MDTF-diagnostics#667

[Ciheim]

Can confirm that each of these works. We may want to consider ignoring hidden files again to help avoid checking files multiple times:

Skipping .fregrid_remap_file_288_by_180.nc
Skipping .fregrid_remap_file_288_by_180.nc
Skipping .fregrid_remap_file_144_by_90.nc
Skipping ap.nc
Skipping b.nc
Skipping .fregrid_remap_file_288_by_180.nc
Skipping b.nc
Skipping ap.nc
Skipping .fregrid_remap_file_288_by_180.nc

[Ciheim]

Also, do we want to hide this warning from pandas?

/home/Ciheim.Brown/conda/envs/catalogbuilder/lib/python3.7/site-packages/pandas/core/indexing.py:670: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
self._setitem_with_indexer(indexer, value)

[aradhakrishnanGFDL]

@Ciheim @ceblanton do not merge the open PRs, I found a bug and it's best to test a bit further and get rid of it. time_range is not populating either in this PR or the other one..

[aradhakrishnanGFDL]

@Ciheim @ceblanton do not merge the open PRs, I found a bug and it's best to test a bit further and get rid of it. time_range is not populating either in this PR or the other one..

no bug in the code, looks ok. The problem was that I used the fre-cli conda environment and it is not up-to-date, a bit stale even as it does not recognize changes made to the --config option. Action needed at the fre-cli front .