Restore NC3 Subsetting for Calculated Variables #920
Conversation
There was a problem hiding this comment.
This looks like a reasonable solution. Could you clarify if the issue is for the "NetCDF-3 Classic" option or the "NetCDF-3 NC" option.
Before we merge this, should we check with end users to confirm that removing the dims attribute is acceptable? Perhaps by sharing a sample file?
|
@nsoontie I've noticed this problem when using NetCDF-3 Classic, NetCDF-3 64-bit, and NetCDF-4 Classic but not the default NetCDF-4 or NetCDF-3 NC formats. It looks like we essentially build the variables manually without the We can certainly check with the end users to see if the |
author htmlboss <nmiri127@gmail.com> 1632436592 +0000 committer htmlboss <nmiri127@gmail.com> 1638808746 +0000 Stop duplicate API calls for Variables dropdown Start using improved variable dropdown Also had to clean up how interpolation options get changed when variable changes. In addition, cleaned the sh*t out of the Range component. Still not done yet sheesh. make line window work still need 3d variable filter Add two_dimensional field to variables API response Enable filtering of 3D variables Stop duplicate API calls for dataset dropdown in DatasetSelector Restart from scratch and break everything Add loading popup Move variable scale input into DatasetSelector Get Area window working Get LineWindow working Make Point window work fix 301 redirects screwing up cache Switch to axios-extensions because it actually works Update timepicker to use cache-able query improve loading modal title Make DatasetSelector more understandable Remove dead code Replace negative bearing values in quiver tooltip (#910) Add onUpdateOptions to Variable selectors (#915) Restore NC3 Subsetting for Calculated Variables (#920) * Drop-'Dims'-attribute-from-calculated-subset-vars * Cast 'dims' attr to string Updates to a couple of service scripts (#922) Removed the logging information from the launch-web-service.sh script as runserver.sh now contains a Gunicorn --log-file definition. As well, the number of Gunicorn worker threads has been reduced from 2 to 1. Placing a symbolic link in the root of the project so that Javascript & Python are able to access the files. Mount pop up windows with selected dataset in main map fix some alerts Fix a crash when a 2D variable is tried to be selected with a 3D plot
Background
We were notified that users are unable to subset area data using NetCDF-3 formats for calculated variables. When creating a subset from one of these variables the Navigator attempts to add the attributes from the datasetconfig.json as variable attributes in the resulting NetCDF file. This includes the 'dims' attribute is a list of strings describing the dimensions used for the variable calculation. It seems that xarray will try to encode variables like this one using the NC_STRING format which is only compatible with NetCDF-4 files, causing the error. More detail can be found in Issue #919.
Closes #919
Why did you take this approach?
My initial approach was to cast the 'dims; variable to another datatype to see if xarray would encode it using a compatible format however nothing seemed to work. In the end I felt that it was best to drop this attribute entirely since the dimensions should be listed in the resulting NC file anyway. All calculated variables given in the datasetconfig.json file need to have a 'dims' attribute so we shouldn't need to add logic to check for this attribute before dropping it.
Anything in particular that should be highlighted?
I found that some of the calculated variables are returned without named dimensions resulting in variables with dimensions
dim_0, dim_1, etc.so having a 'dims' attribute could help here. Since this seems to be the root of issue #897 I decided to address that in the fix I'm working on there.Checks