From a5cae51efce2d7b04859a34a19f16308b34423b0 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 23 Mar 2022 19:28:06 -0600
Subject: [PATCH 01/17] Turn on the xarray convention for NCZarr format

re: https://github.com/pydata/xarray/issues/6374

As a result of a discussion about Xarray (see above issue),
I decided to turn on the xarray convention for NCZarr datasets
where possible so that xarray can read a larger set of nczarr
generated datasets.

This causes the following changes:
* If the user wants to generate a pure zarr file, then the mode "zarr" must be explicitly used; it is no longer the case that "mode=xarray" or mode="noxarray"
implies "mode=zarr".
* It is still the case that "mode=noxarray" will turn off the XArray convention.

The following conditions will cause ''_ARRAY_DIMENSIONS'' to not be written.
* The variable is not in the root group,
* Any dimension referenced by the variable is not in the root group.
---
 .github/workflows/run_tests_ubuntu.yml |  2 +-
 RELEASE_NOTES.md                       |  1 +
 docs/nczarr.md                         | 23 ++++++++------
 libdispatch/dinfermodel.c              | 11 +++----
 libnczarr/zarr.c                       |  5 ++-
 libnczarr/zsync.c                      | 43 +++++++++++++++++---------
 nc_test4/tst_broken_files.c            |  5 +--
 nczarr_test/ref_newformatpure.cdl      |  3 +-
 nczarr_test/run_purezarr.sh            |  6 ++--
 9 files changed, 59 insertions(+), 40 deletions(-)

diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index 723b6a2415..eb410325a7 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [ pull_request ]
+on: [pull_request]
 
 jobs:
 
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 6195b414c3..0072a3e6e9 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -7,6 +7,7 @@ This file contains a high-level description of this package's evolution. Release
 
 ## 4.8.2 - TBD
 
+* [Enhancement] Turn on the XArray convention for NCZarr files by default. See [Github #????](https://github.com/Unidata/netcdf-c/pull/????).
 * [Enhancement] Improve filter support. More specifically (1) add nc_inq_filter_avail to check if a filter is available, (2) add the notion of standard filters, (3) cleanup szip support to fix interaction with NCZarr. See [Github #2245](https://github.com/Unidata/netcdf-c/pull/2245).
 * [Bug Fix] Require that the type of the variable in nc_def_var_filter is not variable length. See [Github #/2231](https://github.com/Unidata/netcdf-c/pull/2231).
 * [File Change] Apply HDF5 v1.8 format compatibility when writing to previous files, as well as when creating new files.  The superblock version remains at 2 for newly created files.  Full backward read/write compatibility for netCDF-4 is maintained in all cases.  See [Github #2176](https://github.com/Unidata/netcdf-c/issues/2176).
diff --git a/docs/nczarr.md b/docs/nczarr.md
index af10387c60..7071db7c78 100644
--- a/docs/nczarr.md
+++ b/docs/nczarr.md
@@ -136,16 +136,18 @@ Note that It should be the case that zipping a _file_
 format directory tree will produce a file readable by the
 _zip_ storage format, and vice-versa.
 
-By default, _mode=zarr_ also supports the XArray _\_ARRAY\_DIMENSIONS_ convention. The _noxarray_ mode tells the library to disable the XArray support.
+By default, the XArray convention is supported and used for
+both NCZarr files and pure Zarr files. This
+means that every variable in the root group whose named dimensions
+are also in the root group will have an attribute called
+*\_ARRAY\_DIMENSIONS* that stores those dimension names.
+The _noxarray_ mode tells the library to disable the XArray support.
 
 The netcdf-c library is capable of inferring additional mode flags based on the flags it finds. Currently we have the following inferences.
-
-- _xarray_ => _zarr_
-- _noxarray_ => _zarr_
 - _zarr_ => _nczarr_
 
-So for example: ````...#mode=noxarray,zip```` is equivalent to this.
-````...#mode=nczarr,zarr,noxarray,zip
+So for example: ````...#mode=zarr,zip```` is equivalent to this.
+````...#mode=nczarr,zarr,zip
 ````
 <!--
 - log=&lt;output-stream&gt;: this control turns on logging output,
@@ -434,10 +436,13 @@ The value of this attribute is a list of dimension names (strings).
 An example might be ````["time", "lon", "lat"]````.
 It is essentially equivalent to the ````_NCZARR_ARRAY "dimrefs" list````, except that the latter uses fully qualified names so the referenced dimensions can be anywhere in the dataset.
 
-As of _netcdf-c_ version 4.8.1, The Xarray ''_ARRAY_DIMENSIONS'' attribute is supported.
-This attribute will be read/written by default, but can be suppressed if the mode value "noxarray" is specified.
+As of _netcdf-c_ version 4.8.2, The Xarray ''_ARRAY_DIMENSIONS'' attribute is supported for both NCZarr and pure Zarr.
+If possible, this attribute will be read/written by default,
+but can be suppressed if the mode value "noxarray" is specified.
 If detected, then these dimension names are used to define shared dimensions.
-Note that "noxarray" or "xarray" implies pure zarr format.
+The following conditions will cause ''_ARRAY_DIMENSIONS'' to not be written.
+* The variable is not in the root group,
+* Any dimension referenced by the variable is not in the root group.
 
 # Examples {#nczarr_examples}
 
diff --git a/libdispatch/dinfermodel.c b/libdispatch/dinfermodel.c
index e98ff912f8..e71125e64f 100644
--- a/libdispatch/dinfermodel.c
+++ b/libdispatch/dinfermodel.c
@@ -137,9 +137,9 @@ static const struct MACRODEF {
 {"dap4","mode",{"dap4",NULL}},
 {"s3","mode",{"s3","nczarr",NULL}},
 {"bytes","mode",{"bytes",NULL}},
-{"xarray","mode",{"nczarr","zarr","xarray",NULL}},
-{"noxarray","mode",{"nczarr","zarr","noxarray",NULL}},
-{"zarr","mode",{"nczarr","zarr","xarray",NULL}},
+{"xarray","mode",{"nczarr", NULL}},
+{"noxarray","mode",{"nczarr", "noxarray", NULL}},
+{"zarr","mode",{"nczarr","zarr", NULL}},
 {NULL,NULL,{NULL}}
 };
 
@@ -149,9 +149,8 @@ static const struct MODEINFER {
     char* inference;
 } modeinferences[] = {
 {"zarr","nczarr"},
-{"zarr","xarray"},
-{"xarray","zarr"},
-{"noxarray","zarr"},
+{"xarray","nczarr"},
+{"noxarray","nczarr"},
 {NULL,NULL}
 };
 
diff --git a/libnczarr/zarr.c b/libnczarr/zarr.c
index afc06ddd16..bce7ca6f18 100644
--- a/libnczarr/zarr.c
+++ b/libnczarr/zarr.c
@@ -384,13 +384,12 @@ applycontrols(NCZ_FILE_INFO_T* zinfo)
     }
     /* Process the modelist first */
     zinfo->controls.mapimpl = NCZM_DEFAULT;
+    zinfo->controls.flags |= FLAG_XARRAYDIMS; /* Always support XArray convention where possible */
     for(i=0;i<nclistlength(modelist);i++) {
         const char* p = nclistget(modelist,i);
-	if(strcasecmp(p,PUREZARRCONTROL)==0) zinfo->controls.flags |= (FLAG_PUREZARR|FLAG_XARRAYDIMS);
-	else if(strcasecmp(p,XARRAYCONTROL)==0) zinfo->controls.flags |= (FLAG_XARRAYDIMS|FLAG_PUREZARR); /*xarray=>zarr*/
+	if(strcasecmp(p,PUREZARRCONTROL)==0) zinfo->controls.flags |= (FLAG_PUREZARR);
 	else if(strcasecmp(p,NOXARRAYCONTROL)==0) {
 	    noflags |= FLAG_XARRAYDIMS;
-	    zinfo->controls.flags |= FLAG_PUREZARR; /*noxarray=>zarr*/
 	}
 	else if(strcasecmp(p,"zip")==0) zinfo->controls.mapimpl = NCZM_ZIP;
 	else if(strcasecmp(p,"file")==0) zinfo->controls.mapimpl = NCZM_FILE;
diff --git a/libnczarr/zsync.c b/libnczarr/zsync.c
index 4ca2781685..03872365a3 100644
--- a/libnczarr/zsync.c
+++ b/libnczarr/zsync.c
@@ -695,20 +695,33 @@ ncz_sync_atts(NC_FILE_INFO_T* file, NC_OBJ* container, NCindex* attlist, int isc
 
     if(container->sort == NCVAR) { 
         if(isrootgroup && isxarray) {
+	    int dimsinroot = 1;
 	    /* Insert the XARRAY _ARRAY_ATTRIBUTE attribute */
 	    if((stat = NCJnew(NCJ_ARRAY,&jdimrefs)))
 	        goto done;
-	    /* Walk the dimensions and capture the names */
+	    /* Walk the dimensions to check in root group */
 	    for(i=0;i<var->ndims;i++) {
 	        NC_DIM_INFO_T* dim = var->dim[i];
-		char* dimname = strdup(dim->hdr.name);
-		if(dimname == NULL) {stat = NC_ENOMEM; goto done;}
-	        NCJaddstring(jdimrefs,NCJ_STRING,dimname);
-   	        nullfree(dimname); dimname = NULL;
+		/* Verify that the dimension is in the root group */
+		if(dim->container && dim->container->parent != NULL) {
+		    dimsinroot = 0; /* dimension is not in root */
+		    break;
+		}
+	    }
+	    if(dimsinroot) {
+		/* Walk the dimensions and capture the names */
+		for(i=0;i<var->ndims;i++) {
+		    char* dimname;
+	            NC_DIM_INFO_T* dim = var->dim[i];
+		    dimname = strdup(dim->hdr.name);
+		    if(dimname == NULL) {stat = NC_ENOMEM; goto done;}
+	            NCJaddstring(jdimrefs,NCJ_STRING,dimname);
+   	            nullfree(dimname); dimname = NULL;
+		}
+	        /* Add the _ARRAY_DIMENSIONS attribute */
+	        if((stat = NCJinsert(jatts,NC_XARRAY_DIMS,jdimrefs))) goto done;
+	        jdimrefs = NULL;
 	    }
-	    /* Add the _ARRAY_DIMENSIONS attribute */
-	    if((stat = NCJinsert(jatts,NC_XARRAY_DIMS,jdimrefs))) goto done;
-	    jdimrefs = NULL;
         }
     }
     /* Add Quantize Attribute */
@@ -736,9 +749,11 @@ ncz_sync_atts(NC_FILE_INFO_T* file, NC_OBJ* container, NCindex* attlist, int isc
 	    /* Insert the _NCZARR_ATTR attribute */
             if((stat = NCJnew(NCJ_DICT,&jdict)))
                 goto done;
-            if((stat = NCJinsert(jdict,"types",jtypes))) goto done;
+	    if(jtypes != NULL)
+                {if((stat = NCJinsert(jdict,"types",jtypes))) goto done;}
             jtypes = NULL;
-            if((stat = NCJinsert(jatts,NCZ_V2_ATTR,jdict))) goto done;      
+	    if(jdict != NULL)
+                {if((stat = NCJinsert(jatts,NCZ_V2_ATTR,jdict))) goto done;}
             jdict = NULL;
 	}
         /* write .zattrs path */
@@ -1449,7 +1464,7 @@ define_vars(NC_FILE_INFO_T* file, NC_GRP_INFO_T* grp, NClist* varnames)
 
     if(zinfo->controls.flags & FLAG_PUREZARR) purezarr = 1;
     if(zinfo->controls.flags & FLAG_NCZARR_V1) formatv1 = 1;
-    if(zinfo->controls.flags & FLAG_XARRAYDIMS) {purezarr = 1; xarray = 1;}
+    if(zinfo->controls.flags & FLAG_XARRAYDIMS) {xarray = 1;}
 
     /* Load each var in turn */
     for(i = 0; i < nclistlength(varnames); i++) {
@@ -2246,10 +2261,8 @@ computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarra
 
     assert(zfile && zvar);
 
-    /* xarray => purezarr */
-    assert(!xarray || purezarr);
-
-    if(xarray) {/* Read in the attributes to get xarray dimdef attribute; Note that it might not exist */
+    if(purezarr && xarray) {/* Read in the attributes to get xarray dimdef attribute; Note that it might not exist */
+	/* Note that if xarray && !purezarr, then xarray will be superceded by the nczarr dimensions key */
         char zdimname[4096];
 	if(zvar->xarray == NULL) {
 	    assert(nclistlength(dimnames) == 0);
diff --git a/nc_test4/tst_broken_files.c b/nc_test4/tst_broken_files.c
index af68b26905..3834b9c701 100644
--- a/nc_test4/tst_broken_files.c
+++ b/nc_test4/tst_broken_files.c
@@ -27,8 +27,9 @@ main() {
         if(fwrite(TRUNCATED_FILE_CONTENT, sizeof(char), sizeof(TRUNCATED_FILE_CONTENT), fp) != sizeof(TRUNCATED_FILE_CONTENT)) ERR;
         fclose(fp);
 
-        int  ncid;
-        if (nc_open(FILE_NAME, 0, &ncid) != NC_EHDFERR) ERR;
+        int  ncid, stat;
+        stat = nc_open(FILE_NAME, 0, &ncid);
+        if (stat != NC_EHDFERR && stat != NC_ENOFILTER && stat != NC_ENOTNC) ERR;
     }
 
     {
diff --git a/nczarr_test/ref_newformatpure.cdl b/nczarr_test/ref_newformatpure.cdl
index 2f86124339..080df793f1 100644
--- a/nczarr_test/ref_newformatpure.cdl
+++ b/nczarr_test/ref_newformatpure.cdl
@@ -1,9 +1,10 @@
 netcdf ref_oldformat {
 dimensions:
+	lat = 8 ;
 	_zdim_8 = 8 ;
 	_zdim_10 = 10 ;
 variables:
-	int lat(_zdim_8) ;
+	int lat(lat) ;
 		lat:_FillValue = -1 ;
 		lat:lat_attr = "latitude" ;
 data:
diff --git a/nczarr_test/run_purezarr.sh b/nczarr_test/run_purezarr.sh
index a74760c916..b068994c19 100755
--- a/nczarr_test/run_purezarr.sh
+++ b/nczarr_test/run_purezarr.sh
@@ -15,7 +15,7 @@ testcase() {
 zext=$1
 
 echo "*** Test: pure zarr write then read; format=$zext"
-fileargs tmp_purezarr "mode=noxarray,$zext"
+fileargs tmp_purezarr "mode=zarr,noxarray,$zext"
 deletemap $zext $file
 ${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_purezarr_base.cdl
 ${NCDUMP} $fileurl > tmp_purezarr_${zext}.cdl
@@ -27,9 +27,9 @@ fileargs tmp_xarray "mode=zarr,$zext"
 ${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_purezarr_base.cdl
 ${NCDUMP} $fileurl > tmp_xarray_${zext}.cdl
 diff -b ${srcdir}/ref_xarray.cdl tmp_xarray_${zext}.cdl
-echo "*** Test: pure zarr reading nczarr; format=$zext"
-fileargs tmp_nczarr "mode=nczarr,$zext"
 
+echo "*** Test: pure zarr reading nczarr; format=$zext"
+fileargs tmp_nczarr "mode=nczarr,noxarray,$zext"
 deletemap $zext $file
 ${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_whole.cdl
 fileargs tmp_nczarr "mode=zarr,$zext"

From 9bc1c1bc7e80c1f454126c29409363203cb2bb12 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 23 Mar 2022 19:31:33 -0600
Subject: [PATCH 02/17] Update release notes

---
 RELEASE_NOTES.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 0072a3e6e9..1496176770 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -7,7 +7,7 @@ This file contains a high-level description of this package's evolution. Release
 
 ## 4.8.2 - TBD
 
-* [Enhancement] Turn on the XArray convention for NCZarr files by default. See [Github #????](https://github.com/Unidata/netcdf-c/pull/????).
+* [Enhancement] Turn on the XArray convention for NCZarr files by default. See [Github #2257](https://github.com/Unidata/netcdf-c/pull/2257).
 * [Enhancement] Improve filter support. More specifically (1) add nc_inq_filter_avail to check if a filter is available, (2) add the notion of standard filters, (3) cleanup szip support to fix interaction with NCZarr. See [Github #2245](https://github.com/Unidata/netcdf-c/pull/2245).
 * [Bug Fix] Require that the type of the variable in nc_def_var_filter is not variable length. See [Github #/2231](https://github.com/Unidata/netcdf-c/pull/2231).
 * [File Change] Apply HDF5 v1.8 format compatibility when writing to previous files, as well as when creating new files.  The superblock version remains at 2 for newly created files.  Full backward read/write compatibility for netCDF-4 is maintained in all cases.  See [Github #2176](https://github.com/Unidata/netcdf-c/issues/2176).

From bee267e8670257d23491e4516b396ec2e4cac2a9 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 23 Mar 2022 20:14:12 -0600
Subject: [PATCH 03/17] Fix float precision issue in ncdap_tests

---
 ncdap_test/tst_utils.sh | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ncdap_test/tst_utils.sh b/ncdap_test/tst_utils.sh
index 20417adae9..e2fc8cb2fa 100644
--- a/ncdap_test/tst_utils.sh
+++ b/ncdap_test/tst_utils.sh
@@ -11,7 +11,10 @@ PARAMS="[log]"
 
 OCLOGFILE=/dev/null
 
-DUMPFLAGS=
+# Reduce the precision
+if test "x$FP_ISOSX" != x ; then
+DUMPFLAGS="-p7,14"
+fi
 
 # Locate directories
 testdata3="${srcdir}/testdata3"

From 4bed17e0da1c10d66ad97adbf091429fc5b88a0c Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 23 Mar 2022 20:48:18 -0600
Subject: [PATCH 04/17] precision fix

---
 ncdap_test/tst_utils.sh | 2 --
 1 file changed, 2 deletions(-)

diff --git a/ncdap_test/tst_utils.sh b/ncdap_test/tst_utils.sh
index e2fc8cb2fa..3e43f0c0b8 100644
--- a/ncdap_test/tst_utils.sh
+++ b/ncdap_test/tst_utils.sh
@@ -12,9 +12,7 @@ PARAMS="[log]"
 OCLOGFILE=/dev/null
 
 # Reduce the precision
-if test "x$FP_ISOSX" != x ; then
 DUMPFLAGS="-p7,14"
-fi
 
 # Locate directories
 testdata3="${srcdir}/testdata3"

From 70b6df9b04be1b8fa3e660ff7065609405170ba0 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 23 Mar 2022 21:01:58 -0600
Subject: [PATCH 05/17] OSX only

---
 ncdap_test/tst_utils.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/ncdap_test/tst_utils.sh b/ncdap_test/tst_utils.sh
index 3e43f0c0b8..e2fc8cb2fa 100644
--- a/ncdap_test/tst_utils.sh
+++ b/ncdap_test/tst_utils.sh
@@ -12,7 +12,9 @@ PARAMS="[log]"
 OCLOGFILE=/dev/null
 
 # Reduce the precision
+if test "x$FP_ISOSX" != x ; then
 DUMPFLAGS="-p7,14"
+fi
 
 # Locate directories
 testdata3="${srcdir}/testdata3"

From 72c90b41c89b7892be5a576efd57dfbb9136f261 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Fri, 25 Mar 2022 20:54:03 -0600
Subject: [PATCH 06/17] Add Release Notes warning

---
 RELEASE_NOTES.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 1496176770..579d330b28 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -7,7 +7,7 @@ This file contains a high-level description of this package's evolution. Release
 
 ## 4.8.2 - TBD
 
-* [Enhancement] Turn on the XArray convention for NCZarr files by default. See [Github #2257](https://github.com/Unidata/netcdf-c/pull/2257).
+* [Enhancement] Turn on the XArray convention for NCZarr files by default. WARNING, this now means that the mode must contain "nczarr" or "zarr" even if "xarray" or "noxarray" is specified. See [Github #2257](https://github.com/Unidata/netcdf-c/pull/2257).
 * [Enhancement] Improve filter support. More specifically (1) add nc_inq_filter_avail to check if a filter is available, (2) add the notion of standard filters, (3) cleanup szip support to fix interaction with NCZarr. See [Github #2245](https://github.com/Unidata/netcdf-c/pull/2245).
 * [Bug Fix] Require that the type of the variable in nc_def_var_filter is not variable length. See [Github #/2231](https://github.com/Unidata/netcdf-c/pull/2231).
 * [File Change] Apply HDF5 v1.8 format compatibility when writing to previous files, as well as when creating new files.  The superblock version remains at 2 for newly created files.  Full backward read/write compatibility for netCDF-4 is maintained in all cases.  See [Github #2176](https://github.com/Unidata/netcdf-c/issues/2176).

From 2ea8aec566315f02a52fdf0eb0786b1e59b971c6 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Sat, 26 Mar 2022 13:15:10 -0600
Subject: [PATCH 07/17] Change so explicit xarray => pure zarr

---
 .github/workflows/run_tests_osx.yml       |   2 +-
 .github/workflows/run_tests_ubuntu.yml    |   2 +-
 .github/workflows/run_tests_win_mingw.yml |   2 +-
 .github/workflows/tmp0.yml                | 233 ++++++++++++++++++++++
 RELEASE_NOTES.md                          |   2 +-
 libdispatch/dinfermodel.c                 |   4 +-
 libnczarr/zarr.c                          |   8 +-
 7 files changed, 244 insertions(+), 9 deletions(-)
 create mode 100644 .github/workflows/tmp0.yml

diff --git a/.github/workflows/run_tests_osx.yml b/.github/workflows/run_tests_osx.yml
index 8e9d65a6fb..aa35ae8795 100644
--- a/.github/workflows/run_tests_osx.yml
+++ b/.github/workflows/run_tests_osx.yml
@@ -7,7 +7,7 @@
 name: Run macOS-based netCDF Tests
 
 
-on: [pull_request]
+on: [pull_request,push]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index eb410325a7..1ede4ad90c 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [pull_request]
+on: [pull_request,push]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_win_mingw.yml b/.github/workflows/run_tests_win_mingw.yml
index 913920a3aa..aaf999e6a4 100644
--- a/.github/workflows/run_tests_win_mingw.yml
+++ b/.github/workflows/run_tests_win_mingw.yml
@@ -7,7 +7,7 @@
 name: Run MSYS2, MinGW64-based Tests
 
 
-on: [pull_request]
+on: [pull_request,push]
 
 jobs:
 
diff --git a/.github/workflows/tmp0.yml b/.github/workflows/tmp0.yml
new file mode 100644
index 0000000000..b2c343efe7
--- /dev/null
+++ b/.github/workflows/tmp0.yml
@@ -0,0 +1,233 @@
+###
+# Build hdf4, hdf5 dependencies and cache them in a combined directory.
+###
+
+name: Run netCDF Tests
+
+on: [pull_request]
+
+jobs:
+
+  build-deps:
+
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        hdf5: [ 1.8.21, 1.10.7 ]
+
+    steps:
+
+      - uses: actions/checkout@v2
+
+      - name: Install System dependencies
+        shell: bash -l {0}
+        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
+
+        ###
+        # Installing libhdf4 and libhdf5
+        ###
+      - name: Cache libhdf5-${{ matrix.hdf5 }}
+        id: cache-hdf5
+        uses: actions/cache@v2
+        with:
+          path: ~/environments/${{ matrix.hdf5 }}
+          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
+
+
+      - name: Build libhdf5-${{ matrix.hdf5 }}
+        if: steps.cache-hdf5.outputs.cache-hit != 'true'
+        run: |
+          set -x
+          wget https://support.hdfgroup.org/ftp/HDF/releases/HDF4.2.15/src/hdf-4.2.15.tar.bz2
+          tar -jxf hdf-4.2.15.tar.bz2
+          pushd hdf-4.2.15
+          ./configure --prefix=${HOME}/environments/${{ matrix.hdf5 }} --disable-static --enable-shared --disable-fortran --disable-netcdf --with-szlib
+          make -j
+          make install -j
+          popd
+          wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-$(echo ${{ matrix.hdf5 }} | cut -d. -f 1,2)/hdf5-${{ matrix.hdf5 }}/src/hdf5-${{ matrix.hdf5 }}.tar.bz2
+          tar -jxf hdf5-${{ matrix.hdf5 }}.tar.bz2
+          pushd hdf5-${{ matrix.hdf5 }}
+          ./configure --disable-static --enable-shared --prefix=${HOME}/environments/${{ matrix.hdf5 }} --enable-hl --with-szlib
+          make -j
+          make install -j
+          popd
+
+
+  nc-autotools:
+
+    needs: build-deps
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        hdf5: [ 1.8.21, 1.10.7 ]
+        use_nc4: [ nc3, nc4 ]
+        use_dap: [ dap_off, dap_on ]
+    steps:
+
+      - uses: actions/checkout@v2
+
+      - name: Install System dependencies
+        shell: bash -l {0}
+        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
+
+###
+# Set Environmental Variables
+###
+
+      - run: echo "CFLAGS=-I${HOME}/environments/${{ matrix.hdf5 }}/include" >> $GITHUB_ENV
+      - run: echo "LDFLAGS=-L${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
+      - run: echo "LD_LIBRARY_PATH=${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
+      - run: |
+          echo "ENABLE_HDF4=--disable-hdf4" >> $GITHUB_ENV
+          echo "ENABLE_NC4=--disable-netcdf-4" >> $GITHUB_ENV
+        if: matrix.use_nc4 == 'nc3'
+      - run: |
+          echo "ENABLE_HDF4=--enable-hdf4" >> $GITHUB_ENV
+          echo "ENABLE_NC4=--enable-netcdf-4" >> $GITHUB_ENV
+        if: matrix.use_nc4 == 'nc4'
+      - run: echo "ENABLE_DAP=--disable-dap" >> $GITHUB_ENV
+        if: matrix.use_dap == 'dap_off'
+      - run: echo "ENABLE_DAP=--enable-dap" >> $GITHUB_ENV
+        if: matrix.use_dap == 'dap_on'
+
+###
+# Fetch Cache
+###
+
+      - name: Fetch HDF Cache
+        id: cache-hdf
+        uses: actions/cache@v2
+        with:
+          path: ~/environments/${{ matrix.hdf5 }}
+          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
+
+      - name: Check Cache
+        shell: bash -l {0}
+        run: ls ${HOME}/environments && ls ${HOME}/environments/${{ matrix.hdf5 }} && ls ${HOME}/environments/${{ matrix.hdf5}}/lib
+
+###
+# Configure and build
+###
+
+      - name: Run autoconf
+        shell: bash -l {0}
+        run: autoreconf -if
+
+      - name: Configure
+        shell: bash -l {0}
+        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} ./configure ${ENABLE_HDF4} ${ENABLE_NC4} ${ENABLE_DAP}
+
+      - name: Look at config.log if error
+        shell: bash -l {0}
+        run: cat config.log
+        if: ${{ failure() }}
+
+      - name: Print Summary
+        shell: bash -l {0}
+        run: cat libnetcdf.settings
+
+      - name: Build Library and Utilities
+        shell: bash -l {0}
+        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j
+        if: ${{ success() }}
+
+      - name: Build Tests
+        shell: bash -l {0}
+        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check TESTS="" -j
+        if: ${{ success() }}
+
+      - name: Run Tests
+        shell: bash -l {0}
+        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check -j
+        if: ${{ success() }}
+
+      #- name: Make Distcheck
+      #  shell: bash -l {0}
+      #  run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make distcheck -j
+
+  nc-cmake:
+
+    needs: build-deps
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        hdf5: [ 1.8.21, 1.10.7 ]
+        use_nc4: [ nc3, nc4 ]
+        use_dap: [ dap_off, dap_on ]
+    steps:
+
+      - uses: actions/checkout@v2
+
+      - name: Install System dependencies
+        shell: bash -l {0}
+        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
+
+###
+# Set Environmental Variables
+###
+
+      - run: echo "CMAKE_PREFIX_PATH=${HOME}/environments/${{ matrix.hdf5 }}/" >> $GITHUB_ENV
+      - run: echo "LD_LIBRARY_PATH=${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
+      - run: |
+          echo "ENABLE_HDF4=OFF" >> $GITHUB_ENV
+          echo "ENABLE_NC4=OFF" >> $GITHUB_ENV
+        if: matrix.use_nc4 == 'nc3'
+      - run: |
+          echo "ENABLE_HDF4=ON" >> $GITHUB_ENV
+          echo "ENABLE_NC4=ON" >> $GITHUB_ENV
+        if: matrix.use_nc4 == 'nc4'
+      - run: echo "ENABLE_DAP=OFF" >> $GITHUB_ENV
+        if: matrix.use_dap == 'dap_off'
+      - run: echo "ENABLE_DAP=ON" >> $GITHUB_ENV
+        if: matrix.use_dap == 'dap_on'
+      - run: echo "CTEST_OUTPUT_ON_FAILURE=1" >> $GITHUB_ENV
+
+###
+# Fetch Cache
+###
+
+      - name: Fetch HDF Cache
+        id: cache-hdf5
+        uses: actions/cache@v2
+        with:
+          path: ~/environments/${{ matrix.hdf5 }}
+          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
+
+      - name: Check Cache
+        shell: bash -l {0}
+        run: ls ${HOME}/environments && ls ${HOME}/environments/${{ matrix.hdf5 }} && ls ${HOME}/environments/${{ matrix.hdf5}}/lib
+
+###
+# Configure and build
+###
+
+      - name: Perform out-of-directory build
+        shell: bash -l {0}
+        run: |
+          mkdir build
+          cd build
+          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} cmake .. -DENABLE_HDF4=${ENABLE_HDF4} -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH} -DENABLE_DAP=${ENABLE_DAP} -DENABLE_NETCDF_4=${ENABLE_NC4}
+
+      - name: Print Summary
+        shell: bash -l {0}
+        run: |
+          cd build
+          cat libnetcdf.settings
+
+      - name: Build All
+        shell: bash -l {0}
+        run: |
+          cd build
+          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j 12
+        if: ${{ success() }}
+
+      - name: Run Tests
+        shell: bash -l {0}
+        run: |
+          cd build
+          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} ctest -j 12 .
+        if: ${{ success() }}
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 579d330b28..ba6f311229 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -7,7 +7,7 @@ This file contains a high-level description of this package's evolution. Release
 
 ## 4.8.2 - TBD
 
-* [Enhancement] Turn on the XArray convention for NCZarr files by default. WARNING, this now means that the mode must contain "nczarr" or "zarr" even if "xarray" or "noxarray" is specified. See [Github #2257](https://github.com/Unidata/netcdf-c/pull/2257).
+* [Enhancement] Turn on the XArray convention for NCZarr files by default. WARNING, this means that the mode should explicitly specify nczarr" or "zarr" even if "xarray" or "noxarray" is specified. See [Github #2257](https://github.com/Unidata/netcdf-c/pull/2257).
 * [Enhancement] Improve filter support. More specifically (1) add nc_inq_filter_avail to check if a filter is available, (2) add the notion of standard filters, (3) cleanup szip support to fix interaction with NCZarr. See [Github #2245](https://github.com/Unidata/netcdf-c/pull/2245).
 * [Bug Fix] Require that the type of the variable in nc_def_var_filter is not variable length. See [Github #/2231](https://github.com/Unidata/netcdf-c/pull/2231).
 * [File Change] Apply HDF5 v1.8 format compatibility when writing to previous files, as well as when creating new files.  The superblock version remains at 2 for newly created files.  Full backward read/write compatibility for netCDF-4 is maintained in all cases.  See [Github #2176](https://github.com/Unidata/netcdf-c/issues/2176).
diff --git a/libdispatch/dinfermodel.c b/libdispatch/dinfermodel.c
index e71125e64f..6e02b7d121 100644
--- a/libdispatch/dinfermodel.c
+++ b/libdispatch/dinfermodel.c
@@ -137,7 +137,7 @@ static const struct MACRODEF {
 {"dap4","mode",{"dap4",NULL}},
 {"s3","mode",{"s3","nczarr",NULL}},
 {"bytes","mode",{"bytes",NULL}},
-{"xarray","mode",{"nczarr", NULL}},
+{"xarray","mode",{"zarr", NULL}},
 {"noxarray","mode",{"nczarr", "noxarray", NULL}},
 {"zarr","mode",{"nczarr","zarr", NULL}},
 {NULL,NULL,{NULL}}
@@ -149,7 +149,7 @@ static const struct MODEINFER {
     char* inference;
 } modeinferences[] = {
 {"zarr","nczarr"},
-{"xarray","nczarr"},
+{"xarray","zarr"},
 {"noxarray","nczarr"},
 {NULL,NULL}
 };
diff --git a/libnczarr/zarr.c b/libnczarr/zarr.c
index bce7ca6f18..671b0c2ace 100644
--- a/libnczarr/zarr.c
+++ b/libnczarr/zarr.c
@@ -387,10 +387,12 @@ applycontrols(NCZ_FILE_INFO_T* zinfo)
     zinfo->controls.flags |= FLAG_XARRAYDIMS; /* Always support XArray convention where possible */
     for(i=0;i<nclistlength(modelist);i++) {
         const char* p = nclistget(modelist,i);
-	if(strcasecmp(p,PUREZARRCONTROL)==0) zinfo->controls.flags |= (FLAG_PUREZARR);
-	else if(strcasecmp(p,NOXARRAYCONTROL)==0) {
+	if(strcasecmp(p,PUREZARRCONTROL)==0)
+	    zinfo->controls.flags |= (FLAG_PUREZARR);
+	else if(strcasecmp(p,XARRAYCONTROL)==0)
+	    zinfo->controls.flags |= FLAG_PUREZARR;
+	else if(strcasecmp(p,NOXARRAYCONTROL)==0)
 	    noflags |= FLAG_XARRAYDIMS;
-	}
 	else if(strcasecmp(p,"zip")==0) zinfo->controls.mapimpl = NCZM_ZIP;
 	else if(strcasecmp(p,"file")==0) zinfo->controls.mapimpl = NCZM_FILE;
 	else if(strcasecmp(p,"s3")==0) zinfo->controls.mapimpl = NCZM_S3;

From 0825302e02f454805f2ecdfb09c3bc4834f4a033 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Sat, 26 Mar 2022 13:15:39 -0600
Subject: [PATCH 08/17] Remove debugging

---
 .github/workflows/run_tests_osx.yml       | 2 +-
 .github/workflows/run_tests_ubuntu.yml    | 2 +-
 .github/workflows/run_tests_win_mingw.yml | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/run_tests_osx.yml b/.github/workflows/run_tests_osx.yml
index aa35ae8795..8e9d65a6fb 100644
--- a/.github/workflows/run_tests_osx.yml
+++ b/.github/workflows/run_tests_osx.yml
@@ -7,7 +7,7 @@
 name: Run macOS-based netCDF Tests
 
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index 1ede4ad90c..eb410325a7 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_win_mingw.yml b/.github/workflows/run_tests_win_mingw.yml
index aaf999e6a4..913920a3aa 100644
--- a/.github/workflows/run_tests_win_mingw.yml
+++ b/.github/workflows/run_tests_win_mingw.yml
@@ -7,7 +7,7 @@
 name: Run MSYS2, MinGW64-based Tests
 
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 

From 13dd15b76caf5c32881369aeeff5c31163af1f9f Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Mon, 28 Mar 2022 10:04:18 -0600
Subject: [PATCH 09/17] minor fix

---
 .github/workflows/tmp0.yml | 233 -------------------------------------
 1 file changed, 233 deletions(-)
 delete mode 100644 .github/workflows/tmp0.yml

diff --git a/.github/workflows/tmp0.yml b/.github/workflows/tmp0.yml
deleted file mode 100644
index b2c343efe7..0000000000
--- a/.github/workflows/tmp0.yml
+++ /dev/null
@@ -1,233 +0,0 @@
-###
-# Build hdf4, hdf5 dependencies and cache them in a combined directory.
-###
-
-name: Run netCDF Tests
-
-on: [pull_request]
-
-jobs:
-
-  build-deps:
-
-    runs-on: ubuntu-latest
-
-    strategy:
-      matrix:
-        hdf5: [ 1.8.21, 1.10.7 ]
-
-    steps:
-
-      - uses: actions/checkout@v2
-
-      - name: Install System dependencies
-        shell: bash -l {0}
-        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
-
-        ###
-        # Installing libhdf4 and libhdf5
-        ###
-      - name: Cache libhdf5-${{ matrix.hdf5 }}
-        id: cache-hdf5
-        uses: actions/cache@v2
-        with:
-          path: ~/environments/${{ matrix.hdf5 }}
-          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
-
-
-      - name: Build libhdf5-${{ matrix.hdf5 }}
-        if: steps.cache-hdf5.outputs.cache-hit != 'true'
-        run: |
-          set -x
-          wget https://support.hdfgroup.org/ftp/HDF/releases/HDF4.2.15/src/hdf-4.2.15.tar.bz2
-          tar -jxf hdf-4.2.15.tar.bz2
-          pushd hdf-4.2.15
-          ./configure --prefix=${HOME}/environments/${{ matrix.hdf5 }} --disable-static --enable-shared --disable-fortran --disable-netcdf --with-szlib
-          make -j
-          make install -j
-          popd
-          wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-$(echo ${{ matrix.hdf5 }} | cut -d. -f 1,2)/hdf5-${{ matrix.hdf5 }}/src/hdf5-${{ matrix.hdf5 }}.tar.bz2
-          tar -jxf hdf5-${{ matrix.hdf5 }}.tar.bz2
-          pushd hdf5-${{ matrix.hdf5 }}
-          ./configure --disable-static --enable-shared --prefix=${HOME}/environments/${{ matrix.hdf5 }} --enable-hl --with-szlib
-          make -j
-          make install -j
-          popd
-
-
-  nc-autotools:
-
-    needs: build-deps
-    runs-on: ubuntu-latest
-
-    strategy:
-      matrix:
-        hdf5: [ 1.8.21, 1.10.7 ]
-        use_nc4: [ nc3, nc4 ]
-        use_dap: [ dap_off, dap_on ]
-    steps:
-
-      - uses: actions/checkout@v2
-
-      - name: Install System dependencies
-        shell: bash -l {0}
-        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
-
-###
-# Set Environmental Variables
-###
-
-      - run: echo "CFLAGS=-I${HOME}/environments/${{ matrix.hdf5 }}/include" >> $GITHUB_ENV
-      - run: echo "LDFLAGS=-L${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
-      - run: echo "LD_LIBRARY_PATH=${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
-      - run: |
-          echo "ENABLE_HDF4=--disable-hdf4" >> $GITHUB_ENV
-          echo "ENABLE_NC4=--disable-netcdf-4" >> $GITHUB_ENV
-        if: matrix.use_nc4 == 'nc3'
-      - run: |
-          echo "ENABLE_HDF4=--enable-hdf4" >> $GITHUB_ENV
-          echo "ENABLE_NC4=--enable-netcdf-4" >> $GITHUB_ENV
-        if: matrix.use_nc4 == 'nc4'
-      - run: echo "ENABLE_DAP=--disable-dap" >> $GITHUB_ENV
-        if: matrix.use_dap == 'dap_off'
-      - run: echo "ENABLE_DAP=--enable-dap" >> $GITHUB_ENV
-        if: matrix.use_dap == 'dap_on'
-
-###
-# Fetch Cache
-###
-
-      - name: Fetch HDF Cache
-        id: cache-hdf
-        uses: actions/cache@v2
-        with:
-          path: ~/environments/${{ matrix.hdf5 }}
-          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
-
-      - name: Check Cache
-        shell: bash -l {0}
-        run: ls ${HOME}/environments && ls ${HOME}/environments/${{ matrix.hdf5 }} && ls ${HOME}/environments/${{ matrix.hdf5}}/lib
-
-###
-# Configure and build
-###
-
-      - name: Run autoconf
-        shell: bash -l {0}
-        run: autoreconf -if
-
-      - name: Configure
-        shell: bash -l {0}
-        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} ./configure ${ENABLE_HDF4} ${ENABLE_NC4} ${ENABLE_DAP}
-
-      - name: Look at config.log if error
-        shell: bash -l {0}
-        run: cat config.log
-        if: ${{ failure() }}
-
-      - name: Print Summary
-        shell: bash -l {0}
-        run: cat libnetcdf.settings
-
-      - name: Build Library and Utilities
-        shell: bash -l {0}
-        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j
-        if: ${{ success() }}
-
-      - name: Build Tests
-        shell: bash -l {0}
-        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check TESTS="" -j
-        if: ${{ success() }}
-
-      - name: Run Tests
-        shell: bash -l {0}
-        run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check -j
-        if: ${{ success() }}
-
-      #- name: Make Distcheck
-      #  shell: bash -l {0}
-      #  run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make distcheck -j
-
-  nc-cmake:
-
-    needs: build-deps
-    runs-on: ubuntu-latest
-
-    strategy:
-      matrix:
-        hdf5: [ 1.8.21, 1.10.7 ]
-        use_nc4: [ nc3, nc4 ]
-        use_dap: [ dap_off, dap_on ]
-    steps:
-
-      - uses: actions/checkout@v2
-
-      - name: Install System dependencies
-        shell: bash -l {0}
-        run: sudo apt update && sudo apt install -y libaec-dev zlib1g-dev automake autoconf libcurl4-openssl-dev libjpeg-dev wget curl bzip2 m4 flex bison cmake
-
-###
-# Set Environmental Variables
-###
-
-      - run: echo "CMAKE_PREFIX_PATH=${HOME}/environments/${{ matrix.hdf5 }}/" >> $GITHUB_ENV
-      - run: echo "LD_LIBRARY_PATH=${HOME}/environments/${{ matrix.hdf5 }}/lib" >> $GITHUB_ENV
-      - run: |
-          echo "ENABLE_HDF4=OFF" >> $GITHUB_ENV
-          echo "ENABLE_NC4=OFF" >> $GITHUB_ENV
-        if: matrix.use_nc4 == 'nc3'
-      - run: |
-          echo "ENABLE_HDF4=ON" >> $GITHUB_ENV
-          echo "ENABLE_NC4=ON" >> $GITHUB_ENV
-        if: matrix.use_nc4 == 'nc4'
-      - run: echo "ENABLE_DAP=OFF" >> $GITHUB_ENV
-        if: matrix.use_dap == 'dap_off'
-      - run: echo "ENABLE_DAP=ON" >> $GITHUB_ENV
-        if: matrix.use_dap == 'dap_on'
-      - run: echo "CTEST_OUTPUT_ON_FAILURE=1" >> $GITHUB_ENV
-
-###
-# Fetch Cache
-###
-
-      - name: Fetch HDF Cache
-        id: cache-hdf5
-        uses: actions/cache@v2
-        with:
-          path: ~/environments/${{ matrix.hdf5 }}
-          key: hdf5-${{ runner.os }}-${{ matrix.hdf5 }}
-
-      - name: Check Cache
-        shell: bash -l {0}
-        run: ls ${HOME}/environments && ls ${HOME}/environments/${{ matrix.hdf5 }} && ls ${HOME}/environments/${{ matrix.hdf5}}/lib
-
-###
-# Configure and build
-###
-
-      - name: Perform out-of-directory build
-        shell: bash -l {0}
-        run: |
-          mkdir build
-          cd build
-          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} cmake .. -DENABLE_HDF4=${ENABLE_HDF4} -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH} -DENABLE_DAP=${ENABLE_DAP} -DENABLE_NETCDF_4=${ENABLE_NC4}
-
-      - name: Print Summary
-        shell: bash -l {0}
-        run: |
-          cd build
-          cat libnetcdf.settings
-
-      - name: Build All
-        shell: bash -l {0}
-        run: |
-          cd build
-          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j 12
-        if: ${{ success() }}
-
-      - name: Run Tests
-        shell: bash -l {0}
-        run: |
-          cd build
-          LD_LIBRARY_PATH=${LD_LIBRARY_PATH} ctest -j 12 .
-        if: ${{ success() }}

From 7b3128b3e4b496f332b4845ecc037c790460e62f Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 6 Apr 2022 15:54:19 -0600
Subject: [PATCH 10/17] update file permission

---
 nczarr_test/ref_jsonconvention.cdl | 13 +++++++++++++
 nczarr_test/run_jsonconvention.sh  | 31 ++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 nczarr_test/ref_jsonconvention.cdl
 create mode 100755 nczarr_test/run_jsonconvention.sh

diff --git a/nczarr_test/ref_jsonconvention.cdl b/nczarr_test/ref_jsonconvention.cdl
new file mode 100644
index 0000000000..c4a52b8104
--- /dev/null
+++ b/nczarr_test/ref_jsonconvention.cdl
@@ -0,0 +1,13 @@
+netcdf tmp_jsonconvention {
+dimensions:
+	d1 = 1 ;
+variables:
+	int v(d1) ;
+		v:varconvention = "{\n\"key1\": [1,2,3], \"key2\": {\"key3\": \"abc\"}}" ;
+
+// global attributes:
+		:grpconvention = "{\"key1\": [1,2,3], \n\"key2\": {\"key3\": \"abc\"}}" ;
+data:
+
+ v = _ ;
+}
diff --git a/nczarr_test/run_jsonconvention.sh b/nczarr_test/run_jsonconvention.sh
new file mode 100755
index 0000000000..5d96052f1e
--- /dev/null
+++ b/nczarr_test/run_jsonconvention.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+if test "x$srcdir" = x ; then srcdir=`pwd`; fi 
+. ../test_common.sh
+
+. "$srcdir/test_nczarr.sh"
+
+# This shell script tests support for:
+# read/write using json convention
+
+set -e
+
+testcase() {
+zext=$1
+
+echo "*** Test: write then read using json convention"
+fileargs tmp_jsonconvention "mode=nczarr,$zext"
+deletemap $zext $file
+${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_jsonconvention.cdl
+${NCDUMP} $fileurl > tmp_jsonconvention_${zext}.cdl
+# remove '\n' from ref file before comparing
+rm -f tmp_jsonconvention.cdl
+sed -e 's|\\n||g' < ref_jsonconvention.cdl > tmp_jsonconvention.cdl
+diff -b ${srcdir}/tmp_jsonconvention.cdl tmp_jsonconvention_${zext}.cdl
+}
+
+testcase file
+#if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testcase zip; fi
+#if test "x$FEATURE_S3TESTS" = xyes ; then testcase s3; fi
+
+exit 0

From 90fe06592ec3a71e2fa72e318e85e66fd9b27b77 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 6 Apr 2022 15:54:42 -0600
Subject: [PATCH 11/17] ckp

---
 include/ncjson.h      |   5 +-
 libdispatch/ncjson.c  |  43 ++++++++++++----
 libnczarr/zsync.c     | 114 ++++++++++++++++++++++++++----------------
 nczarr_test/zisjson.c | 109 ++++++++++++++++++++++++++++++++++++----
 4 files changed, 208 insertions(+), 63 deletions(-)

diff --git a/include/ncjson.h b/include/ncjson.h
index 86a91ef172..c065bc2ed1 100644
--- a/include/ncjson.h
+++ b/include/ncjson.h
@@ -56,9 +56,12 @@ struct NCJconst {int bval; long long ival; double dval; char* sval;};
 extern "C" {
 #endif
 
-/* Parse a JSON string */
+/* Parse a string to NCjson*/
 DLLEXPORT int NCJparse(const char* text, unsigned flags, NCjson** jsonp);
 
+/* Parse a counted string to NCjson*/
+DLLEXPORT int NCJparsen(size_t len, const char* text, unsigned flags, NCjson** jsonp);
+
 /* Reclaim a JSON tree */
 DLLEXPORT extern void NCJreclaim(NCjson* json);
 
diff --git a/libdispatch/ncjson.c b/libdispatch/ncjson.c
index e862175648..c0f85dd3f4 100644
--- a/libdispatch/ncjson.c
+++ b/libdispatch/ncjson.c
@@ -94,6 +94,7 @@ static int NCJyytext(NCJparser*, char* start, size_t pdlen);
 static void NCJreclaimArray(struct NCjlist*);
 static void NCJreclaimDict(struct NCjlist*);
 static int NCJunescape(NCJparser* parser);
+static int unescape1(int c);
 static int listappend(struct NCjlist* list, NCjson* element);
 
 #ifndef NETCDF_JSON_H
@@ -109,24 +110,28 @@ static int bytesappendc(NCJbuf* bufp, const char c);
 
 int
 NCJparse(const char* text, unsigned flags, NCjson** jsonp)
+{
+    return NCJparsen(strlen(text),text,flags,jsonp);
+}
+
+int
+NCJparsen(size_t len, const char* text, unsigned flags, NCjson** jsonp)
 {
     int stat = NCJ_OK;
-    size_t len;
     NCJparser* parser = NULL;
     NCjson* json = NULL;
 
     /* Need at least 1 character of input */
-    if(text == NULL || text[0] == '\0')
+    if(len == 0 || text == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
     if(jsonp == NULL) goto done;
     parser = calloc(1,sizeof(NCJparser));
     if(parser == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
-    len = strlen(text);
     parser->text = (char*)malloc(len+1+1);
     if(parser->text == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
-    strcpy(parser->text,text);
+    memcpy(parser->text,text,len);
     parser->text[len] = '\0';
     parser->text[len+1] = '\0';
     parser->pos = &parser->text[0];
@@ -334,16 +339,21 @@ NCJlex(NCJparser* parser)
 	c = *parser->pos;
 	if(c == '\0') {
 	    token = NCJ_EOF;
-	} else if(c <= ' ' || c == '\177') {
+	} else if(c <= ' ' || c == '\177') {/* ignore whitespace */
+	    parser->pos++;
+	    continue;
+	} else if(c == NCJ_ESCAPE) {
 	    parser->pos++;
-	    continue; /* ignore whitespace */
+	    c = *parser->pos;
+	    *parser->pos = unescape1(c);
+	    continue;
 	} else if(strchr(JSON_WORD, c) != NULL) {
 	    start = parser->pos;
 	    for(;;) {
 		c = *parser->pos++;
 		if(c == '\0' || strchr(JSON_WORD,c) == NULL) break; /* end of word */
 	    }
-	    /* Pushback c if not whitespace */
+	    /* Pushback c */
 	    parser->pos--;
 	    count = ((parser->pos) - start);
 	    if(NCJyytext(parser,start,count)) goto done;
@@ -604,6 +614,21 @@ NCJunescape(NCJparser* parser)
     return NCJTHROW(NCJ_OK);    
 }
 
+/* Unescape a single character */
+static int
+unescape1(int c)
+{
+    switch (c) {
+    case 'b': c = '\b'; break;
+    case 'f': c = '\f'; break;
+    case 'n': c = '\n'; break;
+    case 'r': c = '\r'; break;
+    case 't': c = '\t'; break;
+    default: c = c; break;/* technically not Json conformant */
+    }
+    return c;
+}
+
 #ifdef NCJDEBUG
 static char*
 tokenname(int token)
@@ -896,7 +921,7 @@ NCJunparseR(const NCjson* json, NCJbuf* buf, unsigned flags)
 	if(json->list.len > 0 && json->list.contents != NULL) {
 	    int shortlist = 0;
 	    for(i=0;!shortlist && i < json->list.len;i+=2) {
-		if(i > 0) bytesappendc(buf,NCJ_COMMA);
+		if(i > 0) {bytesappendc(buf,NCJ_COMMA);bytesappendc(buf,' ');};
 		NCJunparseR(json->list.contents[i],buf,flags); /* key */
 		bytesappendc(buf,NCJ_COLON);
 		bytesappendc(buf,' ');
@@ -945,7 +970,7 @@ escape(const char* text, NCJbuf* buf)
 	case '\n': replace = 'n'; break;
 	case '\r': replace = 'r'; break;
 	case '\t': replace = 't'; break;
-	case NCJ_QUOTE: replace = '\''; break;
+	case NCJ_QUOTE: replace = '\"'; break;
 	case NCJ_ESCAPE: replace = '\\'; break;
 	default: break;
 	}
diff --git a/libnczarr/zsync.c b/libnczarr/zsync.c
index 7711032499..aa564211e9 100644
--- a/libnczarr/zsync.c
+++ b/libnczarr/zsync.c
@@ -12,6 +12,10 @@
 
 #undef FILLONCLOSE
 
+/*mnemonics*/
+#define DICTOPEN '{'
+#define DICTCLOSE '}'
+
 /* Forward */
 static int ncz_collect_dims(NC_FILE_INFO_T* file, NC_GRP_INFO_T* grp, NCjson** jdimsp);
 static int ncz_sync_var(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int isclose);
@@ -37,6 +41,8 @@ static int computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, s
 static int inferattrtype(NCjson* values, nc_type* typeidp);
 static int mininttype(unsigned long long u64, int negative);
 static int computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarray, int ndims, NClist* dimnames, size64_t* shapes, NC_DIM_INFO_T** dims);
+static int read_dict(NCjson** valuep);
+static int write_dict(size_t len, const void* data, NCjson** jsonp);
 
 /**************************************************/
 /**************************************************/
@@ -776,6 +782,7 @@ ncz_sync_atts(NC_FILE_INFO_T* file, NC_OBJ* container, NCindex* attlist, int isc
 Note that this does not push to the file.
 Also note that attributes of length 1 are stored as singletons, not arrays.
 This is to be more consistent with pure zarr.
+Also implements the JSON dictionary convention.
 @param attlist - [in] the attributes to dictify
 @param jattrsp - [out] the json'ized att list
 @return NC_NOERR
@@ -785,7 +792,7 @@ static int
 ncz_jsonize_atts(NCindex* attlist, NCjson** jattrsp)
 {
     int stat = NC_NOERR;
-    int i;
+    int i, isdict;
     NCjson* jattrs = NULL;
     NCjson* akey = NULL;
     NCjson* jdata = NULL;
@@ -795,9 +802,18 @@ ncz_jsonize_atts(NCindex* attlist, NCjson** jattrsp)
     /* Iterate over the attribute list */
     for(i=0;i<ncindexsize(attlist);i++) {
 	NC_ATT_INFO_T* att = (NC_ATT_INFO_T*)ncindexith(attlist,i);
+	isdict = 0;
 	/* Create the attribute dict value*/
-	if((stat = NCZ_stringconvert(att->nc_typeid,att->len,att->data,&jdata)))
-	    goto done;
+	if(att->nc_typeid == NC_CHAR
+	   && ((char*)att->data)[0] == DICTOPEN
+   	   && ((char*)att->data)[att->len-1] == DICTCLOSE) {
+	    /* this is subject to the JSON dictionary convention? */
+	    if(write_dict(att->len,att->data,&jdata)==NC_NOERR) isdict=1;
+	}
+	if(!isdict) {
+	    if((stat = NCZ_stringconvert(att->nc_typeid,att->len,att->data,&jdata)))
+	        goto done;
+	}
 	if((stat = NCJinsert(jattrs,att->hdr.name,jdata))) goto done;
 	jdata = NULL;
     }
@@ -1011,12 +1027,18 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
     if(typeid == NC_NAT) if((stat = inferattrtype(values,&typeid))) goto done;
     if(typeid == NC_NAT) {stat = NC_EBADTYPE; goto done;}
 
+    if((stat = NC4_inq_atomic_type(typeid, NULL, &typelen)))
+        goto done;
+
     /* Collect the length of the attribute; might be a singleton  */
     switch (NCJsort(values)) {
-    case NCJ_DICT: stat = NC_ENCZARR; goto done;
     case NCJ_ARRAY:
 	count = NCJlength(values);
 	break;
+    case NCJ_DICT:
+	/* Apply the JSON dictionary convention and convert to string */
+	if((stat = read_dict(&values))) goto done;
+	/* fall thru */
     case NCJ_STRING: /* requires special handling as an array of characters; also look out for empty string */
 	if(typeid == NC_CHAR) {
 	    count = strlen(NCJstring(values));
@@ -1029,10 +1051,8 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
 	break;
     }
 
-    if(count > 0) {
+    if(count > 0 && data == NULL) {
         /* Allocate data space */
-        if((stat = NC4_inq_atomic_type(typeid, NULL, &typelen)))
-	    goto done;
         if(typeid == NC_CHAR)
             data = malloc(typelen*(count+1));
         else
@@ -1079,7 +1099,9 @@ inferattrtype(NCjson* value, nc_type* typeidp)
     case NCJ_NULL:
         typeid = NC_CHAR;
 	return NC_NOERR;
-    case NCJ_DICT: /* fall thru */
+    case NCJ_DICT:
+    	typeid = NC_CHAR;
+	goto done;
     case NCJ_UNDEF:
 	return NC_EINVAL;
     default: /* atomic */
@@ -2289,42 +2311,48 @@ computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarra
     return THROW(stat);
 }
 
-#if 0
-Not currently used
-Special compatibility case:
-       if the value of the attribute is a dictionary,
-       or an array with non-atomic values, then
-       then stringify it and pretend it is of char type.
-/* Return 1 if this json is not an
-atomic value or an array of atomic values.
-That is, it does not look like valid
-attribute data.
+/**
+Implement the JSON convention for dictionaries.
+
+Reading: If the value of the attribute is a dictionary, then stringify
+         it as the value and make the attribute be of type "char".
+
+Writing: if the attribute is of type char and looks like a JSON dictionary,
+	 then parse it as JSON and use that as its value in .zattrs.
 */
+
 static int
-iscomplexjson(NCjson* j)
+read_dict(NCjson** jvaluep)
 {
-    int i;
-    switch(NCJsort(j)) {
-    case NCJ_ARRAY:
-	/* verify that the elements of the array are not complex */
-	for(i=0;i<NCJlength(j);i++) {
-	    switch (NCJith(j,NCJsort(i)))) {
-	    case NCJ_DICT:
-    	    case NCJ_ARRAY:
-	    case NCJ_UNDEF:
-	    case NCJ_NULL:
-		return 1;
-	    default: break;
-	    }
-	}
-	return 0;
-    case NCJ_DICT:
-    case NCJ_UNDEF:
-    case NCJ_NULL:
-	break;
-    default:
-        return 0;
-    }
-    return 1;
+    int stat = NC_NOERR;
+    NCjson* jdict = NULL;
+    NCjson* jtext = NULL;
+    char* text = NULL;
+
+    assert(jvaluep != NULL && *jvaluep != NULL);
+    jdict = *jvaluep;
+    if(NCJsort(jdict) != NCJ_DICT)  {stat = NC_EINVAL; goto done;}
+    if(NCJunparse(jdict,0,&text)) {stat = NC_EINVAL; goto done;}
+    if(NCJnewstring(NCJ_STRING,text,&jtext)) {stat = NC_EINVAL; goto done;}
+    *jvaluep = jtext;
+done:
+    nullfree(text);
+    return stat;
 }
-#endif
+
+static int
+write_dict(size_t len, const void* data, NCjson** jsonp)
+{
+    int stat = NC_NOERR;
+    NCjson* jdict = NULL;
+
+    assert(jsonp != NULL);
+    if(NCJparsen(len,(char*)data,0,&jdict))
+        {stat = NC_EINVAL; goto done;}
+    if(NCJsort(jdict) != NCJ_DICT)
+        {stat = NC_EINVAL; goto done;}
+    *jsonp = jdict;
+done:
+    return stat;
+}
+
diff --git a/nczarr_test/zisjson.c b/nczarr_test/zisjson.c
index 186ae45f6c..e13c69f23f 100644
--- a/nczarr_test/zisjson.c
+++ b/nczarr_test/zisjson.c
@@ -7,36 +7,122 @@
    Output 1 or 0.
 */
 
-#include "config.h"
+
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 
 #include "stdlib.h"
 #include "stdio.h"
+#include "string.h"
 
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>
 #endif
 
+#if defined(_WIN32) && !defined(__MINGW32__)
+#include "XGetopt.h"
+#else
+#include <getopt.h>
+#endif
+
 #include "netcdf.h"
 #include "nclist.h"
 #include "ncjson.h"
 
 #define MAXREAD 8192
 
+/* Command line options */
+struct Jsonpptions {
+    int trace;
+} jsonoptions;
+
+static const char*
+sortname(int thesort)
+{
+    switch(thesort) {
+    default: break;
+    case NCJ_INT: return "NCJ_INT";
+    case NCJ_DOUBLE: return "NCJ_DOUBLE";
+    case NCJ_BOOLEAN: return "NCJ_BOOLEAN";
+    case NCJ_STRING: return "NCJ_STRING";
+    case NCJ_DICT: return "NCJ_DICT";
+    case NCJ_ARRAY: return "NCJ_ARRAY";
+    case NCJ_NULL: return "NCJ_NULL";
+    }
+    return "?";
+}
+
+
+static void
+jsontrace(NCjson* json, int depth)
+{
+    int i;
+    if(json == NULL) goto done;
+    printf("[%d] sort=%s",depth,sortname(NCJsort(json)));
+    switch(NCJsort(json)) {
+    case NCJ_INT:
+    case NCJ_DOUBLE:
+    case NCJ_BOOLEAN:
+    case NCJ_STRING:
+	printf(" string=|%s|\n",NCJstring(json));
+	break;
+    case NCJ_NULL:
+	printf("\n");
+	break;
+    case NCJ_ARRAY:
+	printf("\n");
+	for(i=0;i<NCJlength(json);i++)
+	    jsontrace(NCJith(json,i),depth+1);
+	break;
+    case NCJ_DICT:
+	printf("\n");
+	for(i=0;i<NCJlength(json);i+=2) {
+	    jsontrace(NCJith(json,i),depth+1);
+	    jsontrace(NCJith(json,i+1),depth+1);
+	} break;
+    default: break;
+    }
+done:
+    return;
+}
+
 int
 main(int argc, char** argv)
 {
     int stat = NC_NOERR;
     char text[MAXREAD+1];
     NCjson* json = NULL;
-    int i, red;
+    int i, red, c;
     FILE* f = NULL;
 
-    if(argc > 1) {
-	/* use argv[1] as input */
-	f = fopen(argv[1],"r");
-	if(f == NULL) {fprintf(stderr,"No such file: %s\n",argv[1]); exit(1);}
-    } else
-        f = stdin;
+    nc_initialize();
+    memset((void*)&jsonoptions,0,sizeof(jsonoptions));
+
+    while ((c = getopt(argc, argv, "t")) != EOF) {
+	switch(c) {
+	case 't': jsonoptions.trace = 1; break;
+	case '?':
+	   fprintf(stderr,"unknown option\n");
+	   exit(1);
+	}
+    }	
+
+    /* get file argument */
+    argc -= optind;
+    argv += optind;
+
+    if (argc > 1) {
+	fprintf(stderr, "zisjson: only one input file argument permitted\n");
+	exit(1);
+    }
+    if (argc == 0)
+	f = stdin;
+    else {
+        /* use argv[0] as input */
+        f = fopen(argv[0],"r");
+        if(f == NULL) {fprintf(stderr,"No such file: %s\n",argv[1]); exit(1);}
+    }
 
     /* Read json from stdin */
     for(i=0;;i++) {
@@ -54,9 +140,12 @@ main(int argc, char** argv)
 	stat = NC_EEMPTY;
     } else {
         stat = NCJparse(text,0,&json);
-	NCJreclaim(json);
+	if(!stat) {
+	    if(jsonoptions.trace) jsontrace(json,0);
+	    NCJreclaim(json);
+	}
     }
-    printf("%d",(stat==NC_NOERR?1:0)); /* parse success|failure */    
+    printf("%d",(stat?0:1)); /* parse success|failure */    
     if(f != stdin) fclose(f);
     return 0;
 }

From d5798aff2eef6e83ab5cf2e36e854f6119df117b Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 6 Apr 2022 15:57:17 -0600
Subject: [PATCH 12/17] testing

---
 .github/workflows/run_tests_ubuntu.yml    | 2 +-
 .github/workflows/run_tests_win_mingw.yml | 2 +-
 nczarr_test/Makefile.am                   | 5 +++--
 nczarr_test/run_jsonconvention.sh         | 4 ++--
 4 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index 93f402c150..15cbf017f6 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [ pull_request ]
+on: [pull_request,push]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_win_mingw.yml b/.github/workflows/run_tests_win_mingw.yml
index 913920a3aa..aaf999e6a4 100644
--- a/.github/workflows/run_tests_win_mingw.yml
+++ b/.github/workflows/run_tests_win_mingw.yml
@@ -7,7 +7,7 @@
 name: Run MSYS2, MinGW64-based Tests
 
 
-on: [pull_request]
+on: [pull_request,push]
 
 jobs:
 
diff --git a/nczarr_test/Makefile.am b/nczarr_test/Makefile.am
index e9efa73f7e..968b5a821e 100644
--- a/nczarr_test/Makefile.am
+++ b/nczarr_test/Makefile.am
@@ -61,6 +61,7 @@ TESTS += run_purezarr.sh
 TESTS += run_interop.sh
 TESTS += run_misc.sh
 TESTS += run_nczarr_fill.sh
+TESTS += run_jsonconvention.sh
 
 endif
 
@@ -127,7 +128,7 @@ run_ut_map.sh run_ut_mapapi.sh run_ut_misc.sh run_ut_chunk.sh run_ncgen4.sh \
 run_nccopyz.sh run_fillonlyz.sh run_chunkcases.sh test_nczarr.sh run_perf_chunks1.sh run_s3_cleanup.sh \
 run_purezarr.sh run_interop.sh run_misc.sh \
 run_filter.sh run_specific_filters.sh \
-run_newformat.sh run_nczarr_fill.sh run_quantize.sh
+run_newformat.sh run_nczarr_fill.sh run_quantize.sh run_jsonconvention.sh
 
 EXTRA_DIST += \
 ref_ut_map_create.cdl ref_ut_map_writedata.cdl ref_ut_map_writemeta2.cdl ref_ut_map_writemeta.cdl \
@@ -147,7 +148,7 @@ ref_bzip2.cdl ref_filtered.cdl ref_multi.cdl \
 ref_any.cdl ref_oldformat.cdl ref_oldformat.zip ref_newformatpure.cdl \
 ref_quotes.zip ref_quotes.cdl \
 ref_groups.h5 ref_byte.zarr.zip ref_byte_fill_value_null.zarr.zip \
-ref_groups_regular.cdl ref_byte.cdl ref_byte_fill_value_null.cdl
+ref_groups_regular.cdl ref_byte.cdl ref_byte_fill_value_null.cdl ref_jsonconvention.cdl
 
 # Interoperability files
 EXTRA_DIST += ref_power_901_constants.zip ref_power_901_constants.cdl ref_quotes.zip ref_quotes.cdl 
diff --git a/nczarr_test/run_jsonconvention.sh b/nczarr_test/run_jsonconvention.sh
index 5d96052f1e..fc9c84d1dc 100755
--- a/nczarr_test/run_jsonconvention.sh
+++ b/nczarr_test/run_jsonconvention.sh
@@ -25,7 +25,7 @@ diff -b ${srcdir}/tmp_jsonconvention.cdl tmp_jsonconvention_${zext}.cdl
 }
 
 testcase file
-#if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testcase zip; fi
-#if test "x$FEATURE_S3TESTS" = xyes ; then testcase s3; fi
+if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testcase zip; fi
+if test "x$FEATURE_S3TESTS" = xyes ; then testcase s3; fi
 
 exit 0

From 56167a777541f8aa73513163cc7d80cef222fa6b Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 6 Apr 2022 17:16:40 -0600
Subject: [PATCH 13/17] final

---
 nc_test4/tst_broken_files.c       | 3 ++-
 nczarr_test/CMakeLists.txt        | 3 ++-
 nczarr_test/run_jsonconvention.sh | 4 ++--
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/nc_test4/tst_broken_files.c b/nc_test4/tst_broken_files.c
index af68b26905..22bf3e8802 100644
--- a/nc_test4/tst_broken_files.c
+++ b/nc_test4/tst_broken_files.c
@@ -28,7 +28,8 @@ main() {
         fclose(fp);
 
         int  ncid;
-        if (nc_open(FILE_NAME, 0, &ncid) != NC_EHDFERR) ERR;
+        int stat=nc_open(FILE_NAME, 0, &ncid);
+        if (stat != NC_EHDFERR && stat != NC_ENOTNC) ERR;
     }
 
     {
diff --git a/nczarr_test/CMakeLists.txt b/nczarr_test/CMakeLists.txt
index 03db4557fb..ef31d15df1 100644
--- a/nczarr_test/CMakeLists.txt
+++ b/nczarr_test/CMakeLists.txt
@@ -73,7 +73,7 @@ IF(ENABLE_TESTS)
   BUILD_BIN_TEST(zmapio ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zmapio PUBLIC ../libnczarr)
   BUILD_BIN_TEST(zhex)
-  BUILD_BIN_TEST(zisjson)
+  BUILD_BIN_TEST(zisjson ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zisjson PUBLIC ../libnczarr)
   BUILD_BIN_TEST(zs3parse ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zs3parse PUBLIC ../libnczarr)
@@ -108,6 +108,7 @@ IF(ENABLE_TESTS)
     add_sh_test(nczarr_test run_interop)
     add_sh_test(nczarr_test run_misc)
     add_sh_test(nczarr_test run_nczarr_fill)
+    add_sh_test(nczarr_test run_jsonconvention)
 
     BUILD_BIN_TEST(test_quantize ${TSTCOMMONSRC})
     add_sh_test(nczarr_test run_quantize)
diff --git a/nczarr_test/run_jsonconvention.sh b/nczarr_test/run_jsonconvention.sh
index fc9c84d1dc..7cf786a685 100755
--- a/nczarr_test/run_jsonconvention.sh
+++ b/nczarr_test/run_jsonconvention.sh
@@ -20,8 +20,8 @@ ${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_jsonconvention.cdl
 ${NCDUMP} $fileurl > tmp_jsonconvention_${zext}.cdl
 # remove '\n' from ref file before comparing
 rm -f tmp_jsonconvention.cdl
-sed -e 's|\\n||g' < ref_jsonconvention.cdl > tmp_jsonconvention.cdl
-diff -b ${srcdir}/tmp_jsonconvention.cdl tmp_jsonconvention_${zext}.cdl
+sed -e 's|\\n||g' < ${srcdir}/ref_jsonconvention.cdl > tmp_jsonconvention.cdl
+diff -b tmp_jsonconvention.cdl tmp_jsonconvention_${zext}.cdl
 }
 
 testcase file

From 9f78be8bb8af0cbc2293b2e1cd8984894762cc6b Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Wed, 6 Apr 2022 18:22:59 -0600
Subject: [PATCH 14/17] Allow the read/write of JSON-valued Zarr attributes.

A number of other packages that read/write Zarr insert
attributes whose value is a dictionary containing specialized
information.  An example is the GDAL Driver convention (see
https://gdal.org/drivers/raster/zarr.html).

In order to handle such attributes, this PR enforces a special
convention. It applies to both pure Zarr an NCZarr format as
written by the netdf-c library.

The convention is as follows:

## Reading
Suppose an attribute is read from *.zattrs* and it has a JSON
value that is a a dictionary.  In this case, the JSON dictionary
is converted to a string value.  It then appears in the netcdf-c
API as if it is a character valued attribute of the same name,
and whose value is the "stringified" dictionary.

# Writing
Suppose an attribute is of type character and its *value* *looks like*
a JSON dictionary. In this case, it is parsed to JSON
and written as the value of the attribute in the NCZarr file.
Here the *value* is the concatenation of all the characters
in the attributes netcdf-c value.
The term "looks like" means that the *value*'s first character is
"{", its last value is "}", and it can be successfully parsed
by a JSON parser.

A test case, *nczarr_test/run_jsonconventions.sh* was also added.

## Misc. Unrelated Changes

1. Fix an error in nc_test4/tst_broken_files.c
2. Modify the internal JSON parser API.
3. Modify the nczarr_test/zisjson program is modified to support
   this convention.
---
 .github/workflows/run_tests_osx.yml    |   2 +-
 .github/workflows/run_tests_ubuntu.yml |   2 +-
 include/ncjson.h                       |   5 +-
 libdispatch/ncjson.c                   |  43 ++++++++--
 libnczarr/zsync.c                      | 114 +++++++++++++++----------
 nc_test4/tst_broken_files.c            |   3 +-
 nczarr_test/CMakeLists.txt             |   3 +-
 nczarr_test/Makefile.am                |   5 +-
 nczarr_test/ref_jsonconvention.cdl     |  13 +++
 nczarr_test/run_jsonconvention.sh      |  31 +++++++
 nczarr_test/zisjson.c                  | 109 ++++++++++++++++++++---
 11 files changed, 261 insertions(+), 69 deletions(-)
 create mode 100644 nczarr_test/ref_jsonconvention.cdl
 create mode 100755 nczarr_test/run_jsonconvention.sh

diff --git a/.github/workflows/run_tests_osx.yml b/.github/workflows/run_tests_osx.yml
index aa35ae8795..8e9d65a6fb 100644
--- a/.github/workflows/run_tests_osx.yml
+++ b/.github/workflows/run_tests_osx.yml
@@ -7,7 +7,7 @@
 name: Run macOS-based netCDF Tests
 
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index a1f6544249..92b5bfca66 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [ pull_request ]
+on: [pull_request]
 
 jobs:
 
diff --git a/include/ncjson.h b/include/ncjson.h
index 86a91ef172..c065bc2ed1 100644
--- a/include/ncjson.h
+++ b/include/ncjson.h
@@ -56,9 +56,12 @@ struct NCJconst {int bval; long long ival; double dval; char* sval;};
 extern "C" {
 #endif
 
-/* Parse a JSON string */
+/* Parse a string to NCjson*/
 DLLEXPORT int NCJparse(const char* text, unsigned flags, NCjson** jsonp);
 
+/* Parse a counted string to NCjson*/
+DLLEXPORT int NCJparsen(size_t len, const char* text, unsigned flags, NCjson** jsonp);
+
 /* Reclaim a JSON tree */
 DLLEXPORT extern void NCJreclaim(NCjson* json);
 
diff --git a/libdispatch/ncjson.c b/libdispatch/ncjson.c
index e862175648..c0f85dd3f4 100644
--- a/libdispatch/ncjson.c
+++ b/libdispatch/ncjson.c
@@ -94,6 +94,7 @@ static int NCJyytext(NCJparser*, char* start, size_t pdlen);
 static void NCJreclaimArray(struct NCjlist*);
 static void NCJreclaimDict(struct NCjlist*);
 static int NCJunescape(NCJparser* parser);
+static int unescape1(int c);
 static int listappend(struct NCjlist* list, NCjson* element);
 
 #ifndef NETCDF_JSON_H
@@ -109,24 +110,28 @@ static int bytesappendc(NCJbuf* bufp, const char c);
 
 int
 NCJparse(const char* text, unsigned flags, NCjson** jsonp)
+{
+    return NCJparsen(strlen(text),text,flags,jsonp);
+}
+
+int
+NCJparsen(size_t len, const char* text, unsigned flags, NCjson** jsonp)
 {
     int stat = NCJ_OK;
-    size_t len;
     NCJparser* parser = NULL;
     NCjson* json = NULL;
 
     /* Need at least 1 character of input */
-    if(text == NULL || text[0] == '\0')
+    if(len == 0 || text == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
     if(jsonp == NULL) goto done;
     parser = calloc(1,sizeof(NCJparser));
     if(parser == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
-    len = strlen(text);
     parser->text = (char*)malloc(len+1+1);
     if(parser->text == NULL)
 	{stat = NCJTHROW(NCJ_ERR); goto done;}
-    strcpy(parser->text,text);
+    memcpy(parser->text,text,len);
     parser->text[len] = '\0';
     parser->text[len+1] = '\0';
     parser->pos = &parser->text[0];
@@ -334,16 +339,21 @@ NCJlex(NCJparser* parser)
 	c = *parser->pos;
 	if(c == '\0') {
 	    token = NCJ_EOF;
-	} else if(c <= ' ' || c == '\177') {
+	} else if(c <= ' ' || c == '\177') {/* ignore whitespace */
+	    parser->pos++;
+	    continue;
+	} else if(c == NCJ_ESCAPE) {
 	    parser->pos++;
-	    continue; /* ignore whitespace */
+	    c = *parser->pos;
+	    *parser->pos = unescape1(c);
+	    continue;
 	} else if(strchr(JSON_WORD, c) != NULL) {
 	    start = parser->pos;
 	    for(;;) {
 		c = *parser->pos++;
 		if(c == '\0' || strchr(JSON_WORD,c) == NULL) break; /* end of word */
 	    }
-	    /* Pushback c if not whitespace */
+	    /* Pushback c */
 	    parser->pos--;
 	    count = ((parser->pos) - start);
 	    if(NCJyytext(parser,start,count)) goto done;
@@ -604,6 +614,21 @@ NCJunescape(NCJparser* parser)
     return NCJTHROW(NCJ_OK);    
 }
 
+/* Unescape a single character */
+static int
+unescape1(int c)
+{
+    switch (c) {
+    case 'b': c = '\b'; break;
+    case 'f': c = '\f'; break;
+    case 'n': c = '\n'; break;
+    case 'r': c = '\r'; break;
+    case 't': c = '\t'; break;
+    default: c = c; break;/* technically not Json conformant */
+    }
+    return c;
+}
+
 #ifdef NCJDEBUG
 static char*
 tokenname(int token)
@@ -896,7 +921,7 @@ NCJunparseR(const NCjson* json, NCJbuf* buf, unsigned flags)
 	if(json->list.len > 0 && json->list.contents != NULL) {
 	    int shortlist = 0;
 	    for(i=0;!shortlist && i < json->list.len;i+=2) {
-		if(i > 0) bytesappendc(buf,NCJ_COMMA);
+		if(i > 0) {bytesappendc(buf,NCJ_COMMA);bytesappendc(buf,' ');};
 		NCJunparseR(json->list.contents[i],buf,flags); /* key */
 		bytesappendc(buf,NCJ_COLON);
 		bytesappendc(buf,' ');
@@ -945,7 +970,7 @@ escape(const char* text, NCJbuf* buf)
 	case '\n': replace = 'n'; break;
 	case '\r': replace = 'r'; break;
 	case '\t': replace = 't'; break;
-	case NCJ_QUOTE: replace = '\''; break;
+	case NCJ_QUOTE: replace = '\"'; break;
 	case NCJ_ESCAPE: replace = '\\'; break;
 	default: break;
 	}
diff --git a/libnczarr/zsync.c b/libnczarr/zsync.c
index 7711032499..aa564211e9 100644
--- a/libnczarr/zsync.c
+++ b/libnczarr/zsync.c
@@ -12,6 +12,10 @@
 
 #undef FILLONCLOSE
 
+/*mnemonics*/
+#define DICTOPEN '{'
+#define DICTCLOSE '}'
+
 /* Forward */
 static int ncz_collect_dims(NC_FILE_INFO_T* file, NC_GRP_INFO_T* grp, NCjson** jdimsp);
 static int ncz_sync_var(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int isclose);
@@ -37,6 +41,8 @@ static int computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, s
 static int inferattrtype(NCjson* values, nc_type* typeidp);
 static int mininttype(unsigned long long u64, int negative);
 static int computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarray, int ndims, NClist* dimnames, size64_t* shapes, NC_DIM_INFO_T** dims);
+static int read_dict(NCjson** valuep);
+static int write_dict(size_t len, const void* data, NCjson** jsonp);
 
 /**************************************************/
 /**************************************************/
@@ -776,6 +782,7 @@ ncz_sync_atts(NC_FILE_INFO_T* file, NC_OBJ* container, NCindex* attlist, int isc
 Note that this does not push to the file.
 Also note that attributes of length 1 are stored as singletons, not arrays.
 This is to be more consistent with pure zarr.
+Also implements the JSON dictionary convention.
 @param attlist - [in] the attributes to dictify
 @param jattrsp - [out] the json'ized att list
 @return NC_NOERR
@@ -785,7 +792,7 @@ static int
 ncz_jsonize_atts(NCindex* attlist, NCjson** jattrsp)
 {
     int stat = NC_NOERR;
-    int i;
+    int i, isdict;
     NCjson* jattrs = NULL;
     NCjson* akey = NULL;
     NCjson* jdata = NULL;
@@ -795,9 +802,18 @@ ncz_jsonize_atts(NCindex* attlist, NCjson** jattrsp)
     /* Iterate over the attribute list */
     for(i=0;i<ncindexsize(attlist);i++) {
 	NC_ATT_INFO_T* att = (NC_ATT_INFO_T*)ncindexith(attlist,i);
+	isdict = 0;
 	/* Create the attribute dict value*/
-	if((stat = NCZ_stringconvert(att->nc_typeid,att->len,att->data,&jdata)))
-	    goto done;
+	if(att->nc_typeid == NC_CHAR
+	   && ((char*)att->data)[0] == DICTOPEN
+   	   && ((char*)att->data)[att->len-1] == DICTCLOSE) {
+	    /* this is subject to the JSON dictionary convention? */
+	    if(write_dict(att->len,att->data,&jdata)==NC_NOERR) isdict=1;
+	}
+	if(!isdict) {
+	    if((stat = NCZ_stringconvert(att->nc_typeid,att->len,att->data,&jdata)))
+	        goto done;
+	}
 	if((stat = NCJinsert(jattrs,att->hdr.name,jdata))) goto done;
 	jdata = NULL;
     }
@@ -1011,12 +1027,18 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
     if(typeid == NC_NAT) if((stat = inferattrtype(values,&typeid))) goto done;
     if(typeid == NC_NAT) {stat = NC_EBADTYPE; goto done;}
 
+    if((stat = NC4_inq_atomic_type(typeid, NULL, &typelen)))
+        goto done;
+
     /* Collect the length of the attribute; might be a singleton  */
     switch (NCJsort(values)) {
-    case NCJ_DICT: stat = NC_ENCZARR; goto done;
     case NCJ_ARRAY:
 	count = NCJlength(values);
 	break;
+    case NCJ_DICT:
+	/* Apply the JSON dictionary convention and convert to string */
+	if((stat = read_dict(&values))) goto done;
+	/* fall thru */
     case NCJ_STRING: /* requires special handling as an array of characters; also look out for empty string */
 	if(typeid == NC_CHAR) {
 	    count = strlen(NCJstring(values));
@@ -1029,10 +1051,8 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
 	break;
     }
 
-    if(count > 0) {
+    if(count > 0 && data == NULL) {
         /* Allocate data space */
-        if((stat = NC4_inq_atomic_type(typeid, NULL, &typelen)))
-	    goto done;
         if(typeid == NC_CHAR)
             data = malloc(typelen*(count+1));
         else
@@ -1079,7 +1099,9 @@ inferattrtype(NCjson* value, nc_type* typeidp)
     case NCJ_NULL:
         typeid = NC_CHAR;
 	return NC_NOERR;
-    case NCJ_DICT: /* fall thru */
+    case NCJ_DICT:
+    	typeid = NC_CHAR;
+	goto done;
     case NCJ_UNDEF:
 	return NC_EINVAL;
     default: /* atomic */
@@ -2289,42 +2311,48 @@ computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarra
     return THROW(stat);
 }
 
-#if 0
-Not currently used
-Special compatibility case:
-       if the value of the attribute is a dictionary,
-       or an array with non-atomic values, then
-       then stringify it and pretend it is of char type.
-/* Return 1 if this json is not an
-atomic value or an array of atomic values.
-That is, it does not look like valid
-attribute data.
+/**
+Implement the JSON convention for dictionaries.
+
+Reading: If the value of the attribute is a dictionary, then stringify
+         it as the value and make the attribute be of type "char".
+
+Writing: if the attribute is of type char and looks like a JSON dictionary,
+	 then parse it as JSON and use that as its value in .zattrs.
 */
+
 static int
-iscomplexjson(NCjson* j)
+read_dict(NCjson** jvaluep)
 {
-    int i;
-    switch(NCJsort(j)) {
-    case NCJ_ARRAY:
-	/* verify that the elements of the array are not complex */
-	for(i=0;i<NCJlength(j);i++) {
-	    switch (NCJith(j,NCJsort(i)))) {
-	    case NCJ_DICT:
-    	    case NCJ_ARRAY:
-	    case NCJ_UNDEF:
-	    case NCJ_NULL:
-		return 1;
-	    default: break;
-	    }
-	}
-	return 0;
-    case NCJ_DICT:
-    case NCJ_UNDEF:
-    case NCJ_NULL:
-	break;
-    default:
-        return 0;
-    }
-    return 1;
+    int stat = NC_NOERR;
+    NCjson* jdict = NULL;
+    NCjson* jtext = NULL;
+    char* text = NULL;
+
+    assert(jvaluep != NULL && *jvaluep != NULL);
+    jdict = *jvaluep;
+    if(NCJsort(jdict) != NCJ_DICT)  {stat = NC_EINVAL; goto done;}
+    if(NCJunparse(jdict,0,&text)) {stat = NC_EINVAL; goto done;}
+    if(NCJnewstring(NCJ_STRING,text,&jtext)) {stat = NC_EINVAL; goto done;}
+    *jvaluep = jtext;
+done:
+    nullfree(text);
+    return stat;
 }
-#endif
+
+static int
+write_dict(size_t len, const void* data, NCjson** jsonp)
+{
+    int stat = NC_NOERR;
+    NCjson* jdict = NULL;
+
+    assert(jsonp != NULL);
+    if(NCJparsen(len,(char*)data,0,&jdict))
+        {stat = NC_EINVAL; goto done;}
+    if(NCJsort(jdict) != NCJ_DICT)
+        {stat = NC_EINVAL; goto done;}
+    *jsonp = jdict;
+done:
+    return stat;
+}
+
diff --git a/nc_test4/tst_broken_files.c b/nc_test4/tst_broken_files.c
index af68b26905..22bf3e8802 100644
--- a/nc_test4/tst_broken_files.c
+++ b/nc_test4/tst_broken_files.c
@@ -28,7 +28,8 @@ main() {
         fclose(fp);
 
         int  ncid;
-        if (nc_open(FILE_NAME, 0, &ncid) != NC_EHDFERR) ERR;
+        int stat=nc_open(FILE_NAME, 0, &ncid);
+        if (stat != NC_EHDFERR && stat != NC_ENOTNC) ERR;
     }
 
     {
diff --git a/nczarr_test/CMakeLists.txt b/nczarr_test/CMakeLists.txt
index 03db4557fb..ef31d15df1 100644
--- a/nczarr_test/CMakeLists.txt
+++ b/nczarr_test/CMakeLists.txt
@@ -73,7 +73,7 @@ IF(ENABLE_TESTS)
   BUILD_BIN_TEST(zmapio ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zmapio PUBLIC ../libnczarr)
   BUILD_BIN_TEST(zhex)
-  BUILD_BIN_TEST(zisjson)
+  BUILD_BIN_TEST(zisjson ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zisjson PUBLIC ../libnczarr)
   BUILD_BIN_TEST(zs3parse ${COMMONSRC})
   TARGET_INCLUDE_DIRECTORIES(zs3parse PUBLIC ../libnczarr)
@@ -108,6 +108,7 @@ IF(ENABLE_TESTS)
     add_sh_test(nczarr_test run_interop)
     add_sh_test(nczarr_test run_misc)
     add_sh_test(nczarr_test run_nczarr_fill)
+    add_sh_test(nczarr_test run_jsonconvention)
 
     BUILD_BIN_TEST(test_quantize ${TSTCOMMONSRC})
     add_sh_test(nczarr_test run_quantize)
diff --git a/nczarr_test/Makefile.am b/nczarr_test/Makefile.am
index a7ca56d36c..b2241152ea 100644
--- a/nczarr_test/Makefile.am
+++ b/nczarr_test/Makefile.am
@@ -61,6 +61,7 @@ TESTS += run_purezarr.sh
 TESTS += run_interop.sh
 TESTS += run_misc.sh
 TESTS += run_nczarr_fill.sh
+TESTS += run_jsonconvention.sh
 
 endif
 
@@ -127,7 +128,7 @@ run_ut_map.sh run_ut_mapapi.sh run_ut_misc.sh run_ut_chunk.sh run_ncgen4.sh \
 run_nccopyz.sh run_fillonlyz.sh run_chunkcases.sh test_nczarr.sh run_perf_chunks1.sh run_s3_cleanup.sh \
 run_purezarr.sh run_interop.sh run_misc.sh \
 run_filter.sh run_specific_filters.sh \
-run_newformat.sh run_nczarr_fill.sh run_quantize.sh
+run_newformat.sh run_nczarr_fill.sh run_quantize.sh run_jsonconvention.sh
 
 EXTRA_DIST += \
 ref_ut_map_create.cdl ref_ut_map_writedata.cdl ref_ut_map_writemeta2.cdl ref_ut_map_writemeta.cdl \
@@ -147,7 +148,7 @@ ref_bzip2.cdl ref_filtered.cdl ref_multi.cdl \
 ref_any.cdl ref_oldformat.cdl ref_oldformat.zip ref_newformatpure.cdl \
 ref_quotes.zip ref_quotes.cdl \
 ref_groups.h5 ref_byte.zarr.zip ref_byte_fill_value_null.zarr.zip \
-ref_groups_regular.cdl ref_byte.cdl ref_byte_fill_value_null.cdl
+ref_groups_regular.cdl ref_byte.cdl ref_byte_fill_value_null.cdl ref_jsonconvention.cdl
 
 # Interoperability files
 EXTRA_DIST += ref_power_901_constants.zip ref_power_901_constants.cdl ref_quotes.zip ref_quotes.cdl 
diff --git a/nczarr_test/ref_jsonconvention.cdl b/nczarr_test/ref_jsonconvention.cdl
new file mode 100644
index 0000000000..c4a52b8104
--- /dev/null
+++ b/nczarr_test/ref_jsonconvention.cdl
@@ -0,0 +1,13 @@
+netcdf tmp_jsonconvention {
+dimensions:
+	d1 = 1 ;
+variables:
+	int v(d1) ;
+		v:varconvention = "{\n\"key1\": [1,2,3], \"key2\": {\"key3\": \"abc\"}}" ;
+
+// global attributes:
+		:grpconvention = "{\"key1\": [1,2,3], \n\"key2\": {\"key3\": \"abc\"}}" ;
+data:
+
+ v = _ ;
+}
diff --git a/nczarr_test/run_jsonconvention.sh b/nczarr_test/run_jsonconvention.sh
new file mode 100755
index 0000000000..7cf786a685
--- /dev/null
+++ b/nczarr_test/run_jsonconvention.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+if test "x$srcdir" = x ; then srcdir=`pwd`; fi 
+. ../test_common.sh
+
+. "$srcdir/test_nczarr.sh"
+
+# This shell script tests support for:
+# read/write using json convention
+
+set -e
+
+testcase() {
+zext=$1
+
+echo "*** Test: write then read using json convention"
+fileargs tmp_jsonconvention "mode=nczarr,$zext"
+deletemap $zext $file
+${NCGEN} -4 -b -o "$fileurl" $srcdir/ref_jsonconvention.cdl
+${NCDUMP} $fileurl > tmp_jsonconvention_${zext}.cdl
+# remove '\n' from ref file before comparing
+rm -f tmp_jsonconvention.cdl
+sed -e 's|\\n||g' < ${srcdir}/ref_jsonconvention.cdl > tmp_jsonconvention.cdl
+diff -b tmp_jsonconvention.cdl tmp_jsonconvention_${zext}.cdl
+}
+
+testcase file
+if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testcase zip; fi
+if test "x$FEATURE_S3TESTS" = xyes ; then testcase s3; fi
+
+exit 0
diff --git a/nczarr_test/zisjson.c b/nczarr_test/zisjson.c
index 186ae45f6c..e13c69f23f 100644
--- a/nczarr_test/zisjson.c
+++ b/nczarr_test/zisjson.c
@@ -7,36 +7,122 @@
    Output 1 or 0.
 */
 
-#include "config.h"
+
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 
 #include "stdlib.h"
 #include "stdio.h"
+#include "string.h"
 
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>
 #endif
 
+#if defined(_WIN32) && !defined(__MINGW32__)
+#include "XGetopt.h"
+#else
+#include <getopt.h>
+#endif
+
 #include "netcdf.h"
 #include "nclist.h"
 #include "ncjson.h"
 
 #define MAXREAD 8192
 
+/* Command line options */
+struct Jsonpptions {
+    int trace;
+} jsonoptions;
+
+static const char*
+sortname(int thesort)
+{
+    switch(thesort) {
+    default: break;
+    case NCJ_INT: return "NCJ_INT";
+    case NCJ_DOUBLE: return "NCJ_DOUBLE";
+    case NCJ_BOOLEAN: return "NCJ_BOOLEAN";
+    case NCJ_STRING: return "NCJ_STRING";
+    case NCJ_DICT: return "NCJ_DICT";
+    case NCJ_ARRAY: return "NCJ_ARRAY";
+    case NCJ_NULL: return "NCJ_NULL";
+    }
+    return "?";
+}
+
+
+static void
+jsontrace(NCjson* json, int depth)
+{
+    int i;
+    if(json == NULL) goto done;
+    printf("[%d] sort=%s",depth,sortname(NCJsort(json)));
+    switch(NCJsort(json)) {
+    case NCJ_INT:
+    case NCJ_DOUBLE:
+    case NCJ_BOOLEAN:
+    case NCJ_STRING:
+	printf(" string=|%s|\n",NCJstring(json));
+	break;
+    case NCJ_NULL:
+	printf("\n");
+	break;
+    case NCJ_ARRAY:
+	printf("\n");
+	for(i=0;i<NCJlength(json);i++)
+	    jsontrace(NCJith(json,i),depth+1);
+	break;
+    case NCJ_DICT:
+	printf("\n");
+	for(i=0;i<NCJlength(json);i+=2) {
+	    jsontrace(NCJith(json,i),depth+1);
+	    jsontrace(NCJith(json,i+1),depth+1);
+	} break;
+    default: break;
+    }
+done:
+    return;
+}
+
 int
 main(int argc, char** argv)
 {
     int stat = NC_NOERR;
     char text[MAXREAD+1];
     NCjson* json = NULL;
-    int i, red;
+    int i, red, c;
     FILE* f = NULL;
 
-    if(argc > 1) {
-	/* use argv[1] as input */
-	f = fopen(argv[1],"r");
-	if(f == NULL) {fprintf(stderr,"No such file: %s\n",argv[1]); exit(1);}
-    } else
-        f = stdin;
+    nc_initialize();
+    memset((void*)&jsonoptions,0,sizeof(jsonoptions));
+
+    while ((c = getopt(argc, argv, "t")) != EOF) {
+	switch(c) {
+	case 't': jsonoptions.trace = 1; break;
+	case '?':
+	   fprintf(stderr,"unknown option\n");
+	   exit(1);
+	}
+    }	
+
+    /* get file argument */
+    argc -= optind;
+    argv += optind;
+
+    if (argc > 1) {
+	fprintf(stderr, "zisjson: only one input file argument permitted\n");
+	exit(1);
+    }
+    if (argc == 0)
+	f = stdin;
+    else {
+        /* use argv[0] as input */
+        f = fopen(argv[0],"r");
+        if(f == NULL) {fprintf(stderr,"No such file: %s\n",argv[1]); exit(1);}
+    }
 
     /* Read json from stdin */
     for(i=0;;i++) {
@@ -54,9 +140,12 @@ main(int argc, char** argv)
 	stat = NC_EEMPTY;
     } else {
         stat = NCJparse(text,0,&json);
-	NCJreclaim(json);
+	if(!stat) {
+	    if(jsonoptions.trace) jsontrace(json,0);
+	    NCJreclaim(json);
+	}
     }
-    printf("%d",(stat==NC_NOERR?1:0)); /* parse success|failure */    
+    printf("%d",(stat?0:1)); /* parse success|failure */    
     if(f != stdin) fclose(f);
     return 0;
 }

From 3e2f32a4bcca19fd755bb3c731a05f99e2b8bcce Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Mon, 9 May 2022 14:23:28 -0600
Subject: [PATCH 15/17] update

---
 libnczarr/zsync.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/libnczarr/zsync.c b/libnczarr/zsync.c
index aa564211e9..032ea1c525 100644
--- a/libnczarr/zsync.c
+++ b/libnczarr/zsync.c
@@ -22,7 +22,7 @@ static int ncz_sync_var(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int isclose);
 
 static int ncz_jsonize_atts(NCindex* attlist, NCjson** jattrsp);
 static int load_jatts(NCZMAP* map, NC_OBJ* container, int nczarrv1, NCjson** jattrsp, NClist** atypes);
-static int zconvert(nc_type typeid, size_t typelen, void* dst, NCjson* src);
+static int zconvert(nc_type typeid, size_t typelen, NCjson* src, void* dst);
 static int computeattrinfo(const char* name, NClist* atypes, NCjson* values,
 		nc_type* typeidp, size_t* typelenp, size_t* lenp, void** datap);
 static int parse_group_content(NCjson* jcontent, NClist* dimdefs, NClist* varnames, NClist* subgrps);
@@ -41,7 +41,7 @@ static int computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, s
 static int inferattrtype(NCjson* values, nc_type* typeidp);
 static int mininttype(unsigned long long u64, int negative);
 static int computedimrefs(NC_FILE_INFO_T* file, NC_VAR_INFO_T* var, int purezarr, int xarray, int ndims, NClist* dimnames, size64_t* shapes, NC_DIM_INFO_T** dims);
-static int read_dict(NCjson** valuep);
+static int read_dict(NCjson* jdict, NCjson** jtextp);
 static int write_dict(size_t len, const void* data, NCjson** jsonp);
 
 /**************************************************/
@@ -934,7 +934,7 @@ load_jatts(NCZMAP* map, NC_OBJ* container, int nczarrv1, NCjson** jattrsp, NClis
 
 /* Convert a json value to actual data values of an attribute. */
 static int
-zconvert(nc_type typeid, size_t typelen, void* dst0, NCjson* src)
+zconvert(nc_type typeid, size_t typelen, NCjson* src, void* dst0)
 {
     int stat = NC_NOERR;
     int i;
@@ -1020,6 +1020,7 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
     void* data = NULL;
     size_t typelen;
     nc_type typeid = NC_NAT;
+    NCjson* jtext = NULL;
     int reclaimvalues = 0;
 
     /* Get assumed type */
@@ -1037,7 +1038,9 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
 	break;
     case NCJ_DICT:
 	/* Apply the JSON dictionary convention and convert to string */
-	if((stat = read_dict(&values))) goto done;
+	if((stat = read_dict(values,&jtext))) goto done;
+	values = jtext; jtext = NULL;
+	reclaimvalues = 1;
 	/* fall thru */
     case NCJ_STRING: /* requires special handling as an array of characters; also look out for empty string */
 	if(typeid == NC_CHAR) {
@@ -1060,7 +1063,7 @@ computeattrdata(nc_type* typeidp, NCjson* values, size_t* typelenp, size_t* lenp
         if(data == NULL)
 	    {stat = NC_ENOMEM; goto done;}
         /* convert to target type */
-        if((stat = zconvert(typeid, typelen, data, values)))
+        if((stat = zconvert(typeid, typelen, values, data)))
    	    goto done;
     }
     if(lenp) *lenp = count;
@@ -2322,20 +2325,19 @@ Writing: if the attribute is of type char and looks like a JSON dictionary,
 */
 
 static int
-read_dict(NCjson** jvaluep)
+read_dict(NCjson* jdict, NCjson** jtextp)
 {
     int stat = NC_NOERR;
-    NCjson* jdict = NULL;
     NCjson* jtext = NULL;
     char* text = NULL;
 
-    assert(jvaluep != NULL && *jvaluep != NULL);
-    jdict = *jvaluep;
+    if(jdict == NULL) {stat = NC_EINVAL; goto done;}
     if(NCJsort(jdict) != NCJ_DICT)  {stat = NC_EINVAL; goto done;}
     if(NCJunparse(jdict,0,&text)) {stat = NC_EINVAL; goto done;}
     if(NCJnewstring(NCJ_STRING,text,&jtext)) {stat = NC_EINVAL; goto done;}
-    *jvaluep = jtext;
+    *jtextp = jtext; jtext = NULL;
 done:
+    NCJreclaim(jtext);
     nullfree(text);
     return stat;
 }
@@ -2351,8 +2353,9 @@ write_dict(size_t len, const void* data, NCjson** jsonp)
         {stat = NC_EINVAL; goto done;}
     if(NCJsort(jdict) != NCJ_DICT)
         {stat = NC_EINVAL; goto done;}
-    *jsonp = jdict;
+    *jsonp = jdict; jdict = NULL;
 done:
+    NCJreclaim(jdict);
     return stat;
 }
 

From 8eb0712eb66fa4e7ccc290e63ede276bb8d63f87 Mon Sep 17 00:00:00 2001
From: Dennis Heimbigner <dmh@ucar.edu>
Date: Mon, 9 May 2022 14:45:06 -0600
Subject: [PATCH 16/17] final1

---
 .github/workflows/run_tests_osx.yml       | 2 +-
 .github/workflows/run_tests_ubuntu.yml    | 2 +-
 .github/workflows/run_tests_win_mingw.yml | 2 +-
 RELEASE_NOTES.md                          | 2 ++
 ncdump/tst_nccopy3.sh                     | 1 -
 ncdump/tst_output.sh                      | 1 -
 6 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/run_tests_osx.yml b/.github/workflows/run_tests_osx.yml
index aa35ae8795..8e9d65a6fb 100644
--- a/.github/workflows/run_tests_osx.yml
+++ b/.github/workflows/run_tests_osx.yml
@@ -7,7 +7,7 @@
 name: Run macOS-based netCDF Tests
 
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_ubuntu.yml b/.github/workflows/run_tests_ubuntu.yml
index 826ac43589..a21ac5b001 100644
--- a/.github/workflows/run_tests_ubuntu.yml
+++ b/.github/workflows/run_tests_ubuntu.yml
@@ -4,7 +4,7 @@
 
 name: Run Ubuntu/Linux netCDF Tests
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/.github/workflows/run_tests_win_mingw.yml b/.github/workflows/run_tests_win_mingw.yml
index aaf999e6a4..913920a3aa 100644
--- a/.github/workflows/run_tests_win_mingw.yml
+++ b/.github/workflows/run_tests_win_mingw.yml
@@ -7,7 +7,7 @@
 name: Run MSYS2, MinGW64-based Tests
 
 
-on: [pull_request,push]
+on: [pull_request]
 
 jobs:
 
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index f876461b14..157b3d2abf 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -7,6 +7,8 @@ This file contains a high-level description of this package's evolution. Release
 
 ## 4.8.2 - TBD
 
+* [Enhancement] Allow the read/write of JSON-valued Zarr attributes to allow
+for domain specific info such as used by GDAL/Zarr. See [Github #????](https://github.com/Unidata/netcdf-c/pull/????).
 * [Enhancement] Update the documentation to match the current filter capabilities  See [Github #2249](https://github.com/Unidata/netcdf-c/pull/2249).
 * [Enhancement] Support installation of pre-built standard filters into user-specified location. See [Github #2318](https://github.com/Unidata/netcdf-c/pull/2318).
 * [Enhancement] Improve filter support. More specifically (1) add nc_inq_filter_avail to check if a filter is available, (2) add the notion of standard filters, (3) cleanup szip support to fix interaction with NCZarr. See [Github #2245](https://github.com/Unidata/netcdf-c/pull/2245).
diff --git a/ncdump/tst_nccopy3.sh b/ncdump/tst_nccopy3.sh
index 7dcfc910c5..30b53bb8b1 100755
--- a/ncdump/tst_nccopy3.sh
+++ b/ncdump/tst_nccopy3.sh
@@ -8,7 +8,6 @@
 if test "x$srcdir" = x ; then srcdir=`pwd`; fi
 . ../test_common.sh
 
-set -x
 set -e
 
 echo ""
diff --git a/ncdump/tst_output.sh b/ncdump/tst_output.sh
index 18da9c3e6f..40c7e7cd47 100755
--- a/ncdump/tst_output.sh
+++ b/ncdump/tst_output.sh
@@ -4,7 +4,6 @@ if test "x$srcdir" = x ; then srcdir=`pwd`; fi
 . ../test_common.sh
 
 # This shell script tests the output from several previous tests.
-set -x
 set -e
 
 echo ""

From e5f1d0846bcad84e6aa46939bb7c0f938fbffad5 Mon Sep 17 00:00:00 2001
From: Ward Fisher <wfisher@ucar.edu>
Date: Thu, 12 May 2022 15:07:42 -0600
Subject: [PATCH 17/17] correct links for gliderscope and netcdfninja in
 software.html

---
 docs/static-pages/software.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/static-pages/software.html b/docs/static-pages/software.html
index 13f98a7533..e19b57d9a9 100644
--- a/docs/static-pages/software.html
+++ b/docs/static-pages/software.html
@@ -940,7 +940,7 @@ <h2><a id="gliderscope" name="gliderscope">Gliderccope</a></h2>
     Dr L. Mun Woo <mun.woo@uwa.edu.au>
     at ANFOG (Australian National Facility for Ocean Gliders)
     has developed
-    <a href="http://imos.org.au/facilities/oceangliders/glider-data/gliderscope/">Gliderscope</a>.
+    <a href="https://imos.org.au/gliderscope">Gliderscope</a>.
     Gliderscope is an IMOS (Integrated Marine Observing System)
     oceanographic software package allowing users quick easy visualisation
     of ocean glider data, via a convenient graphical user interface.
@@ -2008,7 +2008,7 @@ <h2><a id="netcdf_ninja" name="netcdf_ninja">NetCDF Ninja</a></h2>
     <p>
     Dr L. Mun Woo of University of Western Australia <mun.woo@uwa.edu.au>
     has developed
-    <a href="http://imos.org.au/facilities/oceangliders/glider-data/netcdfninja/">NetCDF Ninja</a>,
+    <a href="https://imos.org.au/netcdfninja">NetCDF Ninja</a>,
     a graphical user interface that allows users to browse all
     the metadata contained in netCDF files, scrutinise the data using an
     interactive graphical plot and even make small alterations or export