Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NetCDF: Not a valid data type or _FillValue type mismatch #929

Closed
rabernat opened this issue May 22, 2019 · 9 comments
Closed

NetCDF: Not a valid data type or _FillValue type mismatch #929

rabernat opened this issue May 22, 2019 · 9 comments

Comments

@rabernat
Copy link

I am having trouble opening a dataset from a thredds opendap server. I am raising the issue here, well aware that this issue may belong elsewhere. But this has been a useful place to start.

The data is from the OOI gliders DAC: https://gliders.ioos.us/data/

Here is the opendap file:
https://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc.html

import netCDF4
url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
netCDF4.Dataset(url)

gives

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-10-e606fa0a661d> in <module>
      1 import netCDF4
      2 url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
----> 3 netCDF4.Dataset(url)

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.__init__()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()

OSError: [Errno -45] NetCDF: Not a valid data type or _FillValue type mismatch: b'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'

Is this a problem with the dataset? If so, the entirety of the OOI glider data is incompatible with the netCDF C library.

netcdf4 lib version: '4.6.2 of Dec 17 2018 19:36:03 $'
netcdf4-python version: '1.5.0.1'

cc @kwilcox, @rsignell-usgs

$ ncdump -h http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc
netcdf sp062-20190201T1350.nc3 {
dimensions:
	maxStrlen64 = 64 ;
	obs = 211 ;
	profile = 1682 ;
	trajectory = 1 ;
variables:
	char trajectory(trajectory, maxStrlen64) ;
		trajectory:_Encoding = "ISO-8859-1" ;
		trajectory:cf_role = "trajectory_id" ;
		trajectory:comment = "A trajectory is one deployment of a glider." ;
		trajectory:ioos_category = "Identifier" ;
		trajectory:long_name = "Trajectory Name" ;
	char wmo_id(trajectory, maxStrlen64) ;
		wmo_id:_Encoding = "ISO-8859-1" ;
		wmo_id:ioos_category = "Identifier" ;
		wmo_id:long_name = "WMO ID" ;
	int profile_id(trajectory, profile) ;
		profile_id:_FillValue = -999 ;
		profile_id:actual_range = 2, 1692 ;
		profile_id:cf_role = "profile_id" ;
		profile_id:comment = "Sequential profile number within the trajectory.  This value is unique in each file that is part of a single trajectory/deployment." ;
		profile_id:ioos_category = "Identifier" ;
		profile_id:long_name = "Profile ID" ;
		profile_id:valid_max = 2147483647 ;
		profile_id:valid_min = 1 ;
	double time(trajectory, profile) ;
		time:_CoordinateAxisType = "Time" ;
		time:actual_range = 1549033965., 1558503600. ;
		time:ancillary_variables = "profile_time_qc" ;
		time:axis = "T" ;
		time:comment = "Timestamp corresponding to the mid-point of the profile." ;
		time:ioos_category = "Time" ;
		time:long_name = "Profile Time" ;
		time:observation_type = "calculated" ;
		time:platform = "platform" ;
		time:standard_name = "time" ;
		time:time_origin = "01-JAN-1970 00:00:00" ;
		time:units = "seconds since 1970-01-01T00:00:00Z" ;
	double latitude(trajectory, profile) ;
		latitude:_CoordinateAxisType = "Lat" ;
		latitude:_FillValue = -999. ;
		latitude:actual_range = 35.59735, 37.25454 ;
		latitude:ancillary_variables = "profile_lat_qc" ;
		latitude:axis = "Y" ;
		latitude:colorBarMaximum = 90. ;
		latitude:colorBarMinimum = -90. ;
		latitude:comment = "Value is interpolated to provide an estimate of the latitude at the mid-point of the profile." ;
		latitude:ioos_category = "Location" ;
		latitude:long_name = "Profile Latitude" ;
		latitude:observation_type = "calculated" ;
		latitude:platform = "platform" ;
		latitude:standard_name = "latitude" ;
		latitude:units = "degrees_north" ;
		latitude:valid_max = 90. ;
		latitude:valid_min = -90. ;
	double longitude(trajectory, profile) ;
		longitude:_CoordinateAxisType = "Lon" ;
		longitude:_FillValue = -999. ;
		longitude:actual_range = -74.86983, -74.53165 ;
		longitude:ancillary_variables = "profile_lon_qc" ;
		longitude:axis = "X" ;
		longitude:colorBarMaximum = 180. ;
		longitude:colorBarMinimum = -180. ;
		longitude:comment = "Value is interpolated to provide an estimate of the longitude at the mid-point of the profile." ;
		longitude:ioos_category = "Location" ;
		longitude:long_name = "Profile Longitude" ;
		longitude:observation_type = "calculated" ;
		longitude:platform = "platform" ;
		longitude:standard_name = "longitude" ;
		longitude:units = "degrees_east" ;
		longitude:valid_max = 180. ;
		longitude:valid_min = -180. ;
	double time_uv(trajectory, profile) ;
		time_uv:actual_range = 1549033350., 1558501320. ;
		time_uv:ancillary_variables = "time_uv_qc" ;
		time_uv:calendar = "gregorian" ;
		time_uv:comment = "The depth-averaged current is an estimate of the net current measured while the glider is underwater.  The value is calculated over the entire underwater segment, which may consist of 1 or more dives." ;
		time_uv:ioos_category = "Time" ;
		time_uv:long_name = "Depth-averaged Time" ;
		time_uv:observation_type = "calculated" ;
		time_uv:standard_name = "time" ;
		time_uv:time_origin = "01-JAN-1970 00:00:00" ;
		time_uv:units = "seconds since 1970-01-01T00:00:00Z" ;
	double lat_uv(trajectory, profile) ;
		lat_uv:_FillValue = -999. ;
		lat_uv:actual_range = 35.598, 37.25459 ;
		lat_uv:ancillary_variables = "lat_uv_qc" ;
		lat_uv:colorBarMaximum = 90. ;
		lat_uv:colorBarMinimum = -90. ;
		lat_uv:comment = "The depth-averaged current is an estimate of the net current measured while the glider is underwater.  The value is calculated over the entire underwater segment, which may consist of 1 or more dives." ;
		lat_uv:ioos_category = "Location" ;
		lat_uv:long_name = "Depth-averaged Latitude" ;
		lat_uv:observation_type = "calculated" ;
		lat_uv:platform = "platform" ;
		lat_uv:standard_name = "latitude" ;
		lat_uv:units = "degrees_north" ;
		lat_uv:valid_max = 90. ;
		lat_uv:valid_min = -90. ;
	double lon_uv(trajectory, profile) ;
		lon_uv:_FillValue = -999. ;
		lon_uv:actual_range = -74.86999, -74.53115 ;
		lon_uv:ancillary_variables = "lon_uv_qc" ;
		lon_uv:colorBarMaximum = 180. ;
		lon_uv:colorBarMinimum = -180. ;
		lon_uv:comment = "The depth-averaged current is an estimate of the net current measured while the glider is underwater.  The value is calculated over the entire underwater segment, which may consist of 1 or more dives." ;
		lon_uv:ioos_category = "Location" ;
		lon_uv:long_name = "Depth-averaged Longitude" ;
		lon_uv:observation_type = "calculated" ;
		lon_uv:platform = "platform" ;
		lon_uv:standard_name = "longitude" ;
		lon_uv:units = "degrees_east" ;
		lon_uv:valid_max = 180. ;
		lon_uv:valid_min = -180. ;
	double u(trajectory, profile) ;
		u:_FillValue = -999. ;
		u:actual_range = -0.29431844, 0.32835614 ;
		u:ancillary_variables = "u_qc" ;
		u:colorBarMaximum = 0.5 ;
		u:colorBarMinimum = -0.5 ;
		u:comment = "The depth-averaged current is an estimate of the net current measured while the glider is underwater.  The value is calculated over the entire underwater segment, which may consist of 1 or more dives." ;
		u:coordinates = "lon_uv lat_uv time_uv" ;
		u:ioos_category = "Currents" ;
		u:long_name = "Depth-averaged Eastward Sea Water Velocity" ;
		u:observation_type = "calculated" ;
		u:platform = "platform" ;
		u:standard_name = "eastward_sea_water_velocity" ;
		u:units = "m s-1" ;
		u:valid_max = 10. ;
		u:valid_min = -10. ;
	double v(trajectory, profile) ;
		v:_FillValue = -999. ;
		v:actual_range = -0.507894629, 0.512060978 ;
		v:ancillary_variables = "v_qc" ;
		v:colorBarMaximum = 0.5 ;
		v:colorBarMinimum = -0.5 ;
		v:comment = "The depth-averaged current is an estimate of the net current measured while the glider is underwater.  The value is calculated over the entire underwater segment, which may consist of 1 or more dives." ;
		v:coordinates = "lon_uv lat_uv time_uv" ;
		v:ioos_category = "Currents" ;
		v:long_name = "Depth-averaged Northward Sea Water Velocity" ;
		v:observation_type = "calculated" ;
		v:platform = "platform" ;
		v:standard_name = "northward_sea_water_velocity" ;
		v:units = "m s-1" ;
		v:valid_max = 10. ;
		v:valid_min = -10. ;
	double precise_time(trajectory, profile, obs) ;
		precise_time:_FillValue = NaN ;
		precise_time:actual_range = 1549032120., 1558505880. ;
		precise_time:ancillary_variables = "time_qc" ;
		precise_time:calendar = "gregorian" ;
		precise_time:comment = "Timestamp at each point in the time-series" ;
		precise_time:coordinates = "time latitude longitude depth" ;
		precise_time:ioos_category = "Time" ;
		precise_time:long_name = "Precise Time" ;
		precise_time:observation_type = "measured" ;
		precise_time:standard_name = "time" ;
		precise_time:time_origin = "01-JAN-1970 00:00:00" ;
		precise_time:units = "seconds since 1970-01-01T00:00:00Z" ;
	float depth(trajectory, profile, obs) ;
		depth:_CoordinateAxisType = "Height" ;
		depth:_CoordinateZisPositive = "down" ;
		depth:_FillValue = -999.f ;
		depth:actual_range = -0.357344f, 995.6315f ;
		depth:ancillary_variables = "depth_qc" ;
		depth:axis = "Z" ;
		depth:colorBarMaximum = 2000. ;
		depth:colorBarMinimum = 0. ;
		depth:colorBarPalette = "OceanDepth" ;
		depth:instrument = "instrument_ctd" ;
		depth:ioos_category = "Location" ;
		depth:long_name = "Depth" ;
		depth:observation_type = "calculated" ;
		depth:platform = "platform" ;
		depth:positive = "down" ;
		depth:reference_datum = "sea-surface" ;
		depth:standard_name = "depth" ;
		depth:units = "m" ;
		depth:valid_max = 2000.f ;
		depth:valid_min = 0.f ;
	float pressure(trajectory, profile, obs) ;
		pressure:_FillValue = -999.f ;
		pressure:actual_range = -0.36f, 1005.36f ;
		pressure:ancillary_variables = "pressure_qc" ;
		pressure:colorBarMaximum = 2000. ;
		pressure:colorBarMinimum = 0. ;
		pressure:coordinates = "time latitude longitude depth" ;
		pressure:instrument = "instrument_ctd" ;
		pressure:ioos_category = "Pressure" ;
		pressure:long_name = "Sea Water Pressure" ;
		pressure:observation_type = "measured" ;
		pressure:platform = "platform" ;
		pressure:positive = "down" ;
		pressure:reference_datum = "sea-surface" ;
		pressure:standard_name = "sea_water_pressure" ;
		pressure:units = "dbar" ;
		pressure:valid_max = 2000.f ;
		pressure:valid_min = 0.f ;
	float temperature(trajectory, profile, obs) ;
		temperature:_FillValue = -999.f ;
		temperature:actual_range = 4.344f, 25.222f ;
		temperature:ancillary_variables = "temperature_qc" ;
		temperature:colorBarMaximum = 32. ;
		temperature:colorBarMinimum = 0. ;
		temperature:coordinates = "time latitude longitude depth" ;
		temperature:instrument = "instrument_ctd" ;
		temperature:ioos_category = "Temperature" ;
		temperature:long_name = "Sea Water Temperature" ;
		temperature:observation_type = "measured" ;
		temperature:platform = "platform" ;
		temperature:standard_name = "sea_water_temperature" ;
		temperature:units = "Celsius" ;
		temperature:valid_max = 40.f ;
		temperature:valid_min = -5.f ;
	float conductivity(trajectory, profile, obs) ;
		conductivity:_FillValue = -999.f ;
		conductivity:actual_range = 2.242157f, 5.519074f ;
		conductivity:ancillary_variables = "conductivity_qc" ;
		conductivity:colorBarMaximum = 9. ;
		conductivity:colorBarMinimum = 0. ;
		conductivity:coordinates = "time latitude longitude depth" ;
		conductivity:instrument = "instrument_ctd" ;
		conductivity:ioos_category = "Salinity" ;
		conductivity:long_name = "Sea Water Electrical Conductivity" ;
		conductivity:observation_type = "measured" ;
		conductivity:platform = "platform" ;
		conductivity:standard_name = "sea_water_electrical_conductivity" ;
		conductivity:units = "S m-1" ;
		conductivity:valid_max = 10.f ;
		conductivity:valid_min = 0.f ;
	float salinity(trajectory, profile, obs) ;
		salinity:_FillValue = -999.f ;
		salinity:actual_range = 19.598f, 36.632f ;
		salinity:ancillary_variables = "salinity_qc" ;
		salinity:colorBarMaximum = 37. ;
		salinity:colorBarMinimum = 30. ;
		salinity:coordinates = "time latitude longitude depth" ;
		salinity:instrument = "instrument_ctd" ;
		salinity:ioos_category = "Salinity" ;
		salinity:long_name = "Sea Water Practical Salinity" ;
		salinity:observation_type = "calculated" ;
		salinity:platform = "platform" ;
		salinity:standard_name = "sea_water_practical_salinity" ;
		salinity:units = "1" ;
		salinity:valid_max = 40.f ;
		salinity:valid_min = 0.f ;
	float density(trajectory, profile, obs) ;
		density:_FillValue = -999.f ;
		density:ancillary_variables = "density_qc" ;
		density:colorBarMaximum = 1032. ;
		density:colorBarMinimum = 1020. ;
		density:coordinates = "time latitude longitude depth" ;
		density:instrument = "instrument_ctd" ;
		density:ioos_category = "Other" ;
		density:long_name = "Sea Water Density" ;
		density:observation_type = "calculated" ;
		density:platform = "platform" ;
		density:standard_name = "sea_water_density" ;
		density:units = "kg m-3" ;
		density:valid_max = 1040.f ;
		density:valid_min = 1015.f ;
	double precise_lat(trajectory, profile, obs) ;
		precise_lat:_FillValue = -999. ;
		precise_lat:actual_range = 35.5967, 37.2547 ;
		precise_lat:ancillary_variables = "lat_qc" ;
		precise_lat:ancillary_varibles = "precise_lat_qc" ;
		precise_lat:colorBarMaximum = 90. ;
		precise_lat:colorBarMinimum = -90. ;
		precise_lat:comment = "Interpolated latitude at each point in the time-series" ;
		precise_lat:coordinate_reference_frame = "urn:ogc:crs:EPSG::4326" ;
		precise_lat:coordinates = "time latitude longitude depth" ;
		precise_lat:ioos_category = "Location" ;
		precise_lat:long_name = "Precise Latitude" ;
		precise_lat:observation_type = "measured" ;
		precise_lat:platform = "platform" ;
		precise_lat:reference = "WGS84" ;
		precise_lat:standard_name = "latitude" ;
		precise_lat:units = "degrees_north" ;
		precise_lat:valid_max = 90. ;
		precise_lat:valid_min = -90. ;
	double precise_lon(trajectory, profile, obs) ;
		precise_lon:_FillValue = -999. ;
		precise_lon:actual_range = -74.8703, -74.5298 ;
		precise_lon:ancillary_variables = "lon_qc" ;
		precise_lon:ancillary_varibles = "precise_lon_qc" ;
		precise_lon:colorBarMaximum = 180. ;
		precise_lon:colorBarMinimum = -180. ;
		precise_lon:comment = "Interpolated longitude at each point in the time-series" ;
		precise_lon:coordinate_reference_frame = "urn:ogc:crs:EPSG::4326" ;
		precise_lon:coordinates = "time latitude longitude depth" ;
		precise_lon:ioos_category = "Location" ;
		precise_lon:long_name = "Precise Longitude" ;
		precise_lon:observation_type = "measured" ;
		precise_lon:platform = "platform" ;
		precise_lon:reference = "WGS84" ;
		precise_lon:standard_name = "longitude" ;
		precise_lon:units = "degrees_east" ;
		precise_lon:valid_max = 180. ;
		precise_lon:valid_min = -180. ;
	byte platform_meta(trajectory, profile, obs) ;
		platform_meta:_FillValue = 127b ;
		platform_meta:comment = "Spray Glider sp062" ;
		platform_meta:coordinates = "time latitude longitude depth" ;
		platform_meta:id = "sp062" ;
		platform_meta:instrument = "instrument_ctd" ;
		platform_meta:ioos_category = "Identifier" ;
		platform_meta:long_name = "Platform Metadata" ;
		platform_meta:type = "platform" ;
		platform_meta:units = "1" ;
		platform_meta:wmo_id = "4801983" ;
	byte instrument_ctd(trajectory, profile, obs) ;
		instrument_ctd:_FillValue = 127b ;
		instrument_ctd:coordinates = "time latitude longitude depth" ;
		instrument_ctd:ioos_category = "Identifier" ;
		instrument_ctd:long_name = "CTD Metadata" ;
		instrument_ctd:make_model = "Sea-Bird 41CP" ;
		instrument_ctd:platform = "platform" ;
		instrument_ctd:type = "platform" ;
		instrument_ctd:units = "1" ;
	byte precise_lon_qc(trajectory, profile, obs) ;
		precise_lon_qc:_FillValue = -127b ;
		precise_lon_qc:coordinates = "time latitude longitude depth" ;
		precise_lon_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed interpolated_value missing_value" ;
		precise_lon_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		precise_lon_qc:ioos_category = "Quality" ;
		precise_lon_qc:long_name = "precise_lon Variable Quality Flag" ;
		precise_lon_qc:valid_max = 9b ;
		precise_lon_qc:valid_min = 0b ;
	byte salinity_qc(trajectory, profile, obs) ;
		salinity_qc:_FillValue = -127b ;
		salinity_qc:actual_range = 1b, 9b ;
		salinity_qc:coordinates = "time latitude longitude depth" ;
		salinity_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		salinity_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		salinity_qc:ioos_category = "Quality" ;
		salinity_qc:long_name = "salinity Variable Quality Flag" ;
		salinity_qc:standard_name = "sea_water_salinity status_flag" ;
		salinity_qc:valid_max = 9b ;
		salinity_qc:valid_min = 0b ;
	byte density_qc(trajectory, profile, obs) ;
		density_qc:_FillValue = -127b ;
		density_qc:actual_range = 9b, 9b ;
		density_qc:coordinates = "time latitude longitude depth" ;
		density_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		density_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		density_qc:ioos_category = "Quality" ;
		density_qc:long_name = "density Variable Quality Flag" ;
		density_qc:standard_name = "sea_water_density status_flag" ;
		density_qc:valid_max = 9b ;
		density_qc:valid_min = 0b ;
	byte temperature_qc(trajectory, profile, obs) ;
		temperature_qc:_FillValue = -127b ;
		temperature_qc:actual_range = 1b, 9b ;
		temperature_qc:coordinates = "time latitude longitude depth" ;
		temperature_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		temperature_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		temperature_qc:ioos_category = "Quality" ;
		temperature_qc:long_name = "temperature Variable Quality Flag" ;
		temperature_qc:standard_name = "sea_water_temperature status_flag" ;
		temperature_qc:valid_max = 9b ;
		temperature_qc:valid_min = 0b ;
	byte lat_uv_qc(trajectory, profile, obs) ;
		lat_uv_qc:_FillValue = -127b ;
		lat_uv_qc:actual_range = 1b, 4b ;
		lat_uv_qc:coordinates = "time latitude longitude depth" ;
		lat_uv_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		lat_uv_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		lat_uv_qc:ioos_category = "Quality" ;
		lat_uv_qc:long_name = "lat_uv Variable Quality Flag" ;
		lat_uv_qc:standard_name = "time status_flag" ;
		lat_uv_qc:valid_max = 9b ;
		lat_uv_qc:valid_min = 0b ;
	byte precise_time_qc(trajectory, profile, obs) ;
		precise_time_qc:_FillValue = -127b ;
		precise_time_qc:actual_range = 1b, 8b ;
		precise_time_qc:coordinates = "time latitude longitude depth" ;
		precise_time_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		precise_time_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		precise_time_qc:ioos_category = "Quality" ;
		precise_time_qc:long_name = "precise_time Variable Quality Flag" ;
		precise_time_qc:standard_name = "time status_flag" ;
		precise_time_qc:valid_max = 9b ;
		precise_time_qc:valid_min = 0b ;
	byte longitude_qc(trajectory, profile, obs) ;
		longitude_qc:_FillValue = -127b ;
		longitude_qc:actual_range = 1b, 4b ;
		longitude_qc:coordinates = "time latitude longitude depth" ;
		longitude_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		longitude_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		longitude_qc:ioos_category = "Quality" ;
		longitude_qc:long_name = "longitude Variable Quality Flag" ;
		longitude_qc:standard_name = "longitude status_flag" ;
		longitude_qc:valid_max = 9b ;
		longitude_qc:valid_min = 0b ;
	byte pressure_qc(trajectory, profile, obs) ;
		pressure_qc:_FillValue = -127b ;
		pressure_qc:actual_range = 1b, 1b ;
		pressure_qc:coordinates = "time latitude longitude depth" ;
		pressure_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		pressure_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		pressure_qc:ioos_category = "Quality" ;
		pressure_qc:long_name = "pressure Variable Quality Flag" ;
		pressure_qc:standard_name = "sea_water_pressure status_flag" ;
		pressure_qc:valid_max = 9b ;
		pressure_qc:valid_min = 0b ;
	byte time_uv_qc(trajectory, profile, obs) ;
		time_uv_qc:_FillValue = -127b ;
		time_uv_qc:actual_range = 1b, 4b ;
		time_uv_qc:coordinates = "time latitude longitude depth" ;
		time_uv_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		time_uv_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		time_uv_qc:ioos_category = "Quality" ;
		time_uv_qc:long_name = "time_uv Variable Quality Flag" ;
		time_uv_qc:standard_name = "time status_flag" ;
		time_uv_qc:valid_max = 9b ;
		time_uv_qc:valid_min = 0b ;
	byte latitude_qc(trajectory, profile, obs) ;
		latitude_qc:_FillValue = -127b ;
		latitude_qc:actual_range = 1b, 4b ;
		latitude_qc:coordinates = "time latitude longitude depth" ;
		latitude_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		latitude_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		latitude_qc:ioos_category = "Quality" ;
		latitude_qc:long_name = "latitude Variable Quality Flag" ;
		latitude_qc:standard_name = "latitude status_flag" ;
		latitude_qc:valid_max = 9b ;
		latitude_qc:valid_min = 0b ;
	byte u_qc(trajectory, profile, obs) ;
		u_qc:_FillValue = -127b ;
		u_qc:actual_range = 1b, 4b ;
		u_qc:coordinates = "time latitude longitude depth" ;
		u_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		u_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		u_qc:ioos_category = "Quality" ;
		u_qc:long_name = "u Variable Quality Flag" ;
		u_qc:standard_name = "eastward_sea_water_velocity status_flag" ;
		u_qc:valid_max = 9b ;
		u_qc:valid_min = 0b ;
	byte v_qc(trajectory, profile, obs) ;
		v_qc:_FillValue = -127b ;
		v_qc:actual_range = 1b, 4b ;
		v_qc:coordinates = "time latitude longitude depth" ;
		v_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		v_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		v_qc:ioos_category = "Quality" ;
		v_qc:long_name = "v Variable Quality Flag" ;
		v_qc:standard_name = "northward_sea_water_velocity status_flag" ;
		v_qc:valid_max = 9b ;
		v_qc:valid_min = 0b ;
	byte depth_qc(trajectory, profile, obs) ;
		depth_qc:_FillValue = -127b ;
		depth_qc:actual_range = 1b, 1b ;
		depth_qc:coordinates = "time latitude longitude depth" ;
		depth_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		depth_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		depth_qc:ioos_category = "Quality" ;
		depth_qc:long_name = "depth Variable Quality Flag" ;
		depth_qc:standard_name = "depth status_flag" ;
		depth_qc:valid_max = 9b ;
		depth_qc:valid_min = 0b ;
	byte time_qc(trajectory, profile, obs) ;
		time_qc:_FillValue = -127b ;
		time_qc:actual_range = 1b, 4b ;
		time_qc:coordinates = "time latitude longitude depth" ;
		time_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		time_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		time_qc:ioos_category = "Quality" ;
		time_qc:long_name = "time Variable Quality Flag" ;
		time_qc:standard_name = "time status_flag" ;
		time_qc:valid_max = 9b ;
		time_qc:valid_min = 0b ;
	byte lon_uv_qc(trajectory, profile, obs) ;
		lon_uv_qc:_FillValue = -127b ;
		lon_uv_qc:actual_range = 1b, 4b ;
		lon_uv_qc:coordinates = "time latitude longitude depth" ;
		lon_uv_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		lon_uv_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		lon_uv_qc:ioos_category = "Quality" ;
		lon_uv_qc:long_name = "lon_uv Variable Quality Flag" ;
		lon_uv_qc:standard_name = "time status_flag" ;
		lon_uv_qc:valid_max = 9b ;
		lon_uv_qc:valid_min = 0b ;
	byte precise_lat_qc(trajectory, profile, obs) ;
		precise_lat_qc:_FillValue = -127b ;
		precise_lat_qc:coordinates = "time latitude longitude depth" ;
		precise_lat_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed interpolated_value missing_value" ;
		precise_lat_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		precise_lat_qc:ioos_category = "Quality" ;
		precise_lat_qc:long_name = "precise_lat Variable Quality Flag" ;
		precise_lat_qc:valid_max = 9b ;
		precise_lat_qc:valid_min = 0b ;
	byte conductivity_qc(trajectory, profile, obs) ;
		conductivity_qc:_FillValue = -127b ;
		conductivity_qc:actual_range = 1b, 9b ;
		conductivity_qc:coordinates = "time latitude longitude depth" ;
		conductivity_qc:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_value missing_value" ;
		conductivity_qc:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ;
		conductivity_qc:ioos_category = "Quality" ;
		conductivity_qc:long_name = "conductivity Variable Quality Flag" ;
		conductivity_qc:standard_name = "sea_water_electrical_conductivity status_flag" ;
		conductivity_qc:valid_max = 9b ;
		conductivity_qc:valid_min = 0b ;

// global attributes:
		:acknowledgment = "This deployment supported by NOAA." ;
		:cdm_data_type = "TrajectoryProfile" ;
		:cdm_profile_variables = "time_uv,lat_uv,lon_uv,u,v,profile_id,time,latitude,longitude" ;
		:cdm_trajectory_variables = "trajectory,wmo_id" ;
		:contributor_name = "Bob Simons (bob.simons@noaa.gov)" ;
		:contributor_role = "additional data management" ;
		:Conventions = "Unidata Dataset Discovery v1.0, COARDS, CF-1.6" ;
		:creator_email = "rtodd@whoi.edu" ;
		:creator_name = "Woods Hole Oceanographic Institution" ;
		:creator_url = "http://gliders.whoi.edu" ;
		:date_created = "2019-05-22T04:25:10Z" ;
		:date_issued = "2019-05-22T04:25:10Z" ;
		:date_modified = "2019-05-22T11:41:31Z" ;
		:Easternmost_Easting = -74.53165 ;
		:featureType = "TrajectoryProfile" ;
		:format_version = "IOOS_Glider_NetCDF_v3.0.nc" ;
		:geospatial_lat_max = 37.25454 ;
		:geospatial_lat_min = 35.59735 ;
		:geospatial_lat_units = "degrees_north" ;
		:geospatial_lon_max = -74.53165 ;
		:geospatial_lon_min = -74.86983 ;
		:geospatial_lon_units = "degrees_east" ;
		:geospatial_vertical_max = 995.6315f ;
		:geospatial_vertical_min = -0.357344f ;
		:geospatial_vertical_positive = "down" ;
		:geospatial_vertical_units = "m" ;
		:history = "readsat 20190522T042432 - fixgps3 20190522T042432 - calcvelsat 20190522T042432 - autoqcctd 20190522T042432 - adpsat 20190522T042500 - \n",
			"2019-05-22T11:41:31Z bob.simons@noaa.gov of NOAA NMFS SWFSC ERD used DailyScrippsGliders3.java to convert the source file to format_version=IOOS_Glider_NetCDF_v3.0.nc\n",
			"2019-05-22T16:16:27Z (local files)\n",
			"2019-05-22T16:16:27Z https://gliders.ioos.us/erddap/tabledap/sp062-20190201T1350.ncCFMA" ;
		:id = "sp062-20190201T1350_f070_8f49_1646" ;
		:infoUrl = "http://data.ioos.us/gliders/erddap/" ;
		:institution = "Woods Hole Oceanographic Institution" ;
		:ioos_dac_checksum = "0789085943c4d335aba0551fafded495" ;
		:ioos_dac_completed = "False" ;
		:keywords = "AUVS > Autonomous Underwater Vehicles, Oceans > Ocean Pressure > Water Pressure, Oceans > Ocean Temperature > Water Temperature, Oceans > Salinity/Density > Conductivity, Oceans > Salinity/Density > Density, Oceans > Salinity/Density > Salinity, glider, In Situ Ocean-based platforms > Seaglider, Spray, Slocum, trajectory, underwater glider, water, wmo" ;
		:keywords_vocabulary = "GCMD Science Keywords" ;
		:license = "The data may be used and redistributed for free but is not intended for legal use, since it may contain inaccuracies. No person or group associated with this data makes any warranty, express or implied, including warranties of merchantability and fitness for a particular purpose, or assumes any legal liability for the accuracy, completeness, or usefulness, of this information." ;
		:Metadata_Conventions = "Unidata Dataset Discovery v1.0, COARDS, CF-1.6" ;
		:naming_authority = "edu.ucsd.spray" ;
		:Northernmost_Northing = 37.25454 ;
		:platform_institution = "WHOI" ;
		:platform_type = "Spray Glider" ;
		:processing_level = "Automatic QC has been done to this real time data." ;
		:project = "Woods Hole Oceanographic Institution" ;
		:publisher_email = "drudnick@ucsd.edu" ;
		:publisher_name = "Scripps Institution of Oceanography Instrument Development Group" ;
		:publisher_url = "http:/spray.ucsd.edu" ;
		:references = "Sherman, J., R.E. Davis, W.B. Owens and J. Valdes, 2001. The autonomous underwater glider \'Spray.\' IEEE Oceanic Eng., 26, 437-446.\n",
			"Rudnick, D. L., R. E. Davis, C. C. Eriksen, D. M. Fratantoni, and M. J. Perry, 2004. Underwater gliders for ocean research. Mar. Tech. Soc. J., 38, 73-84.\n",
			"https://ioos.noaa.gov/wp-content/uploads/2015/10/Manual-for-QC-of-Glider-Data_05_09_16.pdf" ;
		:sea_name = "East Coast - US/Canada" ;
		:source = "Data from an underwater glider" ;
		:sourceUrl = "(local files)" ;
		:Southernmost_Northing = 35.59735 ;
		:standard_name_vocabulary = "CF Standard Name Table v27" ;
		:subsetVariables = "trajectory,wmo_id,time_uv,lat_uv,lon_uv,u,v,profile_id,time,latitude,longitude" ;
		:summary = "Spray glider profile data from Woods Hole Oceanographic Institution. (This deployment supported by NOAA.)" ;
		:time_coverage_end = "2019-05-22T05:40:00Z" ;
		:time_coverage_start = "2019-02-01T15:12:45Z" ;
		:title = "sp062-20190201T1350" ;
		:Westernmost_Easting = -74.86983 ;
		:DODS.strlen = 7 ;
		:DODS.dimName = "wmo_id_strlen" ;
}
@jswhit
Copy link
Collaborator

jswhit commented May 26, 2019

ncdump gives me the same error (with version 4.6.3), so it's definitely not an issue with netcdf4-python.

@jswhit
Copy link
Collaborator

jswhit commented May 27, 2019

Seems to be an issue with the opendap server configuration - if I download the netcdf file to disk I can open it and read the data just fine.

@jswhit
Copy link
Collaborator

jswhit commented May 30, 2019

Closing this - if it does turn out to be a netcdf4-python issue feel free to reopen.

@jswhit jswhit closed this as completed May 30, 2019
@kwilcox
Copy link

kwilcox commented May 30, 2019

@lesserwhirls I’ve seen this before but can’t recall the issue or fix. I believe it was a THREDDS thing. Anything seem familiar?

@lesserwhirls
Copy link
Collaborator

If I recall, the issue was that netCDF-C started being strict about _FillValue type matching variable type starting in version 4.6.2. This is fine, except for the case of reading some kinds of data through the TDS OPeNDAP endpoint for types not supported by the DAP2 data model. This should be fixed in version netCDF-C 4.6.3 and above. See Unidata/netcdf-c#1316

@kwilcox
Copy link

kwilcox commented May 30, 2019

Thanks! @rabernat can you try with an updated libnetcdf?

@rabernat
Copy link
Author

rabernat commented May 30, 2019 •
edited
Loading

Unfortunately trying to update libnetcdf gets me into dependency probems with gdal / rasterio

$ conda install -c conda-forge libnetcdf=4.7.0
Fetching package metadata ...........
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - libnetcdf 4.7.0*
  - rasterio -> gdal 1.11.2 -> hdf5 1.8.14
  - rasterio -> gdal 1.11.2 -> libnetcdf 4.3.2 -> curl 7.38.0 -> openssl 1.0.1*
Use "conda info <package>" to see the dependencies for each package.

So it looks like I can't upgrate netcdf without breaking rasterio / gdal. Unfortunately I will not have time to try to resolve this soon.

@croachutas croachutas mentioned this issue Nov 15, 2019
Open
@rsignell-usgs
Copy link

rsignell-usgs commented Feb 25, 2020 •
edited
Loading

I can reproduce this problem with updated netcdf libs:

# Name                    Version                   Build  Channel
h5netcdf                  0.8.0                      py_0    conda-forge
libnetcdf                 4.7.3           nompi_h9f9fd6a_101    conda-forge
netcdf4                   1.5.3           nompi_py37hd35fb8e_102    conda-forge

I can work around it in netcdf4 by the prepending the URL with the '[FillMismatch]' kludge:

import netCDF4
url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
netCDF4.Dataset('[FillMismatch]'+url)

<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, file format DAP2):
    acknowledgment: This deployment supported by NOAA.
    cdm_data_type: TrajectoryProfile
    ...

but this doesn't work in xarray:

import xarray as xr
xr.open_dataset('[FillMismatch]'+url)

FileNotFoundError: [Errno 2] No such file or directory: b'/home/jovyan/hurricane-ike-water-levels/[FillMismatch]http:/gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'

because xarray (not too surprisingly) thinks it's a local file.

@tompc35
Copy link

tompc35 commented Mar 21, 2020

Ran into the same problem here with with libnetcdf version 4.7.1 and netcdf4 version 1.5.3.

Appending #fillmismatch to the url worked in xarray (version 0.15.0).

import xarray as xr
url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
ds = xr.open_dataset(url+'#fillmismatch')

@nikola-rados nikola-rados mentioned this issue Jun 22, 2020
Closed
@saveriogzz saveriogzz mentioned this issue Oct 21, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@kwilcox @lesserwhirls @jswhit @rabernat @rsignell-usgs @tompc35