How do I copy my array forwards in time? #2547
Labels
Comments
|
IIUC, this is actually much easier - reindex on your enlarged |
|
Thanks for your help! you definitely understood me correctly! This doesn't seem to work as it fills my ds = ds.reindex({"time":pd.date_range("2000-01-01","2000-12-31")})
ds = ds.ffill("time") |
|
Your original value needs to be in the time index in order to remain there after the reindex. Currently it looks like a float value: If you have a repro example (link in issue template) it's easier to offer help on the whole issue |
|
Data: Code below: import numpy as np
import pandas as pd
import xarray as xr
from shutil import copyfile
import os
data_dir = "./"
filename = "Rg_dummy.nc"
# get the datetime range
times = pd.date_range("2000-01-01", "2000-12-31", name="time")
var = "Rg"
copyfile(filename, "temp.nc")
ds = xr.open_dataset("temp.nc")
print("Temporary Data read to Python")
# FORWARD FILL FROM THE ORIGINAL DATA to new timesteps
ds.reindex({"time":times})
ds.ffill("time")
# ds.to_netcdf(filename, format="NETCDF3_CLASSIC")
# print(filename, "Written!")
# remove temporary file
os.remove(data_dir+"temp.nc")
print("Temporary Data Removed")
del ds |
|
Can you ensure |
|
The ds.time[0] won't let me set it's value to a datetime. Instead it returns a float: In [19]: ds.time[0].item()
Out[19]: 10509.0And none of the following work: # doesn't change the time value
ds.time[0].values = times[0]
# returns an error because I can't assign to a function call
ds.time[0].item() = times[0]
# returns ValueError: replacement data must match the Variable's shape
ds['time'].values = np.array(times[0])Thanks for your help! |
|
@tommylees112 I had a look at your dataset and noticed that the time coordinate was not encoded in a way that allows xarray to automatically decode the time values into datetimes. Specifically, the units attribute is not in the format of some unit of time (e.g. 'seconds') since a given reference date. This is why Regarding assigning a new value to the time coordinate, the following should work: ds['time'] = np.array([times[0]]) |
|
That all worked great until I tried to write out to a .nc file. data_dir = "./"
filename = "Rg_dummy.nc"
# get the datetime range
times = pd.date_range("2000-01-01", "2000-12-31", name="time")
var = "Rg"
copyfile(data_dir + filename, "temp.nc")
ds = xr.open_dataset("temp.nc")
print("Temporary Data read to Python")
# FORWARD FILL FROM THE ORIGINAL DATA to new timesteps
ds['time'] = np.array([times[0]])
ds.reindex({"time":times})
ds.ffill("time")
ds.to_netcdf(filename, format="NETCDF3_CLASSIC")
print(filename, "Written!")
# remove temporary file
os.remove(data_dir+"temp.nc")
print("Temporary Data Removed")
del dsI get the following Error message: and if I try with the default netcdf writing options ds.to_netcdf(filename)
print(filename, "Written!")I get this error message: Version informationIf this is useful: |
|
Thanks @tommylees112 -- note that the Regarding the issue saving to files -- I can reproduce that issue with older xarray versions. It is related to |
|
@spencerkclark Thanks very much that is awesome! One final Q: How do I set the The current output of
|
|
The Is the time encoding important for your land surface model? That does change in my example (see the units attribute); you might need some special logic to handle that. |
|
@tommylees112 - this issue has sat for a bit of time now. Did you end up with a solution here? If so, is okay with you if we close this out? |
jhamman
added
topic-CF conventions
needs mcve
|
Sorry for the silence! I got pulled away to another project. Unfortunately I wasn't able to finish completing the task in xarray but I found that the easiest way around the problem was to use a combination of two functions: def change_missing_vals_to_9999f(ds, variable):
""" Change the missing values from np.nan to -9999.0f"""
arr = ds[variable].values
# set the values to -9999
arr[np.isnan(arr)] = -9999
# reassign the values back to the array
ds[variable] = (ds[variable].dims, arr)
return ds
def change_missing_data_values(filename):
""" change the values INSIDE the .nc file to -9999.0f """
assert (
filename.split(".")[-1] == "nc"
), "This function only works with .nc files. Filename: {}".format(filename)
print("** Processing {} **").format(filename)
# ONLY OPEN THE DATASET ONCE
ds = xr.open_dataset(filename)
variables = ds.data_vars.keys()
for variable in variables:
print("** Working on variable {} **".format(variable))
ds = change_missing_vals_to_9999f(ds, variable)
# ds.map(change_missing_vals_to_9999f, variable)
# rewrite to netcdf file
ds.to_netcdf(filename)
print("** Written variables {} to filename {} **").format(variables, filename)
returnand then another function using the RUN HERE: |
|
But the original question was answered so thank you very much! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have this
xr.Dataset:I want to copy it through time, adding the time dimension at a given daterange
Something like this:
So I should have output data like:
The text was updated successfully, but these errors were encountered: