Abort if registering a restart too late in init()?

[mnlevy1981]

I'm working on the MARBL driver based off of the NCAR fork, and I ran into something unexpected with register_restart_field(). Basically, we use an iterative solver to compute PH, with an initial guess of PH from the previous time step - to ensure bit-for-bit comparisons when restarting, we need these initial guesses in the restart file. I naively called register_restart_field() from a subroutine called by tracer_flow_control_init() and the fields were dutifully being written to the restart file but not being read in upon restarting. I moved the calls to a subroutine called from call_tracer_register() and everything worked as expected.

So it appears to me that the restart file is read in somewhere between the call to call_tracer_register() and the call to tracer_flow_control_init() -- if that's the case, it would be really nice to get an explicit error message when trying to register a new restart field after the restarts have already been processed. I didn't see anything in the log files to indicate I had done anything wrong, so it was dumb luck that I stumbled on the solution on my own.

[marshallward]

Hi Michael,

I discussed with others here and we agree that there should in principle be no reason to call register_restart_field after restore_state has been called. So it may be possible to introduce a "lock" on restart registration and raising an error.

Having said that, there are two minor and one major issue with doing this right now.

The first minor issue is that we do actually call register_restart_field after restore_state as part of the indirect method for writing the initial state. More on this later.

The second minor issue is that restore_state would not be called on an initialization, so we would also need to find a way to introduce this lock without actually calling restore_state.

I don't think either of these instances would be difficult to resolve.

But there is a larger issue, namely that we are preparing for a major overhaul of the FMS I/O, which includes a new handling of restart files (#1165). Any work that we here would likely cause a conflict with this PR. One of the sections heavily impacted by this PR is the initial state output.

There is also the possibility that this ought to be handled at the FMS level, which we can also look into.

Due to the ongoing FMS update, I'm going to recommend that we hold off on this and leave this issue open until the FMS PR has been merged. Does that work for you?

[mnlevy1981]

Due to the ongoing FMS update, I'm going to recommend that we hold off on this and leave this issue open until the FMS PR has been merged. Does that work for you?

That sounds reasonable to me -- I don't think this is an urgent issue, but did want to open a ticket in case others run into similar problems with restart files seemingly not being read.