Make PT-TEMPO save memory by writing results to file during computation

[gefux]

With the new FileProcessTensor it is possible to write a process tensor to disc part for part. When the final time is larger then the non-Markovian memory time, the first parts of the process tensor are calculated before the entire computation has finished. As a result one can easily save memory during the computation of such a process tensor. It would be good if the code would be changed to do that.

[piperfw]

What is the status of this issue? I noticed if oqupy.pt_tempo_compute call (with a filepath argument) is interrupted a file is left on disk, but this gives a ValueError if subsequently used in a dynamics calculation.

[gefux]

This is because when called with a filepath argument, a FileProcessTensor is created and the corresponding hdf5 file is initiated. It makes sense in my opinion to initiate that file at the beginning of the computation because then there is at least the possibility to write to disc the completed parts of the process tensor as proposed in this issue.
One could start the initialization of the file with a field that signals that the content is not yet a valid process tensor to be able to create a more helpful error message when loaded. I think it will come to no surprise to most users that an interupted computation will not give you a valid process tensor file, so I don't see this to be a significant issue at the moment.

[piperfw]

Thanks for the update, I wasn't sure if this had been worked on. Agree nonissue.

[gefux]

With commit 73e8259 there is now a "writing" flag in the FileProcessTensor set while the tensors are being populated. When a file with this flag on is imported then a warning "File was closed during writing process and hence may be corrupt" is put out.