Discussion Forum

In a COMSOL file, saving is done atomically. If the file is large (several GB for example) this process can take an annoying large amount of time, even on a fast SSD. Because of the flat file format, making any changes could change the internal offsets at which data is stored, so doing nonatomic saves probably won't work. Even if they did, atomic saves are a good safeguard against data loss in the event of an error.

I can only see one obvious solution for this. The data sets could be stored outside the file, so that when saving, only the model and its metadata would need to be overwritten instead of all of the solution data.

Another issue is the fact that sometimes time dependent studies are needed where a stationary study may not converge, or the solution may not actually be steady state but all that is needed is an approximation, for use as input to another model. In this case, huge amounts of time dependent data must be stored just to be able to use the final value (or some chosen time step) for input to another study. It would be beneficial in these cases to be able to modify data sets to remove unnecessary steps after the solution is complete, and only save the chosen time steps.

Is any of this possible? What tricks do you use for dealing with large/multiple data sets?