Restart a simulation when having submitted it with 'Chunk_lance'.

If the simulation was stopped (job was killed) or crashed it can often get re-submitted by executing again the command:

    Chunk_lance

But sometimes it happens that part of the restart files gets overwritten before a month is finished or that the restart files are corrupted or one would simply like to rerun part of a simulation. In that case one cannot just relaunch the month with Chunk_lance but has to get the original (uncorrupted) restart files back (from the month previous to the one one wants to rerun) and restart the simulation from there.

When the model job stops for whatever reason, there is an automatic check in Chunk_lance if the restart files are still the original ones or not. If they got already modified the following message will appear in the "chunk_job listing" (!!!) not in the model listing:

   At least one of the restart files got already rewritten
   Therefore the model could not get restarted automatically
   You have to restart your simulation starting from the previous restart files
            ----- ABORT -----
 
If you see this message one cannot simply restart the simulation with Chunk_lance but as said, one need to restart it from the previous restart file - see below.

Restart a simulation from a previous restart file

1. Make sure the restart files of the previous month are in the execution directory :
          ~/MODEL_EXEC_RUN/${TRUE_HOST}
   If the are not there you need to put them back.
   Click on the following link to learn more about restart files.
2. Go into the config file directory
3. Edit the log file 'chunk_job.log':
   (First I suggest to make a backup copy of the log file.)
   The remove all lines concerning the month you want to rerun and all following lines.
   Click on the following link to learn more about the file 'chunk_job.log'.
4. Make sure the "Scripts-job", ${GEM_exp}_S, is in the config directory
   In case this file is not in your config directory anymore you can find it in your archive under:
          ${CLIMAT_archdir}/Listings/jobs_*.zip
   
5. Execute "Chunk_lance" again (without the '-start'!!!)