Start running GEM - UQAM style

To run GEM one needs appropriate executables (also called "absolutes") as well as a set of scripts and configuration files.
You will find some examples at the end of this page.

Initial setup

1) Set up SSM environment

Do this part only when running on clusters of "The Alliance" (Compute Canada). On our UQAM servers this is already done at account creation.
You need to do part 1) only once!

Please, follow the instructions on the following wiki page:

     Set up the SSM environment

2) Set up the "GEM model environment"

You need to do part 2) only once!

Please, follow the instructions on the following wiki page:

     Set up the GEM model environment

Extra model scripts

You need to do part 3) only once as well - unless there is an update for the scripts.

The name of the personal scripts directory is not optional and depends on the GEM version you want to run.

For GEM 4.8.lts12 set:
    version='v_4.8.lts12.u'
For GEM 5.0 set:
    version='v_5.0-u1.rc1'
For GEM 5.1.1 set:
    version='v_5.1.1-u1'

Create a directory for all your extra or modified model scripts with:

    mkdir -p ~/modeles/GEMDM/${version}/bin

For now copy all scripts from my directory into yours, for example with:

    cp -p ~winger/modeles/GEMDM/${version}/bin_latest_version/* ~/modeles/GEMDM/${version}/bin

You are done now with the general model setup.

The configuration files

GEM needs four respectively five configuration files (short: "config" files) to run:

• configexp.dot.cfg
• gem_settings.nml
• outcfg.out
• physics_input_table

When running GEM5 and up with CLASS/CLASSIC one extra config file is needed, called:

• CLASS_input_table

Click on the link "configuration files" to learn what they are use for!!!

You need to create one "config" directory per simulation.
Since these config files are small and very important they should always be kept under the home directory. To make it easier to help each other I suggest you put them somewhere under:

For GEM 4.8.lts12 set:
    version='v_4.8.12'
For GEM 5.0 set:
    version='v_5.0.0'
For GEM 5.1.1 set:
    version='v_5.1.1'

    ~/gem/${version}/Configs/...

Also, to make it easier to find the config files of a simulation that is already running or even finished running, I suggest to set the basename of the "config" directory to the same name as the experiment name of the simulation. So something like:

    ~/gem/${version}/Configs/experiment1
or
    ~/gem/${version}/Configs/project1/experiment1

But please, chose better names than 'project1' or 'experiment1'!!!

The config directory needs to contain the four/five config files and will also be the directory from which the simulation will get launched.

If you already have a set of config files you can use them. Otherwise, can find example config files you can copy into your config directory under:     ~winger/gem/${version}/Configs/bcmk
For example:

Once you are ready to run your first "real" simulations, ask me (winger.katja@uqam.ca) or one of your colleagues for a set of config files you can start from!

Launching the simulation

1) Set the model environment

You need to set the model environment in every window from which you want to run/launch the model or in which you want to create the model executable. This is not to be confused with the initial setup. Here you just need to set which model version you want to use to get access certain environment variables and get additions to the PATH.

You can do this by executing one of the following aliases, depending on the model version you want to run:

      4812      # for GEM 4.8.lts12
      500        # for GEM 5.0
      511        # for GEM 5.1.1

Among others, these commands also sets the environment variables $gemdyn and $rpnphy which contain the directory name under which you can find all models dynamic resp. physics routines

2) Start the model

1. Go into your config directory (the directory in which you have your config files - see above).
   
   
2. Launch the model with one of the following two commands:
   

   a)  Um_lance

   This is the basic way to submit the model. It can be used when running the model on our UQAM server as well as for short (one job) simulations on clusters of The Alliance. If your simulation consists of more than one month the model will run one job per month. At the end of each month/job the next month will get submitted automatically => which can lead to multiple long queued times on clusters of The Alliance.
   If you are using the little example config files from above you should use 'Um_lance' to submit the example simulation.

   b)  Chunk_lance

   To avoid extra queued times 'Chunk_lance' will execute as many months as it can in the given wall time of the job (BACKEND_time_mod). At the end of each job the next job will get submitted automatically. While 'Chunk_lance' can be used for all types of simulations it is especially useful for longer simulations.

   If the model crashes and you want to resubmit it to continue the simulation, all you have to do is go back into you config file directory, set the model environment, and execute 'Chunk_lance' again.

   If ever you want to restart a simulation from the very beginning, you can do so by executing:

         Chunk_lance  -start

   You can find more information about Chunk_lance on our wiki by clicking 'here'.

While the model is running

Check that your simulation is running

To check if your simulation is running use the command:
            qs

If ever you need to kill a simulation you can do that with:

    qdel JobID

You can find more information on the following wiki:
   Batch jobs
   Check on batch jobs

What happens while the model is running

The model itself will run under the "execution directory" you created above, called:
    ~/MODEL_EXEC_RUN/machine_name/experiment_name

Each simulation consists of several parts which will all get started/executed/submitted automatically after the initial launch with 'Um_lance' or 'Chunk_lance'.

First some "preparation scripts" will get executed, preparing the execution directory as well as updating your gem_settings.nml and outcfg.out. Then the main "model" will get executed. At the end of each month (and upon request (Out3_close_interval_S) even during a month) the model will submit the "post processing". The post processing will collect, combine and archive all the direct model output, calculate monthly means and variances (if CLIMAT_diagnos=1) and archive the restart files. At the end, the "save listings" job will collect all the listings and put them in your archive (CLIMAT_archdir).

Note: The model does not depend on the post processing jobs. Even if one of these jobs aborts, the model will continue running.

Listings

While the model is running the listings (log files) will appear under the directory:
            ~/listings/${TRUE_HOST}
There names will all start with ${GEM_exp}_*.

Once a job/month finished properly they will all get archived under:
            ${CLIMAT_archdir}/Listings/listings_${GEM_exp}.zip

Model output

While the model is running the output will be under:
            ~/MODEL_EXEC_RUN/${TRUE_HOST}/${GEM_exp}/RUNMOD/output/cfg_0000/firststep_...

Whenever the model or the post processing crashes you should find the output produced so far under the above directories.

The post processing collects the model output and archives it. Once the post processing is finished you find all your model output under:
            ${CLIMAT_archdir}
You find this variable in your file configexp.cfg.

Once the post processing is finished you should have the following directories in the above directory":

And then you should also have this file:
     maingemdm_Linux_x86-64.Abs 
or
     maingemdm
which is a copy of the executable.

There are four types of output files:

        dp...  : dynamics on pressure level
        dm... : dynamics on model levels
        pm... : physics on model levels
        pp...  : physics on pressure levels

What to do with the model output

There are several ways to look at the model output, which is in the RPN standard file format, and to modify it.
Have a look at this web page for some examples.