General information

Domain & resolution

First you should decide upon the final(!) (see "Cascading" below) domain and resolution you want to run. For this it is important to know that:

• the more grid points a grid has => the more cores are needed
• the higher the resolution => the shorter the model timestep => the longer the runtime of the simulation

Have a look at the following wiki about how to set up a model grid: Set up a model grid

Number of points

Topology (MPI & OpenMP)

Resolution → timestep

The relation between the horizontal resolution and the model timestep is linear. Here are some examples:

Resolution → timestep → radiation timestep

Calculating the increments on different fields depending on the radiation is rather costly. Therefore, the option exists to calculate the increments only every n seconds but still apply them at every timestep. The parameter to set is called 'KntRad_S' from the 'gem_settings.nml'. It should get set to a multiple 4 to 6 of the model timestep. The unit is seconds.

Resolution → (non) hydrostatic

If the grid resolution is ~10 km or lower the model is hydrostatic and one should set 'Dynamics_hydro_L = .true.' in the 'gem_settings.nml'. For grid resolutions of ~4km and higher one should set 'Dynamics_hydro_L = .false.'

Model levels

The model levels on GEM are terrain following at the surface and pressure levels at the model top. The model levels are generally closer to each other at the surface and further apart higher up. The model is very sensitive to the location of these model levels. They must neither be too far apart nor too close and the transition from terrain following to pressure levels must be very smooth. There are ways to set up these model levels. Several sets of model levels have already been created and can get selected by setting 'CLIMAT_etaname' in the file configexp.cfg. The possible settings for this parameter can get found in the file:
      ~/modeles/GEMDM/*/bin/Climat_eta2preshyb

Vertical sponge & top piloting (LAM only)

Like other models, GEM gets unstable when waves get artificially reflected back into the atmosphere from the model lid. Therefore, a so called "vertical sponge" is used at the model top to absorb these waves so they do not get reflected. For more information and how to set up the vertical sponge (Vspng) have a look at the following wiki: Diffusion & Sponges

In LAM model one also has the option of nesting the top model layers. Which is also called "top piloting".

When top piloting is used the vertical sponge should NOT get used!!!

Global grid → adjust global pressure

Cascadeing

Spectral nudging

Period

Greenhouse gases / Scenario

Initial conditions

Driving data

Geophysical fields

Schemes

Land surface scheme (ISBA, CLASS, SVS)

Lake scheme (non, FLake, CSLM)

Urbain scheme (non, TEB)

Radiation

Roughness length

Limit ice /snow

Gravity wave drag

Emissivity

Condensation

Convection

Tracers to advect

Precipitation (Bourgouin)

Boundary layer

Horizontal diffusion

Output fields

Output fields

Instantaneous / averages / min / max

Initial condition fields

Pilot fields (2-D / 3-D)

Output frequency

Output levels

Size of output files (monthly/daily/hourly/...)

Submission

Account

Wall clock time

On the clusters of the Alliance are different queues for jobs requesting different amounts of walltime as well as memory. To find out which type of queues exist on the cluster you want to run on check out the following wiki of the Alliance: Job_scheduling_policies

In the config file 'configexp.cfg' set 'BACKEND_time_mod' to the amount of seconds you want to request for a single job(1) - not for the whole simulation which can consist of multiple jobs.

In general, the shorter the requested time the shorter the queued time. Therefore, you want to request as little time as possible for a job. However, runtimes on clusters of the Alliance can vary by a lot, normally between -15 % to + 20 % of the average runtime. Hence, one should request runtimes that are about 25 % larger than the average expected runtime.

If you only want to run one job you can just request "enough" runtime. But when you are running a simulation with a sequence of jobs you might want to try to fit the jobs at the higher end of a queue. For Assuming, there are the following queues on a cluster:

• 3 hours or less,
• 12 hours or less,
• 24 hours (1 day) or less,
• 72 hours (3 days) or less,

And the simulation you want to run usually needs 14 hours per job.

Memory