...
Using MPI to parallelize GEM is more efficient than using OpenMP. To parallelize the a simulation using MPI the full model domain (including pilot region) gets divided into "tiles". Each MPI process will then take care of one tile. Meaning there will be as many MPI processes as there will be tiles.
...
To use MPI one needs to set the first two numbers of 'GEM_ptopo' to the number of tiles one would like to use in x- (MPI-x) and y-direction (MPI-y). For example, to use 2 tiles in x- and 2 tiles in y-direction set:
GEM_ptopo=2x2x1
FYI, if the total number of points of the model grid is not exactly dividable by the number of MPI tiles, the first tile(s) will get more points and the other(s) less.
Basic rules to consider
- The minimum tile size is 23x23 points!
- The maximum tile size depends on the schemes used: The "memory heavier" a scheme, the smaller the tiles need to be. For example, 'P3' needs more memory than 'consun' and when adding 'TEB' the simulation will also need much more memory. In general, a maximum tile size of about 100x100 points should not get exceeded.
- Tiles should be as square as possible to minimize the information exchange over the tile borders.
- If tiles cannot be square and one has a choice, it is preferable to have more tiles in y- than in x-direction to help the solver.
...