Global physics fields

As you hopefully already know, GEM can run in parallel, using MPI and/or OpenMP. In the physics, when using MPI, the model domain gets horizontally divided into tiles, or rather cubes. Each MPI process is taking care of one of these cubes.

However, the physics do not see the whole "cube" at once but they are called for one "slab" (ni*nk) at the time. With ni being the number of points in x-direction of the particular tile and nk being the number of vertical levels, which can be atmospheric model levels, soil or ice levels or even "levels" for the different surface fractions.

Busses

Physics fields, that are used in different routines and also available for output, are organized in so called "busses". These busses are very veeeeery long 2-D arrays, filled with one field after the other. For each field there is one row per level, followed by the next field. The starting location of each field in a bus is defined in a variable.

There are 5 of these busses in the physics:

• entry bus (contains initial conditions - almost not used anymore in GEM5)
• dynamic bus (contains fields which get passed from and to the dynamics)
• volatile bus (contains physics fields of which the content is not needed to calculate the next timestep)
• permanent bus (contains physics fields of which the content is needed to calculate the next timestep)
• surface bus (contains only physics fields used by the surface schemes)

Declaring a field in a bus

All fields that are in a bus need to get declared with the command "PHYVARnCx".

The syntax is:

Where :
    'n' is the "real" dimension of the field which can be:  2 (ni*nj), 3 (ni*nj*nk), or 4.
    'name' is the "name" by which the field can get addressed in the physics. This variable is actually an integer which contains the starting position of the field in the bus.
    'description' is a string containing a set of key words followed by a name, description or value. The different keys are separated by a semicolon ';'. These are the accepted keys:

    'condition' is the condition under which the field should get declared. For example, certain fields are only needed when certain schemes are used. So they only need to get declared when the respective scheme is used. Otherwise they should not get declared so they do not use memory space unnecessarily and do not get written in the restart files.

Fields that get only(!) used by the surface schemes need to get declared in rpnphy/surface/sfc_businit.F90.
All other fields need to get declared in rpnphy/base/phyvar.hf.

Entry, dynamic, volatile and permanent bus

As mentioned above, the entry bus is almost not used anymore. It was used for fields which got read by the entry (which does not exist anymore) and then passed to the model.
Fields from the entry bus need to get declared as "VB=e1" - see above.

The dynamic bus contains fields which get passed from and to the dynamics. If possible, do not add fields to the dynamic bus as this gets quite complicated.
Fields from the dynamic bus need to get declared as "VB=d1" - see above.

All pure physics fields that are NOT needed to calculate the next timestep should go in the volatile bus. These are, for example, precipitation rates or instantaneous radiation.
Fields from the volatile bus need to get declared as "VB=v1" - see above.

All fields of which the content is needed in the next timestep need to be in the permanent bus. For example, accumulators need to be in the permanent bus, since they usually get accumulated over several timesteps. The same goes for averages, minimum and maximum fields. But there are also instantaneous fields that need to be in the permanent like snow depth or temperature fields, because they are needed for the next timestep.
Fields from the permanent bus need to get declared as "VB=p0" resp. as "VB=p1" (mandatory to get read at timestep 0) - see above.

Surface bus

This is a special bus which is only valid for the surface schemes which do not have access to any of the busses described above! However, fields from this bus still need to get declared in sfc_businit.F90 or phyvar.hf the in the volatile or permanent bus. To get them copied to the surface bus one needs to add them' in sfcbus_mod.F90 with:

Where:
    'name_sfc' is the name the field will have in the surface bus and
    'name_bus' is the "VN" name the one of the busses above.

The main specialty with this bus is that it gets "adjusted" for each surface fraction. It always contains all surface fields but only for the points for which a certain surface scheme is valid. For example, when a land surface scheme (ISBA, SVS, CLASS) is called, this bus will only contain points with a land fraction greater than a certain threshold. By doing this one avoids nasty if-constructions inside the surface routines, checking that a point has a valid surface fraction, and the bus is more compact, without any "holes", and therefore faster to read.

Accessing fields declared in a bus

There are different ways of accessing fields declared in a bus. But they all have in common that the bus(ses) need(s) to get passed to all routines which want to access fields from the bus. The bus(ses) itself/themselves need to get passes as well as its/their length.

From the basic physics

The main physics subroutine, 'physlb1', passes the three main busses on to 'phyexe'. From there on the busses are generally received and passed on as:

Note that the name for the permanent bus is now 'f' and not 'p' !!!!!!!!!!

To get access to the variable names one needs to also use the module 'phybus':

To be able to treat the fields from the busses like "normal" 1-D (row) or 2-D (slab) fields (remember the physics only see one row or slab at the time - see above) they usually get "transformed" into regular 1-D or 2-D fields by using pointers the following way:

a) Declaration of the field as 1-D or 2-D pointer, for example:

In general, the 1-D or 2-D field name is the same as the starting position of the field in the bus, preceded by a 'z'.

b) Assigning the pointer

In utils/phymkptr.hf several functions are defined to assign a pointer to the right space on the bus, depending on the dimension and type of level (if multi level). They can be used to assign a pointer to an address in a bus so the pointer can then get used in the routine like a "normal" array, for example:

Where
      'zpmoins'/'ztmoins' are the 1- resp. 2-D "arrays" in the routine,
      'pmoins'/'tmoins' the specific position in the bus as declared in phyvar.hf and
      'f'/'p' the name of the bus as received by the routine.

Now, the pointer (zpmoins, ztmoins) can get used in the routine like "normal" arrays.

From the surface schemes

In the main surface routine, surface/sfc_main.F90, all fields needed by the surface schemes get copied from the three main busses (dynamic, permanent and volatile) to the surface bus in calls to the routine 'copybus'. The routine 'copybus' gets called for all surface schemes separately. It copies only points from the three busses to the surface bus for which there is a surface fraction of the respective surface scheme, which greater than a critical value. For example, before the soil routine (ISBA/SVS/CLASS) gets called, 'copybus' will copy all points of the needed fields that contain a soil fraction greater equal 'critmask' to the surface bus. This is done in the calls:

    call copybus3(bus_soil, siz_soil, &
          ptr_soil, nvarsurf, &
          rg_soil, ni_soil, &
          ni, indx_soil, trnch, CP_TO_SFCBUS)

The starting address parameters (the ones declared in sfc_businit.F90 resp. phyvar.hf), will get new values assigned, matching now the start of the variable in the surface bus.
This means that fields in the surface routines are usually shorter than in the rest of the physics and they are usually different for each surface scheme!

After the call to the specific surface routine the output fields from the the specific surface routine will get copied back in the respective main busses, again with the routine 'copybus', for example:

     call copybus3(bus_soil, siz_soil, &
          ptr_soil, nvarsurf, &
          rg_soil, ni_soil, &
          ni, indx_soil, trnch, CP_FROM_SFCBUS)


When called from the main surface routine, surface/sfc_main.F90, the main routines of the different surface schemes then get called, passing to them:

• the surface bus itself
• the size of the surface bus
• the new starting addresses of the variable in the surface bus and
• the number of variables

Inside the respective main surface routines for the different schemes the fields from the surface bus can then get accessed by:

a) Declaration of the field as 1-D or 2-D pointer (same as in other physics routines), for example:

Often, the 1-D or 2-D field name is the same as the starting position of the field in the bus, preceded by a 'z'.

b) Assigning the pointer

Now, the pointer (PS, TA, TS, ZSNOW) can get used in the routine like "normal" arrays.