Note that data on Yin-Yang grids cannot get converted to netCDF format!

cdf2rpn (RPN -> netCDF & netCDF → RPN)

The tool 'cdf2rpn' can get used to convert RPN to netCDF format as well as netCDF to RPN format. One can get the documentation of cdf2rpn by simply executing the command in a terminal:

    cdf2rpn

The general usage is the following:

Convert from NetCDF -> RPN format:

Convert from RPN -> NetCDF format:

One needs to add the key '-dir' (for 'direction')!

General description:

• If you do not add a path to the output file name the file will end up in $TMPDIR!!!
• All fields in a file must have the same timesteps and levels.  
  You cannot have i.e. sigma, hybrid, arbitrary and pressure levels in the same file!!! Nor can there be, for example, a field on 20 pressure levels and another on only 1 pressure level. If your RPN input file contains fields with different types or numbers of levels you can select the fields your want to convert with 'r.diag' and/or 'editfst' - see above.
  
• If converting RPN to NetCDF make sure your RPN file contains all necessary grid descriptors ('^^', '>>', '^>', 'HY','!!')
• All variables you want to convert need to be in the attribute list - see below.
  

Attribute list

The attribute list is a file. It can be considered as a dictionary to translate between RPN ↔ netCDF.

You can find an example file on the UQAM servers under:   ~winger/Scripts/NetCDF_converter/attribut_netcdf.dat

The file consists of:

1) Documentation

At the beginning are several commented lines containing information about the file.

2) Coordinate variables

The first section of this file, after the comments, contains a list of coordinate variables. Do not modify this section!!! It is limited by 
    #---------------- Variables coordonnees (debut) (NE PAS MODIFIER) ---------------------
            :
    #---------------- Variables coordonnees (fin) (NE PAS MODIFIER) ------------------------

3) Global attributes

The second section contains the global attributes. Global attributes are related to the dataset as a whole and should be used to provide a description of the dataset.

4) Fields / Variables

The last section needs to contain an entry for each field/variable that is to be converted. Each entry starts with 'def_attribut' and needs to have the form below.
Make sure the factors are correct!!!

 def_attribut[ name; ccname; mult; add;
               nom_attribut = valeur_attribut;
               nom_attribut = valeur_attribut; N;
               nom_attribut = valeur_attribut; valeur_attribut; N; ]

Where:

   name: name of netCDF variable - mandatory
   cccname: name of RPN variable - mandatory
   mult: multiplication factor for units (type real) DEFAUT 1.0
   add: additive factor for units (type real) DEFAUT 0.0
   nom_attribut: name of the attribute
   valeur_attribut: value associated to "nom_attribut"
   N: integer vale defining the type of "valeur_attribut" (2=character, 3=integer*2, 5=real*4)

Notes: 

• Unit conversion from RPN → netCDF:         unit(RPN) = mult*unit(netCDF)+add 
  
• Unit conversion from netCDF → RPN :        unit(netCDF)= (unit(RPN)-add)/mult 
  
• 'nom_attribut = ...' are only used for conversions to netCDF format.
  
• Never put a space before the ";"!
  
• "[", "]", "=" and ";" are reserved characters. Do not use them otherwise, for example in names or descriptions.

fstd2nc (RPN -> netCDF)

fstd2nc can only get used to convert RPN format for netCDF format. It is less sophisticated than cdf2rpn but also less "picky". 

Pro: It can convert RPN files containing fields with different time steps and on different levels.
Con: It does not have a "dictionary" meaning the names and units in the netCDF file will be the same as in the RPN file. And since units and descriptions are not written in RPN files they will not be in the netCDF file either.

Since fstd2nc is a python function one first needs to load Python, for example with:

The conversion command could then look like:

You can find more info about this tool on the web under:
    https://pypi.org/project/fstd2nc

Compression

In any format, data should get compressed. This does not only save disk space but also speeds up reading and writing because less data have to get read from resp. written on disk. 

There are generally two types of compression: 

• "lossy" compression (removes some of the original information)
• "lossless" compression (preserves all original information)

Lossy compression

Lossy compression removes some of the original information. This can get used if less precision is needed. For example, instead of storing data with 32-bit (or even 64-bit) per real number one could decide to use only 16-bit or even 12-bits. For our usual weather and climate data 16-bits are generally enough.

Lossless compression

Lossless compression preserves all original information but its effectiveness depends very much on the data to be compressed. Smooth fields that contain the same value in several points (like precipitation fields that are often zero) compress very well, whereas "wild" fields, like humidity compress less well.

'zip' and 'gzip' are also compressing files with lossless compression.

Compression in RPN format

Which type of compression is used in an RPN files can be seen with the command 'voir'. The number in the column 'DTY' describes the number of bits used for a real number. If the letter before is upper case, no lossless compression was used. If the letter before is lower case, a lossless compression was used.

Compression in netCDF format

One way to compress netCDF files with lossless compression is by using NCO tools, like 'ncks'. For example: