Sparse Matrix Multiply

Structured to Structured Sparse Matrix Multiply Use Test Case Description

Use Case Description

This Use Test Case suite consists of two top level drivers:

ESMF_StrStrRemapUseTestCase.F90 for remapping between two structured grids
ESMF_UStrUStrRemapUseTestCase.F90 for remapping between two unstructured grids

Both of these programs employ the SparseMatMult routines to remap (or interpolate) a test field from one user specified grid to another. The user must provide three files for the remapping:

Source grid
Destination grid
Source to destination remapping weights

All files are assumed to be in netCDF format and use the structure specified by the SCRIP remapping software.
Currently only first order remapping is supported. The following remapping options are available from SCRIP:

First order conservative
Bilinear
Distance-weighted Average

See the SCRIP documentation for more details.

Test Description: The SparseMatMul performs simple one-way remapping from the source grid to the destination grid. For this example the grids are assumed to be any sort of single tile, logically rectangular structured grid, which is either regional or global in extent. This qualification includes the both the class of rectilinear as well as curvilinear or "tensor" style grids.

The default configuration remaps a global domain T42 spectral atmospheric grid to a 10 km regional ROMS grid consisting of a Lambert conformal conic projection of the Northeast Pacific bordered by Alaska, Western Canada and the Aleutian peninsula. The files necessary for conducting a second test remapping between a global 4/3 degree Displaced Pole POP grid and a 65 km resolution Mercator grid are also located in the directory. Additional grid and remapping files are available from the SCRIP distribution. They They include the following:

POP, 4/3 degree Displaced Pole Grid
T42 Gaussian Grid
1 degree spherical Grid

A suite of regional grids are also available upon request. These include:

Mercator
Lambert Conformal Conic
Stereographic
finite element regional

Once a pair of grids is selected, the remapping weights may be generated using the SCRIP package. The files containing the grid and the remapping weights are specified by the configuration file StrStr_sample.rc as source grid, destination grid, etc. See the readme for a thorough explanation of how to set up the configuration file.

A fortran field is constructed on the source grid whose data is set to the given geometric function:

   A₀ + A₁ * sin(B₁ * latitude) + A₂ * cos(B₂ * longitude)

where the coefficients A₀, A₁, A₂, B₁, and B₂ are specified by the config file. The fortran field is remapped using the SparseMatMul routines. These routines ingest the remapping weights and the corresponding list of linking address pairs. The destination field is the result of a weighted sum of source field values, mathematically expressed as:

do k=1,number_link_pairs
  destination_field(address(2,k)) = destination_field(address(2,k)) + weights(k) * &
  source_field(address(1,k))
enddo

The destination field is compared to the analytical solution in order to provide measurement of the accuracy of the remap. Those values are output for each DE.

Applications: This use test case describes how to remap between any two grids that can be expressed as lists of corner and center values. A few real world examples are:

POP to WRF (Global 4/3 degree POP Displaced Pole Grid to Stereographic, Mercator, etc.)
POP to ROMS
MM5 to POP (T42 Gaussian Grid to Global 4/3 degree POP)
ROMs to WRF
ELCIRC (finite volume) to WRF
ADCIRC (finite element) to WRF
WASH123 to WRF

The only limitation is the ability to construct the remapping weights.

Test Results

Test Configuration

Test Programming Language: Fortran90
Platform: Compaq SC45 (halem)
Platform: SGI IRIX64 (Tempest)
Platform: Apple G5 Dawin/Absoft/Lam (Glass)
Number of DEs: 8
Number of nodes: 8
Communications Method: MPI

Sections

The main program of the test application consists of the following sections.

Initialize
IO
Create Arrays
Construct Test Field
Multiply
Check
Destroy

Initialize

Sets up the VM
Reads the namelist variables

I/O

Inputs the netCDF files containing

Source grid
Destination grid
Source to destination remapping weights

Create Arrays

create source and destination arrays
reshape grid fields from 1D into 2D

Initialize Test

Construct the test field to be remapped,

   A₀ + A₁ * sin(B₁ * latitude) + A₂ * cos(B₂ * longitude)

based on namelist coefficients.

Multiply

call ESMF_ArraySparseMatMulStore to distribute remapping weights
call ESMF_ArraySparseMatMulRun to conduct remapping

Check

Verify the remapped data by comparing it to the geometric function evaluated on the second Grid.
Prints out a PASS/FAIL message.

Destroy

Deallocates memory for created DistGrids and Array objects by calling destroy.