Add threading to the filter_topo and orog_gsl codes

[GeorgeGayno-NOAA]

DESCRIPTION OF CHANGES:

• Add threading to the filter_topo and orog_gsl codes.
• Update the regional_gsl_gwd regression test to run with varying thread counts.

TESTS CONDUCTED:

If there are changes to the build or source code, the tests below must be conducted. Contact a repository manager if you need assistance.

• Compile branch on all Tier 1 machines using Intel (Orion, Jet, Hera, Hercules and WCOSS2). Done using 4c0d64c.
• Compile branch on Hera using GNU. Done using 4c0d64c.
• Compile branch in 'Debug' mode on WCOSS2. Done using 4c0d64c.
• Run unit tests locally on any Tier 1 machine. Done on Hera using 4c0d64c. All tests passed.
• Run grid_gen consistency tests locally on all Tier 1 machines. Used 4c0d64c. All tests passed on Orion, Hercules, Jet and Hera. Tests failed on WCOSS2, but this is not a problem. For details, see: Add threading to the filter_topo and orog_gsl codes #948 (comment)

Describe any additional tests performed.

Some stand-alone tests of the filter_topo program:

• Enable OpenMP threading to speed up orog_gsl and filter_topo programs #939 (comment)
• Enable OpenMP threading to speed up orog_gsl and filter_topo programs #939 (comment)

Some stand-alone tests of the orog_gsl program:

• Enable OpenMP threading to speed up orog_gsl and filter_topo programs #939 (comment)

Some independent tests performed by @BinLiu-NOAA

• Enable OpenMP threading to speed up orog_gsl and filter_topo programs #939 (comment)
• Enable OpenMP threading to speed up orog_gsl and filter_topo programs #939 (comment)

DEPENDENCIES:

None.

DOCUMENTATION:

N/A

ISSUE:

Fixes #939.

[GeorgeGayno-NOAA]

The regional.gsl.gwd regression test failed on WCOSS2 (4c0d64c). The files output from the orog_gsl step differed from the baseline. The test is run with both 15 and 30 threads. The differences as computed by the nccmp utility (15 thread run) are:

CHECK C772_oro_data_ls.tile7.halo0.nc
+ /lfs/h2/emc/global/noscrub/George.Gayno/util/nccmp/nccmp-1.8.5.0/src/nccmp -dmfqS C772_oro_data_ls.tile7.halo0.nc /lfs/h2/emc/nems/noscrub/emc.nems/UFS_UTILS/reg_tests/grid_gen/baseline_data/region
al.gsl.gwd/C772_oro_data_ls.tile7.halo0.nc
Variable  Group Count          Sum     AbsSum          Min         Max       Range         Mean      StdDev
stddev    /      1945 -0.000182526 0.00515588 -3.05176e-05 6.10352e-05 9.15527e-05 -9.38439e-08 4.90475e-06
convexity /     12724 -7.51019e-06 0.00569522 -2.86102e-06  3.8147e-06 6.67572e-06 -5.90238e-10 5.15856e-07

CHECK C772_oro_data_ss.tile7.halo0.nc
+ /lfs/h2/emc/global/noscrub/George.Gayno/util/nccmp/nccmp-1.8.5.0/src/nccmp -dmfqS C772_oro_data_ss.tile7.halo0.nc /lfs/h2/emc/nems/noscrub/emc.nems/UFS_UTILS/reg_tests/grid_gen/baseline_data/regional.gsl.gwd/C772_oro_data_ss.tile7.halo0.nc
Variable  Group Count          Sum     AbsSum          Min         Max       Range         Mean      StdDev
stddev    /       469 -6.74836e-06 0.00164294 -1.52588e-05 3.05176e-05 4.57764e-05 -1.43888e-08 5.81491e-06
convexity /     14718   2.6226e-06 0.00822735 -4.76837e-06 4.76837e-06 9.53674e-06   1.7819e-10 6.73586e-07

stddev is the standard deviation of the terrain. For this case it varies from 0 to around 350 meters. The differences from the baseline are very small. convexity ranges from about 1 to 9. Again, the differences are very small.

Rerunning the case with one thread showed the same exact differences. So, there is no problem with the threading. I suspect the OMP library on WCOSS2 may use different math libraries which create the difference.

In conclusion, the code is working and this PR should be merged.

[GeorgeGayno-NOAA]

@BinLiu-NOAA - can you please review?

[BinLiu-NOAA]

Looks good to me. Thanks for the speeding up through OMP threading!