HYCOM

official
AMD
- AMD-EPYC-with-HYCOM
  - "GLB" stands for Global HYCOM Benchmark
  - GLBT0.72, is smaller and easily runs on a single compute node
  - GLBT0.08, is a more challenging workload and is equivalent to a true production run.
- Guide-To-Running-HYCOM-on-EPYC
Ocean Modeling with HYCOM on AWS Graviton2 - High Performance Computing (HPC) blog - Arm Community blogs - Arm Community
:star:HYCOM · Wiki · Arm HPC Resources / packages · GitLab
compiler
- Compiler Commands

The script Make_global.csh is just Make.csh configured for the typical global tripole case.

The Makefile sources ./config/\$(ARCH)_\$(TYPE) where ARCH defines exactly what machine architecture to target and TYPE is the parallelization strategy and precision (one, omp, mpi, ompi, mpisr, shmem).

If some problems occur, read the error messages, correct the errors and be sure to remove the .o and .mod files before running the Make.com again.

all the input/ouput files of HYCOM are presented as a couple of .a and .b files. The .a file actually contains the binary data and the .b file describes what is in the .a file and how it is stored.

how to run source code 2.3.01

experiment executable(hycom tool, pre/post processing tool)

Prepare basic environment

``` bash=

basic env

sudo yum group install "Development Tools" sudo yum install -y curl findutils gcc-c++ gcc gcc-gfortran git gnupg2 hostname iproute redhat-lsb-core make patch python3 python3-pip python3-setuptools unzip

for runing Make.csh

sudo yum install -y csh make environment-modules

spack

git clone -c feature.manyFiles=true https://github.com/spack/spack.git . spack/share/spack/setup-env.sh spack install intel-oneapi-compilers@2023.0.0 spack compiler add spack location -i intel-oneapi-compilers/compiler/latest/linux/bin/intel64 spack compiler add spack location -i intel-oneapi-compilers/compiler/latest/linux/bin spack compilers

spack install openmpi@4.1.5 %gcc

spack install openmpi@4.1.5 %intel@2021.8.0

spack install intel-oneapi-mpi %gcc

spack install intel-oneapi-mpi %intel@2021.8.0

spack install netcdf-c ^openmpi %gcc

spack install netcdf-fortran ^openmpi %gcc

spack install netcdf-c ^openmpi %intel@2021.8.0 spack install netcdf-fortran ^openmpi %intel@2021.8.0

for environment-modules files

spack module tcl refresh spack load openmpi@4.1.5 spack load intel-oneapi-compilers@2023.0.0

ansbile

sudo yum install ansible-core.x86_64

add fortune compiler path /bin/gfortran

vim /home/smci/.spack/linux/compilers.yaml

compilers: - compiler: spec: gcc@8.5.0 paths: cc: /usr/bin/gcc cxx: /usr/bin/g++ f77: /bin/gfortran fc: /bin/gfortran flags: {} operating_system: rhel8 target: x86_64 modules: [] environment: {} extra_rpaths: []

HYCOM-examples example source from github

wget -O HYCOM-examples.zip https://github.com/HYCOM/HYCOM-examples/archive/refs/heads/master.zip && unzip HYCOM-examples.zip && mv HYCOM-examples-master HYCOM-examples cd HYCOM-examples/datasets/ vim get_datasets.csh

long time because large size(42.13G)

18G CFSR_GOM.tar.gz

16G CFSv2_GOM.tar.gz

802M CORE2_NYF.tar.gz

139M PHC3_HYCOM.tar.gz

3.6G WOA13_HYCOM.tar.gz

3.6G WOA18_HYCOM.tar.gz

csh get_datasets.csh

HYCOM-tools

git clone --recursive https://github.com/HYCOM/HYCOM-tools.git

wget -O HYCOM-tools.zip https://github.com/HYCOM/HYCOM-tools/archive/refs/heads/master.zip && unzip HYCOM-tools.zip && mv HYCOM-tools-master HYCOM-tools

Make_all.csh. This creates all executables in all source directories, except plot and those that depend on the NetCDF library

Running Make_ncdf.csh in the HYCOM-tools root directory creates all netCDF executables in all source directories, including HYCOM-tools/bin

cd HYCOM-tools csh Make_clean.csh vim Make_all.src

setenv ARCH amdIF

vim bin/Make_all.csh

if ($OS == "Linux") then setenv OS LinuxAIF

csh Make_all.csh >& Make_all.log module avail vim Make_ncdf.src

--- intel compiler module unload netcdf-fortran-4.6.0-intel-2021.8.0-5hpf7gh module unload netcdf-c-4.9.0-intel-2021.8.0-etqveet module load netcdf-c-4.9.0-intel-2021.8.0-etqveet module load netcdf-fortran-4.6.0-intel-2021.8.0-5hpf7gh module list setenv NCDFC /home/smci/spack/opt/spack/linux-rhel8-zen2/intel-2021.8.0/netcdf-c-4.9.0-etqveetgf6tzouaveywcqkxgnqb6ef47 setenv NCDF /home/smci/spack/opt/spack/linux-rhel8-zen2/intel-2021.8.0/netcdf-fortran-4.6.0-5hpf7gh76cytq6q7ob5rymcay3gj6qu3 setenv EXTRANCDF "-L${NCDFC}/lib -L${NCDF}/lib -lnetcdf -lnetcdff"

--- gcc

module load costinit

module unload netcdf-fortran-4.6.0-gcc-8.5.0-inxalsx module unload netcdf-c-4.9.0-gcc-8.5.0-2qwncum module load netcdf-c-4.9.0-gcc-8.5.0-2qwncum module load netcdf-fortran-4.6.0-gcc-8.5.0-inxalsx module list setenv NCDFC /home/smci/spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/netcdf-c-4.9.0-2qwncumfartt3qrjqpixzxgipxs3hgrq setenv NCDF /home/smci/spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/netcdf-fortran-4.6.0-inxalsxzrfxb4q4nposghyu44gpw4vuy setenv EXTRANCDF "-L${NCDFC}/lib -L${NCDF}/lib -lnetcdf -lnetcdff"

vim bin/Make_ncdf.csh

if ($OS == "Linux") then setenv OS LinuxAIF

cat config/amdIF_setup csh Make_ncdf.csh >& Make_ncdf.log

### run GLBt0.72

#### data preparation
``` bash=
sed -i 's|setenv S  /p/work1/${user}|setenv S  /home/smci|g' /home/smci/HYCOM-examples/GLBt0.72/datasets/get_datasets.csh
csh get_datasets.csh
# All the input files for the GLBt0.72 configuration can be downloaded from the HYCOM.org website through the GLBt0.72/datasets/get_datasets.csh script.

# Maybe option
# https://github.com/HYCOM/HYCOM-examples/wiki/GLBt0.72#how-to-create-forcing-and-relax-files-for-glbt072
export PATH=$PATH:/home/smci/HYCOM-tools/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(spack location -i netcdf-c@4.9.0%intel)/lib:$(spack location -i netcdf-fortran@4.6.0%intel)/lib
cd /home/smci/HYCOM-examples/GLBt0.72/topo && sed -i 's|setenv SP  /p/work1/${user}|setenv SP  /home/smci|g' *.csh
find /home/smci/HYCOM-examples/GLBt0.72/topo/ -type f -name "regional.grid.csh" -execdir csh {} \;
cd /home/smci/HYCOM-examples/GLBt0.72/force && sed -i 's|setenv SP  /p/work1/${user}|setenv SP  /home/smci|g' *.csh
find /home/smci/HYCOM-examples/GLBt0.72/force/ -type f -name "*.csh" -execdir csh {} \;
cd /home/smci/HYCOM-examples/GLBt0.72/relax && sed -i 's|setenv SP  /p/work1/${user}|setenv SP  /home/smci|g' EXPT.src
find /home/smci/HYCOM-examples/GLBt0.72/relax/ -type f -name "iso_sigma*" -execdir csh {} \;
find /home/smci/HYCOM-examples/GLBt0.72/relax/ -type f -name "z_*" -execdir csh {} \;
find /home/smci/HYCOM-examples/GLBt0.72/relax/ -type f -name "relax*" -execdir csh {} \;
find /home/smci/HYCOM-examples/GLBt0.72/relax/ -type f \( -name "sefold*" -o -name "tbaric*" -o -name "thkdf4*" \)
# in addition for mpi
cd /home/smci/HYCOM-examples/GLBt0.72/topo/partit/15 && sed -i 's|setenv DS /p/work1/${user}|setenv DS  /home/smci|g' *.csh
csh depth_2d_Ssq.csh
csh size_S.csh
csh ppm1.csh
csh resize8.csh

run the experiment

``` bash=

new experiment directory

cd /home/smci/HYCOM-examples/GLBt0.72 mkdir expt_02.0 cp expt_01.2/new_expt.csh expt_02.0 cd expt_02.0 sed -i 's|setenv O 011|setenv O 012|g' new_expt.csh sed -i 's|setenv N 012|setenv N 020|g' new_expt.csh sed -i 's|setenv OS HPE|setenv OS Linux|g' new_expt.csh sed -i 's|setenv SCRN /p/work1/${user}|setenv SCRN /home/smci|g' new_expt.csh

build hycom binary by type

...

set up and ready to run

csh new_expt.csh sed -i 's|setenv QSUBMIT ~wallcraf/bin/q_navo|setenv QSUBMIT /bin/csh|g' EXPT.src vim GLBt0.72/expt_02.0/blkdat.input === 5.0 'hybthn' = HYBGEN: ratio of layer thicknesses to select the thiner 3 'hybmap' = HYBGEN: remapper flag (0=PCM, 1=PLM, 2=PPM, 3=WENO-like) ===

vim EXPT.src vim 020.csh

type one

=== C --- Set parallel configuration, see ../README/README.expt_parallel. C --- NOMP = number of OpenMP threads, 0 for no OpenMP, 1 for inactive OpenMP C --- NMPI = number of MPI tasks, 0 for no MPI C switch ($OS) case 'Linux': setenv NOMP 0 setenv NMPI 0 breaksw

setenv A "a" setenv B "b" setenv Y01 "011" setenv YXX "011"

if ($NOMP == 0) then setenv NOMP 0 endif ===

type mpi

=== C --- Set parallel configuration, see ../README/README.expt_parallel. C --- NOMP = number of OpenMP threads, 0 for no OpenMP, 1 for inactive OpenMP C --- NMPI = number of MPI tasks, 0 for no MPI C switch ($OS) case 'Linux': setenv NOMP 0 setenv NMPI 16 breaksw

setenv A "a" setenv B "b" setenv Y01 "011" setenv YXX "011" ===

spack load openmpi@4.1.5 %intel export PATH=$PATH:/home/smci/HYCOM-tools/bin nohup csh 020.csh >& results.log &

check if run successfully

grep "^normal stop" data/summary_out

#### build hycom binary by type
type is one
``` bash=
git clone --recursive https://github.com/HYCOM/HYCOM-src.git /home/smci/HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_one

cd /home/smci/HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_one
module avail
vim Make.csh
# ARCH xc40-intel-relo_one is from config/xc40-intel-relo_one
--- HPE
unset echo
module purge
#module load compiler/intel/2017.4.196
#module load compiler/intel/2023.0.0
#module load mpt/2.16
module load intel-oneapi-compilers-2023.0.0-gcc-8.5.0-7ccmh6l
module list
set echo
setenv ARCH xc40-intel-relo_one
---

rm *.o *.mod *.log hycom
csh Make.csh >& Make_hycom.log
ls -la hycom

type is mpi ``` bash= git clone --recursive https://github.com/HYCOM/HYCOM-src.git /home/smci/HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi cd /home/smci/HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi module avail vim Make.csh

ARCH intelsse-impi-sm-relo is from config/intelsse-impi-sm-relo

setenv OCN_EOS -DEOS_9T ## EOS 9-term

setenv OCN_EOS -DEOS_17T ## EOS 17-term

setenv OCN_GLB -DARCTIC ## global tripolar simulation

setenv OCN_GLB ""

--- OPENMPI unset echo module purge module load intel-oneapi-compilers-2023.0.0-gcc-8.5.0-7ccmh6l module load openmpi-4.1.5-intel-2021.8.0-v2iv375 module list set echo setenv ARCH intelsse-impi-sm-relo

rm .o .mod *.log hycom csh Make.csh >& Make_hycom.log ls -la hycom

config/intelsse-impi-sm-relo_mpi

FC = mpifort

FCFFLAGS = -traceback -xSSE4.2 -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small

FCFFLAGS = -traceback -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small CC = icc

CCFLAGS = -traceback -xSSE4.2 -O -mcmodel=small

CCFLAGS = -traceback -O -mcmodel=small CPP = cpp -P CPPFLAGS = -DREAL8 -DMPI -DSERIAL_IO -DENDIAN_IO -DNAN2003 -DTIMER -DRELO $(CPP_EXTRAS) LD = $(FC) LDFLAGS = -V $(FCFFLAGS) -static-intel EXTRALIBS =

--- generic make definitions

SHELL = /bin/sh RM = \rm -f

rules.

.c.o: $(CC) $(CPPFLAGS) $(CCFLAGS) -c $*.c

.F90.o: $(FC) $(CPPFLAGS) $(FCFFLAGS) -c $*.F90

### run GOMb0.08

#### data preparation
``` bash=
sed -i 's|setenv S  /p/work1/${user}|setenv S  /home/${user}|g' /home/smci/HYCOM-examples/GLBt0.72/datasets/get_datasets.csh
csh get_datasets.csh

run the experiment

cd /home/smci/HYCOM-examples/GOMb0.08
mkdir expt_02.0
cp expt_01.0/new_expt.csh expt_02.0
cd expt_02.0
sed -i 's|setenv  O 011|setenv  O 012|g' new_expt.csh
sed -i 's|setenv  N 012|setenv  N 020|g' new_expt.csh
sed -i 's|setenv OS   HPE|setenv OS   Linux|g' new_expt.csh
sed -i 's|setenv SCRN /p/work1/${user}|setenv SCRN /home/${user}|g' new_expt.csh
sed -i 's|setenv W  /p/work1/abozec|setenv SCRN /home/${user}|g'

# build hycom binary
git clone --recursive https://github.com/HYCOM/HYCOM-src.git /home/smci/HYCOM-examples/GOMb0.08/src_2.3.01_relo_mpi
vim Make.csh

---
#setenv OCN_EOS -DEOS_9T  ## EOS  9-term
setenv OCN_EOS -DEOS_17T ## EOS 17-term
#setenv OCN_GLB -DARCTIC ## global tripolar simulation
setenv OCN_GLB ""
---

--- OPENMPI
unset echo
module purge
module load intel-oneapi-compilers-2023.0.0-gcc-8.5.0-7ccmh6l
module load openmpi-4.1.5-intel-2021.8.0-v2iv375
module list
set echo
# ARCH smci-intel-sm-relo is from config/smci-intel-sm-relo
setenv ARCH smci-intel-sm-relo
---


rm *.o *.mod *.log hycom
csh Make.csh >& Make_hycom.log
ls -la hycom

# set up and ready to run
cd ../expt_02.0
csh new_expt.csh
sed -i 's|setenv QSUBMIT ~wallcraf/bin/q_navo|setenv QSUBMIT /bin/csh|g' EXPT.src
vim GOMb0.08/expt_02.0/blkdat.input
---
   1.0    'hybthn' = HYBGEN: ratio of layer thicknesses to select the thiner
   3      'hybmap' = HYBGEN: remapper  flag (0=PCM, 1=PLM,  2=PPM, 3=WENO-like)
   ...
   0      'sshflg' = diagnostic SSH flag (0=SSH,1=SSH&stericSSH,2=SSH&stericMONTG)
---

vim 020.csh
# type mpi
===
C --- Set parallel configuration, see ../README/README.expt_parallel.
C --- NOMP = number of OpenMP threads, 0 for no OpenMP, 1 for inactive OpenMP
C --- NMPI = number of MPI    tasks,   0 for no MPI
C
switch ($OS)
case 'Linux':
  setenv NOMP 0
  setenv NMPI 16
  breaksw

setenv A "a"
setenv B "b"
setenv Y01 "011"
setenv YXX "011"
===

cd expt_02.0 && touch 020y011a.limits && echo "-3653.00000  3684.00000" > 020y011a.limits

# begin to run
spack load openmpi@4.1.5 %intel
export PATH=$PATH:/home/smci/HYCOM-tools/bin
nohup csh 020.csh >& results.log &

# check if run successfully
grep "^normal stop" data/summary_out

config/smci-intel-sm-relo_mpi

#
# ---------------------------------------------------------------------------
# common definitions for Intel Fortran and MPI, small memory, real*8
# -fp-model source to maintain bit for bit equality for all NMPI values
# On IBM iDATAPlex, use the mpi/intel/impi module
# Emulate SGI Altix ICE on IBM iDataPlex using -xSSE4.2
# version that uses Serial I/O for HYCOM
# relocatable version
# ---------------------------------------------------------------------------
#
# MACROS      DESCRIPTIONS:
#
# FC:         Fortran 90 compiler.
# FCFFLAGS:   Fortran 90 compilation flags.
# CC:         C compiler.
# CCFLAGS:    C compilation flags.
# CPP:        cpp preprocessor (may be implied by FC).
# CPPFLAGS:   cpp -D macro flags.
# LD:         Loader.
# LDFLAGS:    Loader flags.
# EXTRALIBS:  Extra local libraries (if any).
#
FC            = mpifort
#FCFFLAGS      =        -traceback -xSSE4.2 -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small
FCFFLAGS      = -traceback  -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small
CC            = icc
#CCFLAGS       =        -traceback -xSSE4.2 -O -mcmodel=small
CCFLAGS       = -traceback -O -mcmodel=small
CPP           = cpp -P
CPPFLAGS      = -DREAL8 -DMPI -DSERIAL_IO -DENDIAN_IO -DNAN2003 -DTIMER -DRELO $(CPP_EXTRAS)
LD            = $(FC)
LDFLAGS       = -V $(FCFFLAGS) -static-intel
EXTRALIBS     =

#
# --- generic make definitions
#
SHELL         = /bin/sh
RM            = \rm -f

#
# rules.
#

.c.o:
        $(CC) $(CPPFLAGS) $(CCFLAGS)  -c $*.c

.F90.o:
        $(FC) $(CPPFLAGS) $(FCFFLAGS) -c $*.F90

error

run type one

error in blkinr - input hybmap but should be hybthn

https://github.com/HYCOM/HYCOM-src/commit/8d247021484c5cb7f8ded897c1e2020c028ef7e9 edit GLBt0.72/expt_02.0/blkdat.input

1.0    'hybthn' = HYBGEN: ratio of layer thicknesses to select the thiner
3      'hybmap' = HYBGEN: remapper  flag (0=PCM, 1=PLM,  2=PPM, 3=WENO-like)

error: neg. dep(m) < -10.00

method 1: add dp00x in GLBt0.72/expt_02.0/blkdat.input 100.00 'dp00x ' = deep z-level spacing maximum thickness (m) => error in blkinr - input dp00x but should be isotop

method 2: change isotop in GLBt0.72/expt_02.0/blkdat.input 83.0 'isotop' = shallowest depth for isopycnal layers (m), <0 from file => 100.0 'isotop' = shallowest depth for isopycnal layers (m), <0 from file => error: neg. dep(m) < -10.00

method 3: change sefold in GLBt0.72/expt_02.0/blkdat.input -365.0 'sefold' = e-folding time for SSS relaxation (days) => 0.3 'sefold' = e-folding time for SSS relaxation (days) => error: neg. dp (m) < -10.00

method 4: build hycom with amd-intel-relo_one(copy FCFFLAGS, CCFLAGS, CPPFLAGS from HYCON-tools/config/amdIF_setup) => error: neg. dp (m) < -10.00

run type mpi

xcspmd: patch.input for wrong nreg

ATTN: Alan Wallcraft

change the src_2.3.01-17tsig2_relo_mpi/Make.csh

setenv OCN_EOS -DEOS_9T  ## EOS  9-term
#setenv OCN_EOS -DEOS_17T ## EOS 17-term
setenv OCN_GLB -DARCTIC ## global tripolar simulation
#setenv OCN_GLB ""

=> xcspmd: patch.input for wrong nreg

error in rdmonth - hit end of input

https://groups.google.com/a/hycom.org/g/forum/c/KWAPIMZFk7U/m/T8ahkU1fEQAJ

blkdat.input

0      'sshflg' = diagnostic SSH flag (0=SSH,1=SSH&stericSSH,2=SSH&stericMONTG)

xcspmd: patch.input must be for arctic,

https://groups.google.com/a/hycom.org/g/forum/c/ywUlB7pSalE/m/2OsN9j5JBQAJ

forrtl: severe (24): end-of-file during read, unit 2911, file /home/sanya/hycom/SCSp0.08/expt_01.0/data/SCRATCH/./relax.temp.b.

https://groups.google.com/a/hycom.org/g/forum/c/KWA19-48uHg/m/_MYGMGImBAAJ check the datasets/relax if correct soft link

cd datasets/relax/ && unlink 020 && ln -s 010 020

error in zaiopf - can't open unit 11, for array I/O. cfile = restart_in.a

https://groups.google.com/a/hycom.org/g/forum/c/ccOVIBW2LnM/m/J7S41TCWBgAJ Is this the first month of the run? If so, the limits file should have a negative (or zero) 1st entry and it is typically from a file like 011y011a.limits. This signals that the initial state is from climatology, not from a restart file.

C
C --- local input files.
C
if ( -e /home/smci/HYCOM-examples/GLBt0.72/expt_02.0/data/../020y011a.limits ) then
/bin/cp /home/smci/HYCOM-examples/GLBt0.72/expt_02.0/data/../020y011a.limits limits
else

cat GLBt0.72/expt_02.0/data/limits
-3653.00000  3684.00000

C
C --- local input files.
C
if ( -e /home/smci/HYCOM-examples/GOMb0.08/expt_02.0/data/../020y011a.limits ) then
echo LIMITS
awk -f /home/smci/HYCOM-examples/GOMb0.08/expt_02.0/data/../020.awk y01=011 ab=a
endif

cat GOMb0.08/expt_02.0/data/limits
     3653.00000     3684.00000
C

cd GOMb0.08/expt_02.0 && touch 020y011a.limits && echo "-3653.00000  3684.00000" > 020y011a.limits

AMD CPU with intel compiler

HYCOM-tools/Make_all.src

setenv ARCH amdIF

HYCOM-tools/Make_ncdf.src

module unload netcdf-fortran-4.6.0-intel-2021.8.0-5hpf7gh
module unload netcdf-c-4.9.0-intel-2021.8.0-etqveet
module   load netcdf-c-4.9.0-intel-2021.8.0-etqveet
module   load netcdf-fortran-4.6.0-intel-2021.8.0-5hpf7gh
setenv NCDFC     /home/smci/spack/opt/spack/linux-rhel8-zen2/intel-2021.8.0/netcdf-c-4.9.0-etqveetgf6tzouaveywcqkxgnqb6ef47
setenv NCDF      /home/smci/spack/opt/spack/linux-rhel8-zen2/intel-2021.8.0/netcdf-fortran-4.6.0-5hpf7gh76cytq6q7ob5rymcay3gj6qu3

HYCOM-tools/bin/Make_all.csh HYCOM-tools/bin/Make_ncdf.csh

if ($OS == "Linux") then
 setenv OS LinuxAIF

hycom binary from openmpi

HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi/Make.csh

unset echo
module purge
module load intel-oneapi-compilers-2023.0.0-gcc-8.5.0-7ccmh6l
module load openmpi-4.1.5-intel-2021.8.0-v2iv375
module list
set echo
setenv ARCH intelsse-impi-sm-relo

HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi/config/intelsse-impi-sm-relo_mpi

FC            = mpifort
#FCFFLAGS      =        -traceback -xSSE4.2 -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small
FCFFLAGS      = -traceback  -O3 -fp-model source -r8 -warn nogeneral -mcmodel=small
CC            = icc
#CCFLAGS       =        -traceback -xSSE4.2 -O -mcmodel=small
CCFLAGS       = -traceback -O -mcmodel=small
CPP           = cpp -P
CPPFLAGS      = -DREAL8 -DMPI -DSERIAL_IO -DENDIAN_IO -DNAN2003 -DTIMER -DRELO $(CPP_EXTRAS)
LD            = $(FC)
LDFLAGS       = -V $(FCFFLAGS) -static-intel
EXTRALIBS     =

#
# --- generic make definitions
#
SHELL         = /bin/sh
RM            = \rm -f

#
# rules.
#

.c.o:
        $(CC) $(CPPFLAGS) $(CCFLAGS)  -c $*.c

.F90.o:
        $(FC) $(CPPFLAGS) $(FCFFLAGS) -c $*.F90

AMD CPU with GNU compiler

HYCOM-tools/Make_all.src

setenv ARCH amdIF

HYCOM-tools/Make_ncdf.src

module unload netcdf-fortran-4.6.0-gcc-8.5.0-inxalsx
module unload netcdf-c-4.9.0-gcc-8.5.0-2qwncum
module   load netcdf-c-4.9.0-gcc-8.5.0-2qwncum
module   load netcdf-fortran-4.6.0-gcc-8.5.0-inxalsx
setenv NCDFC     /home/smci/spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/netcdf-c-4.9.0-2qwncumfartt3qrjqpixzxgipxs3hgrq
setenv NCDF      /home/smci/spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/netcdf-fortran-4.6.0-inxalsxzrfxb4q4nposghyu44gpw4vuy

HYCOM-tools/bin/Make_all.csh HYCOM-tools/bin/Make_ncdf.csh

if ($OS == "Linux") then
 setenv OS LinuxGF

hycom binary from openmpi

HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi/Make.csh

unset echo
module purge
module load openmpi-4.1.5-gcc-8.5.0-uqgr6vy
module list
set echo
setenv ARCH aepyc-gnu-relo

HYCOM-examples/GLBt0.72/src_2.3.01-17tsig2_relo_mpi/config/aepyc-gnu-relo_mpi

FC            = mpif90
FCFFLAGS      = -fPIC -fno-second-underscore -O2 -mavx -m64 -fdefault-real-8 -fdefault-double-8
CC            = mpicc
CCFLAGS       = -O -m64
CPP           = cpp -P
CPPFLAGS      = -DREAL8 -DMPI -DSERIAL_IO -DENDIAN_IO -DNAN2003 -DTIMER -DRELO $(CPP_EXTRAS)
LD            = $(FC)
LDFLAGS       = $(FCFFLAGS)
EXTRALIBS     =

#
# --- generic make definitions
#
SHELL         = /bin/sh
RM            = \rm -f

#
# rules.
#

.c.o:
        $(CC) $(CPPFLAGS) $(CCFLAGS)  -c $*.c

.F90.o:
        $(FC) $(CPPFLAGS) $(FCFFLAGS) -c $*.F90

results

Note: The following floating-point exceptions are signalling: IEEE_INVALID_FLAG IEEE_UNDERFLOW_FLAG IEEE_DENORMAL STOP (xcstop)