CMAQ v5.3 compiling error

CMAQ Compiling
1

Hi, all!

I’m trying to compile CMAQ v5.3.1 with Intel19 compiler.
Compiling with nvidia 21.3 compiler was successful to get CCTMv531.exe.
But I got errors during compiling with intel19 compiler.

The error messages are :
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(name2fid.o): In function name2fid_': name2fid.f:(.text+0x21): undefined reference to __kmpc_global_thread_num’
name2fid.f:(.text+0x38): undefined reference to __kmpc_critical' name2fid.f:(.text+0x59): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(getefile.o): In function getefile_': getefile.F:(.text+0x37): undefined reference to __kmpc_global_thread_num’
getefile.F:(.text+0x50): undefined reference to __kmpc_critical' getefile.F:(.text+0x74): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): In function m3msg2_': m3msg2.F:(.text+0x20): undefined reference to __kmpc_global_thread_num’
m3msg2.F:(.text+0x37): undefined reference to __kmpc_critical' m3msg2.F:(.text+0x9e): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): In function m3mesg_': m3msg2.F:(.text+0xed): undefined reference to __kmpc_global_thread_num’
m3msg2.F:(.text+0x104): undefined reference to __kmpc_critical' m3msg2.F:(.text+0x19d): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): In function m3prompt_': m3msg2.F:(.text+0x202): undefined reference to __kmpc_global_thread_num’
m3msg2.F:(.text+0x219): undefined reference to __kmpc_critical' m3msg2.F:(.text+0x31b): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): In function m3parag_': m3msg2.F:(.text+0x463): undefined reference to __kmpc_global_thread_num’
m3msg2.F:(.text+0x47f): undefined reference to __kmpc_critical' m3msg2.F:(.text+0x555): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): In function m3flush_': m3msg2.F:(.text+0x5a7): undefined reference to __kmpc_global_thread_num’
m3msg2.F:(.text+0x5be): undefined reference to __kmpc_critical' m3msg2.F:(.text+0x5e3): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(write3.o): In function write3_': write3.F:(.text+0x7d): undefined reference to __kmpc_global_thread_num’
write3.F:(.text+0x96): undefined reference to __kmpc_critical' write3.F:(.text+0xba): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(close3.o): In function close3_': close3.F90:(.text+0x8e): undefined reference to __kmpc_global_thread_num’
close3.F90:(.text+0xa5): undefined reference to __kmpc_critical' close3.F90:(.text+0x1f2): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(desc3.o): In function desc3_': desc3.F90:(.text+0x24): undefined reference to __kmpc_global_thread_num’
desc3.F90:(.text+0x36): undefined reference to __kmpc_single' desc3.F90:(.text+0x2b7): undefined reference to __kmpc_critical’
desc3.F90:(.text+0x1168): undefined reference to __kmpc_end_critical' desc3.F90:(.text+0x1250): undefined reference to _kmpc_end_single’
desc3.F90:(.text+0x125f): undefined reference to __kmpc_barrier' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(init3.o): In function init3’:
init3.F90:(.text+0x77): undefined reference to __kmpc_global_thread_num' init3.F90:(.text+0x92): undefined reference to _kmpc_critical’
init3.F90:(.text+0xea): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(open3.o): In function open3’:
open3.F90:(.text+0x32): undefined reference to __kmpc_global_thread_num' open3.F90:(.text+0x4c): undefined reference to _kmpc_critical’
open3.F90:(.text+0x75): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(opnfil3.o): In function opnfil3’:
opnfil3.F90:(.text+0x84): undefined reference to __kmpc_global_thread_num' opnfil3.F90:(.text+0x9a): undefined reference to __kmpc_critical’
opnfil3.F90:(.text+0x940): undefined reference to __kmpc_end_critical' opnfil3.F90:(.text+0x95f): undefined reference to _kmpc_critical’
opnfil3.F90:(.text+0x989): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(opnlog3.o): In function opnlog3’:
opnlog3.F90:(.text+0x51): undefined reference to __kmpc_global_thread_num' opnlog3.F90:(.text+0x67): undefined reference to __kmpc_critical’
opnlog3.F90:(.text+0xa8): undefined reference to __kmpc_critical' opnlog3.F90:(.text+0xe8): undefined reference to _kmpc_end_critical’
opnlog3.F90:(.text+0x1216): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdgrnest.o): In function rdgrnest’:
rdgrnest.F90:(.text+0x56): undefined reference to __kmpc_global_thread_num' rdgrnest.F90:(.text+0x6f): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0xa9): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x1df): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0x21e): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x336): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0x37d): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x4a1): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0x4e8): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x60b): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0x652): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x775): undefined reference to __kmpc_critical’
rdgrnest.F90:(.text+0x7bc): undefined reference to __kmpc_end_critical' rdgrnest.F90:(.text+0x8df): undefined reference to _kmpc_critical’
rdgrnest.F90:(.text+0x926): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdprofil.o): In function rdprofil’:
rdprofil.F90:(.text+0x4d): undefined reference to __kmpc_global_thread_num' rdprofil.F90:(.text+0x66): undefined reference to __kmpc_critical’
rdprofil.F90:(.text+0xa1): undefined reference to __kmpc_end_critical' rdprofil.F90:(.text+0x1d7): undefined reference to __kmpc_critical’
rdprofil.F90:(.text+0x217): undefined reference to __kmpc_end_critical' rdprofil.F90:(.text+0x32f): undefined reference to __kmpc_critical’
rdprofil.F90:(.text+0x377): undefined reference to __kmpc_end_critical' rdprofil.F90:(.text+0x49b): undefined reference to __kmpc_critical’
rdprofil.F90:(.text+0x4e3): undefined reference to __kmpc_end_critical' rdprofil.F90:(.text+0x606): undefined reference to __kmpc_critical’
rdprofil.F90:(.text+0x64e): undefined reference to __kmpc_end_critical' rdprofil.F90:(.text+0x771): undefined reference to _kmpc_critical’
rdprofil.F90:(.text+0x7b9): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdtflag.o): In function rdtflag’:
rdtflag.F90:(.text+0x5e): undefined reference to __kmpc_global_thread_num' rdtflag.F90:(.text+0x2d9): undefined reference to __kmpc_critical’
rdtflag.F90:(.text+0x323): undefined reference to __kmpc_end_critical' rdtflag.F90:(.text+0x6b5): undefined reference to _kmpc_critical’
rdtflag.F90:(.text+0x6e6): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdvars.o): In function rdvars’:
rdvars.F90:(.text+0xb5): undefined reference to __kmpc_global_thread_num' rdvars.F90:(.text+0x122): undefined reference to __kmpc_critical’
rdvars.F90:(.text+0x1af): undefined reference to __kmpc_end_critical' rdvars.F90:(.text+0x1f1): undefined reference to __kmpc_end_critical’
rdvars.F90:(.text+0x3bf): undefined reference to __kmpc_critical' rdvars.F90:(.text+0x403): undefined reference to __kmpc_end_critical’
rdvars.F90:(.text+0x4cc): undefined reference to __kmpc_end_critical' rdvars.F90:(.text+0x7a3): undefined reference to _kmpc_critical’
rdvars.F90:(.text+0x839): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(shut3.o): In function shut3’:
shut3.F90:(.text+0x38): undefined reference to __kmpc_global_thread_num' shut3.F90:(.text+0x4e): undefined reference to _kmpc_critical’
shut3.F90:(.text+0x469): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(syncfid.o): In function syncfid’:
syncfid.F90:(.text+0x19): undefined reference to __kmpc_global_thread_num' syncfid.F90:(.text+0x5a): undefined reference to __kmpc_critical’
syncfid.F90:(.text+0x83): undefined reference to __kmpc_end_critical' syncfid.F90:(.text+0x1f4): undefined reference to _kmpc_critical’
syncfid.F90:(.text+0x2ad): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrgrnest.o): In function wrgrnest’:
wrgrnest.F90:(.text+0x2c): undefined reference to __kmpc_global_thread_num' wrgrnest.F90:(.text+0x42): undefined reference to __kmpc_critical’
wrgrnest.F90:(.text+0x56): undefined reference to __kmpc_critical' wrgrnest.F90:(.text+0xbd7): undefined reference to _kmpc_end_critical’
wrgrnest.F90:(.text+0xbeb): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrprofil.o): In function wrprofil’:
wrprofil.F90:(.text+0x3a): undefined reference to __kmpc_global_thread_num' wrprofil.F90:(.text+0x50): undefined reference to __kmpc_critical’
wrprofil.F90:(.text+0x64): undefined reference to __kmpc_critical' wrprofil.F90:(.text+0x834): undefined reference to _kmpc_end_critical’
wrprofil.F90:(.text+0x848): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrvars.o): In function wrvars’:
wrvars.F90:(.text+0x2e): undefined reference to __kmpc_global_thread_num' wrvars.F90:(.text+0x90): undefined reference to __kmpc_critical’
wrvars.F90:(.text+0x128): undefined reference to __kmpc_end_critical' wrvars.F90:(.text+0x20c): undefined reference to __kmpc_critical’
wrvars.F90:(.text+0x250): undefined reference to __kmpc_end_critical' wrvars.F90:(.text+0x480): undefined reference to __kmpc_critical’
wrvars.F90:(.text+0x4d1): undefined reference to __kmpc_end_critical' wrvars.F90:(.text+0x4ee): undefined reference to _kmpc_critical’
wrvars.F90:(.text+0x7e2): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(crtbuf3.o): In function crtbuf3’:
crtbuf3.f:(.text+0x91): undefined reference to __kmpc_global_thread_num' crtbuf3.f:(.text+0xa8): undefined reference to _kmpc_critical’
crtbuf3.f:(.text+0x19e): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(chkfil3.o): In function chkfil3’:
chkfil3.f90:(.text+0x1d): undefined reference to __kmpc_global_thread_num' chkfil3.f90:(.text+0x36): undefined reference to _kmpc_critical’
chkfil3.f90:(.text+0x2537): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rddict3.o): In function rddict3’:
rddict3.f90:(.text+0x81): undefined reference to __kmpc_global_thread_num' rddict3.f90:(.text+0x98): undefined reference to __kmpc_critical’
rddict3.f90:(.text+0xac): undefined reference to __kmpc_critical' rddict3.f90:(.text+0x9d6): undefined reference to _kmpc_end_critical’
rddict3.f90:(.text+0x9ea): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdiddata.o): In function rdiddata’:
rdiddata.f90:(.text+0x4d): undefined reference to __kmpc_global_thread_num' rdiddata.f90:(.text+0x66): undefined reference to __kmpc_critical’
rdiddata.f90:(.text+0xa1): undefined reference to __kmpc_end_critical' rdiddata.f90:(.text+0x1ce): undefined reference to _kmpc_critical’
rdiddata.f90:(.text+0x20e): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdsmatrx.o): In function rdsmatrx’:
rdsmatrx.f90:(.text+0x6f): undefined reference to __kmpc_global_thread_num' rdsmatrx.f90:(.text+0x84): undefined reference to __kmpc_critical’
rdsmatrx.f90:(.text+0xc0): undefined reference to __kmpc_end_critical' rdsmatrx.f90:(.text+0x215): undefined reference to _kmpc_critical’
rdsmatrx.f90:(.text+0x259): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrdict3.o): In function wrdict3’:
wrdict3.f90:(.text+0x41): undefined reference to __kmpc_global_thread_num' wrdict3.f90:(.text+0x57): undefined reference to __kmpc_critical’
wrdict3.f90:(.text+0x6b): undefined reference to __kmpc_critical' wrdict3.f90:(.text+0x987): undefined reference to _kmpc_end_critical’
wrdict3.f90:(.text+0x99b): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wriddata.o): In function wriddata’:
wriddata.f90:(.text+0x3a): undefined reference to __kmpc_global_thread_num' wriddata.f90:(.text+0x50): undefined reference to __kmpc_critical’
wriddata.f90:(.text+0x64): undefined reference to __kmpc_critical' wriddata.f90:(.text+0x309): undefined reference to _kmpc_end_critical’
wriddata.f90:(.text+0x31d): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrsmatrx.o): In function wrsmatrx’:
wrsmatrx.f90:(.text+0x55): undefined reference to __kmpc_global_thread_num' wrsmatrx.f90:(.text+0x6b): undefined reference to __kmpc_critical’
wrsmatrx.f90:(.text+0x7f): undefined reference to __kmpc_critical' wrsmatrx.f90:(.text+0x320): undefined reference to _kmpc_end_critical’
wrsmatrx.f90:(.text+0x334): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(crtfil3.o): In function crtfil3’:
crtfil3.F90:(.text+0x4a): undefined reference to __kmpc_global_thread_num' crtfil3.F90:(.text+0x65): undefined reference to __kmpc_critical’
crtfil3.F90:(.text+0x84): undefined reference to __kmpc_end_critical' crtfil3.F90:(.text+0xab): undefined reference to __kmpc_critical’
crtfil3.F90:(.text+0x2018): undefined reference to __kmpc_end_critical' crtfil3.F90:(.text+0x2035): undefined reference to _kmpc_critical’
crtfil3.F90:(.text+0x205f): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wrtflag.o): In function wrtflag’:
wrtflag.F90:(.text+0x45): undefined reference to __kmpc_global_thread_num' wrtflag.F90:(.text+0xac): undefined reference to __kmpc_critical’
wrtflag.F90:(.text+0xf2): undefined reference to __kmpc_end_critical' wrtflag.F90:(.text+0x578): undefined reference to _kmpc_critical’
wrtflag.F90:(.text+0x5ac): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdatt3.o): In function rdattc’:
rdatt3.F90:(.text+0x53): undefined reference to __kmpc_global_thread_num' rdatt3.F90:(.text+0x34e): undefined reference to _kmpc_critical’
rdatt3.F90:(.text+0x49f): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(rdatt3.o): In function rdatt3’:
rdatt3.F90:(.text+0x520): undefined reference to __kmpc_global_thread_num' rdatt3.F90:(.text+0xb86): undefined reference to __kmpc_end_critical’
/home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wratt3.o): In function wrattc_': wratt3.F90:(.text+0x46): undefined reference to __kmpc_global_thread_num’
wratt3.F90:(.text+0x3bb): undefined reference to __kmpc_critical' wratt3.F90:(.text+0x8ee): undefined reference to _kmpc_end_critical’
wratt3.F90:(.text+0x91a): undefined reference to __kmpc_end_critical' /home/cmaqv5.3.1/lib/x86_64/intel/ioapi/lib/libioapi.a(wratt3.o): In function wratt3’:
wratt3.F90:(.text+0xab4): undefined reference to `__kmpc_global_thread_num’
make: *** [CCTM_v531.exe] Error 1
ERROR while running make command

I think this is due to my config_cmaq.csh

My config_cmaq.csh is :
case intel:

    #> I/O API, netCDF, and MPI library locations
    setenv IOAPI_INCL_DIR   /usr/local/ioapi/3.2/ioapi   #> I/O API include header files
    setenv IOAPI_LIB_DIR    /usr/local/ioapi/3.2/Linux2_x86_64ifort    #> I/O API libraries
    setenv NETCDF_LIB_DIR   /usr/local/netcdf/4.1.3_intel19/lib   #> netCDF C directory path
    setenv NETCDF_INCL_DIR  /usr/local/netcdf/4.1.3_intel19/include   #> netCDF C directory path
    setenv NETCDFF_LIB_DIR  /usr/local/netcdf/4.1.3_intel19/lib  #> netCDF Fortran directory path
    setenv NETCDFF_INCL_DIR /usr/local/netcdf/4.1.3_intel19/include  #> netCDF Fortran directory path
    setenv MPI_LIB_DIR      /usr/local/mpi/intel19/mpich-3.2      #> MPI directory path

    #> Compiler Aliases and Flags
    #> set the compiler flag -qopt-report=5 to get a model optimization report in the build directory with the optrpt extension
    setenv myFC mpiifort
    setenv myCC icc
    setenv myFSTD "-O3 -fno-alias -mp1 -fp-model source -ftz -simd -align all -xHost -vec-guard-write -unroll-aggressive"
    setenv myDBG  "-O0 -g -check bounds -check uninit -fpe0 -fno-alias -ftrapuv -traceback"

setenv myLINK_FLAG #"-qopenmp-simd" openMP not supported w/ CMAQ

    setenv myLINK_FLAG "-qopenmp-simd"
    setenv myFFLAGS "-fixed -132"
    setenv myFRFLAGS "-free"
    setenv myCFLAGS "-O2"
    setenv extra_lib "-lcurl"
    #setenv extra_lib ""
    setenv mpi_lib ""    #> No Library specification needed for mpiifort
                         #> -lmpich for mvapich
                         #> -lmpi for openmpi

    breaksw

What should I revise in config_cmaq.csh??
If this is not the reason, how I can solve this compiling problem ??

Please assist me with any responses.

Thank you.

HyeonYeong Park

2

“-qopenmp-simd” is an nvidia-specific flag.

Your myLINK_FLAG needs to use what is inside the OMPLIBS in the Makeinclude.$BIN that you used to build the I/O API library – in this case, “-qopenmp”

4

Thank you for your replying!

I compiled again using config_cmaq.csh with
’ setenv myLINK_FLAG #"-qopenmp-simd" openMP not supported w/ CMAQ ’ that is default state.
And the compiling was failed.

But I compiled using config_cmaq.csh with
’ setenv myLINK_FLAG “-qopenmp” ’ you mentioned, the compiling was succeeded!

Thank you for your replying again!! :slight_smile:

So, I want to know the reason of these problem and revision of ‘setenv myLINK_FLAG’ when I used intel compiler. Because I didn’t experience these problem and revision when I used nvidia compiler.

If you know that reason, please let me know that!!

I appreciate your help again! :slight_smile:

Thank you.

HyeonYeong Park

5

Different compilers use different flags for the achieving the same things (and indeed sometimes even different versions of the same compiler do – which is a nightmare for model-support!)

I/O API compile-flags are kept in the Makeinclude.$BIN files where you can get at them and change them as necessary (Intel ifort versions 15 and before using “-openmp” whereas versions 16 and later use “-qopenmp”, for example).

The CMAQ build system tries to “hide away” all of that and requires a bit more subtlety on the part of the user to adapt to changes by vendors, etc…

7

I understand what you mean.

Your help is really helpful to me!

But I have one more question about this problem.
When I used nvidia version 21.3 compiler, there was no requirement to revise ‘setenv my LINK_FLAG’.
Does revising and turning on ‘setenv my LINK_FLAG’ depend on kind of compiler??