CCTM compilation

Hi all

I am trying to compile CCTM but an error comes up:

mpiifort -qopenmp se_bndy_copy_info_ext.o se_pe_info_ext.o se_comm_info_ext.o se_data_send_module.o se_domain_info_ext.o se_internal_util_module.o se_data_recv_module.o se_disp_info_ext.o se_subgrid_info_ext.o se_global_sum_module.o se_global_max_module.o se_global_min_module.o se_slice_module.o se_init_module.o se_bndy_copy_module.o se_comm_module.o se_reconfig_grid_info_ext.o se_reconfig_grid_module.o se_gather_module.o se_data_copy_module.o se_term_module.o se_twoway_comm_module.o se_util_module.o se_modules.o swap_sandia.o swap_sandia_routines.o alloc_data_mod.o pinterpb_mod.o parutilio.o piomaps_mod.o boundary.o get_write_map.o growbuf.o gtndxhdv.o interpol.o pinterpb.o pio_init.o pm3err.o pm3exit.o pm3warn.o pshut3.o ptrwrite3.o pwrgrdd.o pwrite3.o readbndy.o subdmap.o wrsubdmap.o UTILIO_DEFN.o HGRD_DEFN.o VGRD_DEFN.o GRID_CONF.o CGRID_SPCS.o PA_DEFN.o PCGRID_DEFN.o AVG_CONC.o DEPVVARS.o BIDI_MOD.o LSM_MOD.o ASX_DATA_MOD.o HGSIM.o MOSAIC_MOD.o RXNS_DATA_MODULE.o BIOG_EMIS.o BEIS_DEFN.o LUS_DEFN.o LTNG_DEFN.o AEROMET_DATA.o AERO_DATA.o UDTYPES.o STK_EMIS.o STK_PRMS.o PTMAP.o PTBILIN.o SSEMIS.o bmatvec.o PTMET.o DEPV_DEFN.o STD_CONC.o CSQY_DATA.o CLOUD_OPTICS.o PHOT_MET_DATA.o DUST_EMIS.o RXNS_FUNC_MODULE.o PRECURSOR_DATA.o MGEMIS.o PT3D_DEFN.o SOA_DEFN.o AERO_EMIS.o AEROSOL_CHEMISTRY.o PAGRD_DEFN.o EMIS_DEFN.o WVEL_DEFN.o distr_env.o mpcomm_init.o advstep.o driver.o hveloc.o sciproc.o wr_aconc.o wr_cgrid.o wr_conc.o wr_init.o diffmsg.o flcheck.o grdcheck.o initscen.o load_cgrid.o opaconc.o opconc.o couplewrf.o advbc_map.o hadvyppm.o hcontvel.o hppm.o rdbcon.o x_ppm.o x_yamo.o y_ppm.o y_yamo.o zfdbc.o vppm.o zadvppmwrf.o deform.o hcdiff3d.o hdiff.o rho_j.o VDIFF_DIAG.o VDIFF_MAP.o SEDIMENTATION.o aero_sedv.o conv_cgrid.o eddyx.o matrix1.o opddep.o opddep_fst.o opddep_mos.o rddepv.o tri.o vdiffacmx.o vdiffproc.o ABFLUX_MOD.o cgrid_depv.o gas_depv_map.o m3dry.o opdepv_diag.o opdepv_fst.o opdepv_mos.o cropcal.o opemis.o tfabove.o tfbelow.o beis3.o checkmem.o chkgrid.o czangle.o getfline.o getparb.o hrno.o parsline.o tmpbeis.o wrdaymsg.o delta_zs.o fire_plmris.o openlayout.o oppt3d_diag.o plmris.o plsprd.o preplm.o ungridb2.o write3_distr.o complex_number_module.o twoway_rrtmg_aero_optics.o AERO_PHOTDATA.o PHOT_MOD.o SEAS_STRAT_O3_MIN.o PHOTOLYSIS_ALBEDO.o o3totcol.o opphot.o phot.o DEGRADE_SETUP_TOX.o hrdata_mod.o degrade_data.o degrade.o final_degrade.o find_degraded.o hrdriver.o hrg1.o hrg2.o hrg3.o hrg4.o hrinit.o hrprodloss.o hrrates.o hrsolver.o init_degrade.o AOD_DEFN.o PMDIAG_DATA.o aero_depv.o aero_driver.o aero_subs.o coags.o getpar.o isocom.o isofwd.o isorev.o opapmdiag.o opavis.o oppmdiag.o opvis.o AQ_DATA.o acmcld.o aqchem.o aq_map.o cldproc_acm.o convcld_acm.o getalpha.o hlconst.o indexn.o opwdep.o rescld.o scavwdep.o PA_GLOBAL.o PA_IPRDEF.o PA_VARS.o PA_PARSE.o PA_IPRVARS.o pa_compmech.o pa_datagen.o pa_errcheck.o pa_getcoef.o pa_getcycle.o pa_getdesc.o pa_getfamily.o pa_getiprout.o pa_getirrout.o pa_getrxns.o pa_getrxnsum.o pa_init.o pa_irr_ctl.o pa_irr.o pa_mkhdr.o pa_molcloss.o pa_molcprod.o pa_output.o pa_read.o pa_report.o pa_setup_ipr.o pa_setup_irr.o pa_update.o pa_wrtpadefn.o cksummer.o findex.o get_envlist.o lstepf.o setup_logdev.o subhdomain.o subhfile.o -L/media/maria/Backup/CMAQ_Project/PREP/mcip/src/lib/x86_64/intel/ioapi/lib -lioapi -L/media/maria/Backup/CMAQ_Project/PREP/mcip/src/lib/x86_64/intel/netcdf/lib -lnetcdff -lnetcdf -L/media/maria/Backup/CMAQ_Project/PREP/mcip/src/lib/x86_64/intel/mpi/lib -lcurl -o CCTM_v52.exe
/media/maria/Backup/intel/compilers_and_libraries_2019.3.199/linux/mpi/intel64/lib/libmpifort.so: undefined reference to `PMPI_Aint_diff’

I am not a specialist about compilers or programming. I was searching on Internet a possible solution, however I didn’t find anything about it.

Do you have any idea?
Appreciate your help

Maria

Hi Maria,

It may be something as simple of adding -lmpi to your config_cmaq.csh

config_cmaq.csh: setenv mpi_lib “-lmpi”

If this doesn’t work, please send the following information:
If you are using modules on a compute server, please use the module list command to see what compiler is loaded:

module list 

If not, use the command which mpiifort to reveal what compiler you are using:

which mpiifort

Thank you for your reply.
I applied the change that you suggested in my config_cmaq.csh, but it didn’t work.

When I typed “which mpiifort” the result is:

/media/maria/Backup/intel/compilers_and_libraries_2019.3.199/linux/mpi/intel64/bin/mpiifort

Then, I decided to change the path in:
config_cmaq.csh: setenv MPI_LIB_DIR

But a new error comes up:
distr_env.c:(.text+0x156): undefined reference to `ompi_mpi_int’

So, I suppose that the change is not in the correct environment variable. Do I have to change the path in other place?

Maria

Hi Maria,

Please try adding the following environment variable to your config_cmaq.csh
under the section
case intel:

setenv myLINK_FLAG "-qopenmp"

Liz

Hi Liz

In my current config_cmaq.csh file, I have that environment variable but the error persists:

setenv IOAPI_MOD_DIR /media/maria/Backup/IOAPI3.1/Linux2_x86_64ifort/ #> I/O API precompiled modules
setenv IOAPI_INCL_DIR /media/maria/Backup/IOAPI3.1/Linux2_x86_64ifort/ #> I/O API include header files
setenv IOAPI_LIB_DIR /media/maria/Backup/IOAPI3.1/Linux2_x86_64ifort/ #> I/O API libraries
setenv NETCDF_LIB_DIR /media/maria/Backup/netcdf-4.1.3/netcdf/lib/ #> netCDF directory path
setenv NETCDF_INCL_DIR /media/maria/Backup/netcdf-4.1.3/netcdf/include/ #> netCDF directory path
setenv MPI_LIB_DIR /home/maria/Escritorio/openmpi-3.1.1/ #> MPI directory path

    #> Compiler Aliases and Flags
    setenv myFC mpiifort
    setenv myCC icc       
    setenv myFSTD "-O3 -fno-alias -mp1 -fp-model source"
    setenv myDBG  "-O0 -g -check bounds -check uninit -fpe0 -fno-alias -ftrapuv -traceback"
    setenv myLINK_FLAG "-qopenmp"
    setenv myFFLAGS "-fixed -132"
    setenv myFRFLAGS "-free"
    setenv myCFLAGS "-O2"
    setenv extra_lib "-lcurl"
    #setenv extra_lib ""
    setenv mpi_lib "" 

I really appreciate your help

Maria

@lizadams Hello

Please, below is the response I got while trying to configure CMAQ:

ERROR: /home/catalyst/Desktop/Build_WRF/CMAQ-5.2.1/lib/x86_64/gcc/netcdf/lib/libnetcdf.a does not exist in your CMAQ_LIB directorysource config_cmaq.csh gcc! Check your installation before proceeding with CMAQ build.

Also, while compiling ICON and BCON:
rm: No match.
mpif90: Command not found.
mpif90: Command not found.
mpif90: Command not found.
mpif90: Command not found.
mpif90: No match.

ERROR BLDMAKE Compile failed

Kindly help, please.

Please use the following instructions to install the required software.

Thanks for your response. But the problems still persist after installing netcdf-c-4.6.3 and netcdf-fortran-4.4.4 following your instructions.
The I/O API Version 3.2 couldn’t be installed on my system too. My system’s linux version is ‘‘x86_64-linux-gnu’’ and is not listed among the versions compatible with the I/0 API.

source config_cmaq.csh gcc 7.3.0
Compiler is set to gcc
ERROR: /home/catalyst/Desktop/Build_WRF/CMAQ-5.2.1/lib/x86_64/gcc7.3.0/netcdf/lib/libnetcdf.a does not exist in your CMAQ_LIB directorysource config_cmaq.csh gcc 7.3.0! Check your installation before proceeding with CMAQ build.

I’ll appreciate your further help. Thanks.

Below is what I got:

Makefile:95: home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/ioapi/Makeinclude.Linux2_x86_64gfort: No such file or directory
make: *** No rule to make target ‘home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/ioapi/Makeinclude.Linux2_x86_64gfort’. Stop.

Where did you get this ioapi-3.2/ioapi/ ? Going to the CMAS web-server, I see:
longleaf:download % tar tfz ioapi-3.2.tar.gz | grep gfort
ioapi/Makeinclude.Linux2_x86_64gfort_360
ioapi/Makeinclude.Linux2_x86_64gfort
ioapi/Makeinclude.WIN_x86_64gfort_medium
ioapi/Makeinclude.Linux2_ia64gfort
ioapi/Makeinclude.Linux2_x86_64gfort_365
ioapi/Makeinclude.Linux2_x86_64gfortmpi
ioapi/Makeinclude.OSX_x86_gfortran
ioapi/Makeinclude.Linux2_x86_64gfort_medium
ioapi/Makeinclude.Linux2_x86_64gfortdbg
ioapi/Makeinclude.WIN_x86gfort
ioapi/Makeinclude.WIN_x86_64gfort

and similarly on the GitHub server… You did follow the I/O API installation instructions https://cjcoats.github.io/ioapi/AVAIL.html#build didn’t you?

Hello,

Below is what I got while building CCTM:

gfortran -c -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/modules -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/include_files -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/mpi -I. pinterpb.f
STATE3.EXT:174:27:

Error: Syntax error in COMMON statement at (1)
STATE3.EXT:191:27:

Error: Syntax error in COMMON statement at (1)
Makefile:414: recipe for target ‘pinterpb.o’ failed
make: *** [pinterpb.o] Error 1
ERROR while running make command

else if ( 0 ) then
endif
mv Makefile Makefile.gcc7.3.0
if ( -e Makefile.gcc7.3.0 && -e Makefile ) rm Makefile
ln -s Makefile.gcc7.3.0 Makefile
if ( 0 != 0 ) then
if ( -e /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/CCTM/scripts/BLD_CCTM_v52_gcc7.3.0/CCTM_v52.cfg ) then
mv CCTM_v52.cfg.bld /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/CCTM/scripts/BLD_CCTM_v52_gcc7.3.0/CCTM_v52.cfg
exit

Please, help me out as usual. Thanks a lot.

See https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#cmaq

Still on CCTM, kindly assist me. Thanks.

driver.o: In function MAIN__': driver.F:(.text+0x11e3): undefined reference tompi_wtime_’
driver.F:(.text+0x1283): undefined reference to mpi_finalize_' se_data_send_module.o: In functionse_data_send_module_MOD_se_5d_data_send’:
se_data_send_module.f:(.text+0x4e7): undefined reference to mpi_send_' se_data_send_module.f:(.text+0x55b): undefined reference tompi_send

se_data_send_module.f:(.text+0x5c7): undefined reference to mpi_send_' se_data_send_module.o: In functionse_data_send_module_MOD_se_4de_data_send’:
se_data_send_module.f:(.text+0x76a): undefined reference to mpi_send_' se_data_send_module.o: In functionse_data_send_module_MOD_se_4d_data_send’:
se_data_send_module.f:(.text+0xe8b): undefined reference to mpi_send_' se_data_send_module.o:se_data_send_module.f:(.text+0xf03): more undefined references tompi_send
’ follow
se_internal_util_module.o: In function __se_internal_util_module_MOD_se_generate_map': se_internal_util_module.f:(.text+0xe45): undefined reference tompi_gather

se_internal_util_module.f:(.text+0xe78): undefined reference to mpi_bcast_' se_data_recv_module.o: In functionse_data_recv_module_MOD_se_5d_data_recv’:
se_data_recv_module.f:(.text+0x1ad): undefined reference to mpi_recv_' se_data_recv_module.o: In functionse_data_recv_module_MOD_se_4de_data_recv’:
se_data_recv_module.f:(.text+0x725): undefined reference to mpi_recv_' se_data_recv_module.f:(.text+0xa6b): undefined reference tompi_recv

se_data_recv_module.o: In function __se_data_recv_module_MOD_se_4d_data_recv': se_data_recv_module.f:(.text+0xc26): undefined reference tompi_recv

se_data_recv_module.o: In function __se_data_recv_module_MOD_se_3de_data_recv': se_data_recv_module.f:(.text+0x1077): undefined reference tompi_recv

se_data_recv_module.o:se_data_recv_module.f:(.text+0x1389): more undefined references to mpi_recv_' follow se_global_sum_module.o: In functionse_global_sum_module_MOD_se_global_rasum’:
se_global_sum_module.f:(.text+0xfb): undefined reference to mpi_reduce_' se_global_sum_module.f:(.text+0x2a1): undefined reference tompi_bcast

se_global_sum_module.o: In function __se_global_sum_module_MOD_se_global_iasum': se_global_sum_module.f:(.text+0x3dc): undefined reference tompi_reduce

se_global_sum_module.f:(.text+0x53d): undefined reference to mpi_bcast_' se_global_sum_module.o: In functionse_global_sum_module_MOD_se_global_rsum’:
se_global_sum_module.f:(.text+0x5b7): undefined reference to mpi_reduce_' se_global_sum_module.f:(.text+0x5fa): undefined reference tompi_bcast

se_global_sum_module.o: In function __se_global_sum_module_MOD_se_global_isum': se_global_sum_module.f:(.text+0x647): undefined reference tompi_reduce

se_global_sum_module.f:(.text+0x686): undefined reference to mpi_bcast_' se_global_max_module.o: In functionse_global_max_module_MOD_se_global_rmax’:
se_global_max_module.f:(.text+0x3a): undefined reference to mpi_reduce_' se_global_max_module.f:(.text+0x5f): undefined reference tompi_bcast

se_global_max_module.o: In function __se_global_max_module_MOD_se_global_imax': se_global_max_module.f:(.text+0xba): undefined reference tompi_reduce

se_global_max_module.f:(.text+0xdf): undefined reference to mpi_bcast_' se_global_min_module.o: In functionse_global_min_module_MOD_se_global_rmin’:
se_global_min_module.f:(.text+0x3e): undefined reference to mpi_reduce_' se_global_min_module.f:(.text+0x63): undefined reference tompi_bcast

se_global_min_module.o: In function __se_global_min_module_MOD_se_global_imin': se_global_min_module.f:(.text+0xbe): undefined reference tompi_reduce

se_global_min_module.f:(.text+0xe3): undefined reference to mpi_bcast_' se_slice_module.o: In functionse_slice_module_MOD_se_slice4r’:
se_slice_module.f:(.text+0x45c): undefined reference to mpi_recv_' se_slice_module.f:(.text+0xa2a): undefined reference tompi_send

se_slice_module.o: In function __se_slice_module_MOD_se_slice4i': se_slice_module.f:(.text+0xfac): undefined reference tompi_recv

se_slice_module.f:(.text+0x14d8): undefined reference to mpi_send_' se_slice_module.o: In functionse_slice_module_MOD_se_slice3r’:
se_slice_module.f:(.text+0x1950): undefined reference to mpi_recv_' se_slice_module.f:(.text+0x1e28): undefined reference tompi_send

se_slice_module.f:(.text+0x1ec6): undefined reference to mpi_recv_' se_slice_module.o: In functionse_slice_module_MOD_se_slice3i’:
se_slice_module.f:(.text+0x2270): undefined reference to mpi_recv_' se_slice_module.f:(.text+0x26a6): undefined reference tompi_send

se_slice_module.f:(.text+0x2744): undefined reference to mpi_recv_' se_slice_module.o: In functionse_slice_module_MOD_se_slice2r’:
se_slice_module.f:(.text+0x2a54): undefined reference to mpi_recv_' se_slice_module.f:(.text+0x2e2b): undefined reference tompi_send

se_slice_module.f:(.text+0x2ea9): undefined reference to mpi_recv_' se_slice_module.o: In functionse_slice_module_MOD_se_slice2i’:
se_slice_module.f:(.text+0x3194): undefined reference to mpi_recv_' se_slice_module.f:(.text+0x34ca): undefined reference tompi_send

se_slice_module.f:(.text+0x3549): undefined reference to mpi_recv_' se_slice_module.o: In functionse_slice_module_MOD_se_slice1r’:
se_slice_module.f:(.text+0x360a): undefined reference to mpi_recv_' se_slice_module.f:(.text+0x364f): undefined reference tompi_send

se_slice_module.o: In function __se_slice_module_MOD_se_slice1i': se_slice_module.f:(.text+0x36ea): undefined reference tompi_recv

se_slice_module.f:(.text+0x372f): undefined reference to mpi_send_' se_init_module.o: In functionse_init_module_MOD_se_init’:
se_init_module.F:(.text+0xa5): undefined reference to mpi_comm_dup_' se_init_module.F:(.text+0xce): undefined reference tompi_comm_dup

se_init_module.F:(.text+0xb4d): undefined reference to mpi_comm_group_' se_init_module.F:(.text+0xeb3): undefined reference tompi_group_incl

se_init_module.F:(.text+0xed1): undefined reference to mpi_group_excl_' se_init_module.F:(.text+0xeed): undefined reference tompi_comm_create

se_init_module.F:(.text+0xf08): undefined reference to mpi_comm_create_' se_gather_module.o: In functionse_gather_module_MOD_se_gather4i’:
se_gather_module.f:(.text+0x271): undefined reference to mpi_gather_' se_gather_module.f:(.text+0x2d9): undefined reference tompi_gather

se_gather_module.o: In function __se_gather_module_MOD_se_gather3i': se_gather_module.f:(.text+0xef1): undefined reference tompi_gather

se_gather_module.f:(.text+0xf5a): undefined reference to mpi_gather_' se_gather_module.o: In functionse_gather_module_MOD_se_gather2i’:
se_gather_module.f:(.text+0x1acd): undefined reference to mpi_gather_' se_gather_module.o:se_gather_module.f:(.text+0x1b32): more undefined references tompi_gather
’ follow
se_twoway_comm_module.o: In function __se_twoway_comm_module_MOD_se_twoway_recv4': se_twoway_comm_module.f:(.text+0x689): undefined reference tompi_recv

se_twoway_comm_module.o: In function __se_twoway_comm_module_MOD_se_twoway_send4': se_twoway_comm_module.f:(.text+0xf5e): undefined reference tompi_send

se_twoway_comm_module.o: In function __se_twoway_comm_module_MOD_se_cmaq_wrf_comm4': se_twoway_comm_module.f:(.text+0x1f94): undefined reference tompi_send

se_twoway_comm_module.f:(.text+0x20c9): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x285b): undefined reference tompi_send

se_twoway_comm_module.f:(.text+0x2919): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x29a9): undefined reference tompi_send

se_twoway_comm_module.o: In function __se_twoway_comm_module_MOD_se_cmaq_wrf_comm3': se_twoway_comm_module.f:(.text+0x34b7): undefined reference tompi_send

se_twoway_comm_module.f:(.text+0x35d4): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x3c4f): undefined reference tompi_send

se_twoway_comm_module.f:(.text+0x3db3): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x3e25): undefined reference tompi_recv

se_twoway_comm_module.o: In function __se_twoway_comm_module_MOD_se_cmaq_wrf_comm2': se_twoway_comm_module.f:(.text+0x4761): undefined reference tompi_send_’
se_twoway_comm_module.f:(.text+0x4879): undefined reference to mpi_recv_' se_twoway_comm_module.o: In functionse_twoway_comm_module_MOD_se_wrf_cmaq_comm4’:
se_twoway_comm_module.f:(.text+0x5a74): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x5ba9): undefined reference tompi_recv

se_twoway_comm_module.f:(.text+0x633b): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x63f9): undefined reference tompi_recv

se_twoway_comm_module.f:(.text+0x6489): undefined reference to mpi_send_' se_twoway_comm_module.o: In functionse_twoway_comm_module_MOD_se_wrf_cmaq_comm3’:
se_twoway_comm_module.f:(.text+0x6f97): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x70b4): undefined reference tompi_recv

se_twoway_comm_module.f:(.text+0x772f): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x7893): undefined reference tompi_send

se_twoway_comm_module.f:(.text+0x7905): undefined reference to mpi_recv_' se_twoway_comm_module.o: In functionse_twoway_comm_module_MOD_se_wrf_cmaq_comm2’:
se_twoway_comm_module.f:(.text+0x8241): undefined reference to mpi_send_' se_twoway_comm_module.f:(.text+0x8359): undefined reference tompi_recv

se_util_module.o: In function __se_util_module_MOD_se_collect': se_util_module.f:(.text+0x2d9): undefined reference tompi_recv

se_util_module.f:(.text+0x60e): undefined reference to mpi_send_' se_util_module.o: In functionse_util_module_MOD_se_in_syn’:
se_util_module.f:(.text+0x65a): undefined reference to mpi_bcast_' se_util_module.o: In functionse_util_module_MOD_se_global_logical’:
se_util_module.f:(.text+0x44a9): undefined reference to mpi_reduce_' se_util_module.f:(.text+0x44d4): undefined reference tompi_bcast

se_util_module.o: In function __se_util_module_MOD_se_my_region': se_util_module.f:(.text+0x460e): undefined reference tompi_reduce

se_util_module.f:(.text+0x4636): undefined reference to mpi_bcast_' se_util_module.o: In functionse_util_module_MOD_se_rsum_chk’:
se_util_module.f:(.text+0x4690): undefined reference to mpi_reduce_' se_util_module.f:(.text+0x46f3): undefined reference tompi_bcast

se_util_module.o: In function __se_util_module_MOD_se_isum_chk': se_util_module.f:(.text+0x47a0): undefined reference tompi_reduce

se_util_module.f:(.text+0x47f6): undefined reference to mpi_bcast_' se_util_module.o: In functionse_util_module_MOD_se_global_min_data’:
se_util_module.f:(.text+0x4a2e): undefined reference to mpi_reduce_' se_util_module.f:(.text+0x4a5b): undefined reference tompi_bcast

se_util_module.f:(.text+0x4b9d): undefined reference to mpi_bcast_' se_util_module.o: In functionse_util_module_MOD_se_barrier’:
se_util_module.f:(.text+0x4d05): undefined reference to mpi_comm_dup_' se_util_module.f:(.text+0x4d12): undefined reference tompi_barrier

se_util_module.f:(.text+0x4d1f): undefined reference to mpi_comm_free_' se_util_module.f:(.text+0x4d3f): undefined reference tompi_barrier

swap_sandia.o: In function __swap_sandia_MOD_swap_init_snl': swap_sandia.f:(.text+0x20): undefined reference tompi_comm_rank

swap_sandia.f:(.text+0x36): undefined reference to mpi_comm_size_' swap_sandia_routines.o: In functionswap1d
’:
swap_sandia_routines.f:(.text+0x96): undefined reference to mpi_irecv_' swap_sandia_routines.f:(.text+0x10f): undefined reference tompi_send

swap_sandia_routines.f:(.text+0x188): undefined reference to mpi_wait_' swap_sandia_routines.f:(.text+0x1d0): undefined reference tompi_wait_’
swap_sandia_routines.o: In function swap2d_': swap_sandia_routines.f:(.text+0x2e2): undefined reference tompi_irecv_’
swap_sandia_routines.f:(.text+0x53e): undefined reference to mpi_send_' swap_sandia_routines.f:(.text+0x590): undefined reference tompi_wait_’
swap_sandia_routines.f:(.text+0x5e0): undefined reference to mpi_wait_' swap_sandia_routines.o: In functionswap3d_’:
swap_sandia_routines.f:(.text+0x910): undefined reference to mpi_irecv_' swap_sandia_routines.f:(.text+0xa34): undefined reference tompi_send_’
swap_sandia_routines.f:(.text+0xa98): undefined reference to mpi_wait_' swap_sandia_routines.f:(.text+0xb18): undefined reference tompi_wait_’
swap_sandia_routines.o: In function swap4d_': swap_sandia_routines.f:(.text+0xd9f): undefined reference tompi_wait_’
swap_sandia_routines.f:(.text+0xf0e): undefined reference to mpi_irecv_' swap_sandia_routines.f:(.text+0x106c): undefined reference tompi_send_’
pm3err.o: In function pm3err_': pm3err.f:(.text+0xce2): undefined reference tompi_abort_’
pm3exit.o: In function pm3exit_': pm3exit.f:(.text+0x8f2): undefined reference tompi_abort_’
pshut3.o: In function pshut3_': pshut3.F:(.text+0x665): undefined reference tompi_bcast_’
ptrwrite3.o: In function ptrwrite3_': ptrwrite3.f:(.text+0x1ff): undefined reference tompi_bcast_’
ptrwrite3.f:(.text+0x250): undefined reference to mpi_bcast_' ptrwrite3.f:(.text+0x279): undefined reference tompi_bcast_’
ptrwrite3.f:(.text+0x372): undefined reference to mpi_bcast_' ptrwrite3.o:ptrwrite3.f:(.text+0x9fc): more undefined references tompi_bcast_’ follow
pwrgrdd.o: In function pwrgrdd_': pwrgrdd.f:(.text+0x651): undefined reference tompi_recv_’
pwrgrdd.f:(.text+0x746): undefined reference to mpi_recv_' pwrgrdd.f:(.text+0xc9a): undefined reference tompi_send_’
pwrgrdd.f:(.text+0xce4): undefined reference to mpi_send_' pwrgrdd.f:(.text+0xd2e): undefined reference tompi_allreduce_’
pwrite3.o: In function pwrite3_': pwrite3.f:(.text+0x206): undefined reference tompi_bcast_’
pwrite3.f:(.text+0x257): undefined reference to mpi_bcast_' pwrite3.f:(.text+0x280): undefined reference tompi_bcast_’
pwrite3.f:(.text+0x36f): undefined reference to mpi_bcast_' pwrite3.f:(.text+0x98c): undefined reference tompi_bcast_’
distr_env.o: In function distr_env_': distr_env.c:(.text+0xd6): undefined reference toompi_mpi_comm_world’
distr_env.c:(.text+0xdd): undefined reference to ompi_mpi_int' distr_env.c:(.text+0xe9): undefined reference toMPI_Bcast’
distr_env.c:(.text+0xf4): undefined reference to ompi_mpi_comm_world' distr_env.c:(.text+0xfb): undefined reference toompi_mpi_char’
distr_env.c:(.text+0x105): undefined reference to MPI_Bcast' distr_env.c:(.text+0x10c): undefined reference toompi_mpi_comm_world’
distr_env.c:(.text+0x111): undefined reference to MPI_Barrier' distr_env.c:(.text+0x148): undefined reference toompi_mpi_comm_world’
distr_env.c:(.text+0x14f): undefined reference to ompi_mpi_int' distr_env.c:(.text+0x163): undefined reference toMPI_Bcast’
distr_env.c:(.text+0x16e): undefined reference to ompi_mpi_comm_world' distr_env.c:(.text+0x175): undefined reference toompi_mpi_char’
distr_env.c:(.text+0x187): undefined reference to MPI_Bcast' mpcomm_init.o: In functionmpcomm_init_’:
mpcomm_init.F:(.text+0x4d): undefined reference to mpi_init_' mpcomm_init.F:(.text+0x60): undefined reference tompi_wtime_’
mpcomm_init.F:(.text+0x82): undefined reference to mpi_comm_size_' mpcomm_init.F:(.text+0xa2): undefined reference tompi_comm_rank_’
setup_logdev.o: In function setup_logdev_': setup_logdev.F:(.text+0x6d): undefined reference tompi_comm_rank_’
collect2: error: ld returned 1 exit status
Makefile:408: recipe for target ‘CCTM_v52.exe’ failed
make: *** [CCTM_v52.exe] Error 1
ERROR while running make command

You have an error with specifying the MPI libraries in the link-line. Note that what is required is dependent upon which MPI implementation you are using – the MPICH implementation of MPI uses different libraries than OPENMPI (and further note that it is also dependent upon the vendor of the underlying C and Fortran libraries…

Ok. I am using OPENMPI and netcdf-4.1.3. What’s the way forward now, please? Kindly assist me. Thanks.

/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wrdict3.o): In function wrdict3_': wrdict3.f90:(.text+0x3a7): undefined reference toGOMP_critical_name_start’
wrdict3.f90:(.text+0x3b3): undefined reference to GOMP_critical_name_start' wrdict3.f90:(.text+0x55e): undefined reference toGOMP_critical_name_end’
wrdict3.f90:(.text+0x56a): undefined reference to GOMP_critical_name_end' wrdict3.f90:(.text+0x8f3): undefined reference toGOMP_critical_name_end’
wrdict3.f90:(.text+0x8ff): undefined reference to GOMP_critical_name_end' wrdict3.f90:(.text+0x15b0): undefined reference toGOMP_critical_name_end’
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wrdict3.o):wrdict3.f90:(.text+0x15bc): more undefined references to GOMP_critical_name_end' follow /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wriddata.o): In functionwriddata_’:
wriddata.f90:(.text+0x44): undefined reference to GOMP_critical_name_start' wriddata.f90:(.text+0x50): undefined reference toGOMP_critical_name_start’
wriddata.f90:(.text+0x184): undefined reference to GOMP_critical_name_end' wriddata.f90:(.text+0x190): undefined reference toGOMP_critical_name_end’
wriddata.f90:(.text+0x225): undefined reference to GOMP_critical_name_end' wriddata.f90:(.text+0x231): undefined reference toGOMP_critical_name_end’
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wrsmatrx.o): In function wrsmatrx_': wrsmatrx.f90:(.text+0xd3): undefined reference toGOMP_critical_name_start’
wrsmatrx.f90:(.text+0xdf): undefined reference to GOMP_critical_name_start' wrsmatrx.f90:(.text+0x214): undefined reference toGOMP_critical_name_end’
wrsmatrx.f90:(.text+0x220): undefined reference to GOMP_critical_name_end' wrsmatrx.f90:(.text+0x2aa): undefined reference toGOMP_critical_name_end’
wrsmatrx.f90:(.text+0x2b6): undefined reference to GOMP_critical_name_end' /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(crtfil3.o): In functioncrtfil3_’:
crtfil3.F90:(.text+0x2c): undefined reference to GOMP_critical_name_start' crtfil3.F90:(.text+0x46): undefined reference toGOMP_critical_name_end’
crtfil3.F90:(.text+0x69): undefined reference to GOMP_critical_name_start' crtfil3.F90:(.text+0x4a5): undefined reference toGOMP_critical_name_end’
crtfil3.F90:(.text+0x570): undefined reference to GOMP_critical_name_end' crtfil3.F90:(.text+0x57c): undefined reference toGOMP_critical_name_start’
crtfil3.F90:(.text+0x5af): undefined reference to GOMP_critical_name_end' crtfil3.F90:(.text+0xa49): undefined reference toGOMP_critical_name_end’
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wrtflag.o): In function wrtflag_': wrtflag.F90:(.text+0xaa): undefined reference toGOMP_critical_name_start’
wrtflag.F90:(.text+0xe3): undefined reference to GOMP_critical_name_end' wrtflag.F90:(.text+0x731): undefined reference toGOMP_critical_name_start’
wrtflag.F90:(.text+0x757): undefined reference to GOMP_critical_name_end' /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(rdatt3.o): In functionmaster.0.rdatt3_’:
rdatt3.F90:(.text+0x3f5): undefined reference to GOMP_critical_name_start' rdatt3.F90:(.text+0x681): undefined reference toGOMP_critical_name_end’
rdatt3.F90:(.text+0x133b): undefined reference to GOMP_critical_name_end' rdatt3.F90:(.text+0x1cd5): undefined reference toGOMP_critical_name_end’
rdatt3.F90:(.text+0x1d12): undefined reference to GOMP_critical_name_end' rdatt3.F90:(.text+0x1da3): undefined reference toGOMP_critical_name_end’
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/lib/libioapi.a(wratt3.o): In function master.0.wratt3_': wratt3.F90:(.text+0x659): undefined reference toGOMP_critical_name_start’
wratt3.F90:(.text+0x6e7): undefined reference to GOMP_critical_name_end' wratt3.F90:(.text+0xb3b): undefined reference toGOMP_critical_name_end’
wratt3.F90:(.text+0x126f): undefined reference to `GOMP_critical_name_end’
collect2: error: ld returned 1 exit status
Makefile:408: recipe for target ‘CCTM_v52.exe’ failed
make: *** [CCTM_v52.exe] Error 1
ERROR while running make command

What’s your underlying compiler for OpenMPI? Since the I/O API is openmp-enabled (with various other programs using omp shared-memory parallelism), you need to include the appropriate OpenMP flag in your link step (-qopenmp for Intel, -fopenmp for GNU, -mp for PGI, …)

Thanks for your response. Below is my Compiler Aliases and Flags:

#> Compiler Aliases and Flags
setenv myFC gfortran
#setenv myFC mpifort
setenv myCC gcc
setenv myFSTD “-O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising”
setenv myDBG “-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace”
#setenv myDBG “$myDBG -fimplicit-none”
setenv myFFLAGS “-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32”
setenv myFRFLAGS “-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32”
setenv myCFLAGS “-O2”
setenv myLINK_FLAG “-fopenmp”
setenv extra_lib “”
#setenv mpi_lib “-lmpi_mpifh” #> -lmpich for mvapich or -lmpi for openmpi
setenv mpi_lib “” #> -lmpich for mvapich or -lmpi for openmpi

sp you need to add -fopenmp to your link-flags

For my CCTM Script"

setenv FC ${myFC} #> path of Fortan compiler; set in config.cmaq
set FP = FC #> path of Fortan preprocessor; set in config.cmaq set CC = {myCC} #> path of C compiler; set in config.cmaq
setenv BLDER {CMAQ_HOME}/UTIL/bldmake/bldmake_{compilerString}.exe #> name of model builder executable

#> Libraries/include files
set LIOAPI = “{IOAPI_DIR}/lib {ioapi_lib}” #> I/O API library directory
set IOAPIMOD = “{IOAPI_DIR}/include" #> I/O API module directory set NETCDF = "{NETCDF_DIR}/lib {netcdf_lib}" #> netCDF library directory #set PNETCDF = "{PNETCDF_DIR}/lib {pnetcdf_lib}" #> Parallel netCDF library directory #set PIO_INC = "{IOAPI_DIR}/src”

#> Compiler flags set in config.cmaq
set FSTD = “{myFSTD}" set DBG = "{myDBG}”
setenv F_FLAGS “{myFFLAGS}" #> F77 flags set F90_FLAGS = "{myFRFLAGS}” #> F90 flags
set CPP_FLAGS = “” #> Fortran preprocessor flags
set C_FLAGS = “{myCFLAGS} -DFLDMN -I" #> C flags set LINK_FLAGS = "{myLINK_FLAG}” # Link flags

I have been able to successfully compile CCTM after so many manipulations of my scripts. Thanks so much for your guide. I’m available to help those facing the challenges I have faced in the course of compiling CMAQ and its related models.

Cheers.

Catalyst