Se_util_module.f:(.text+0x32e): undefined reference to `mpi_reduce_'

CMAQ Compiling
1

Hi everyone,
I got the error related to compile the CCTM like that:

se_data_send_module.o: In function se_data_send_module_mp_se_1d_data_send_': se_data_send_module.f:(.text+0x42b): undefined reference to mpi_send_’
se_data_send_module.o: In function se_data_send_module_mp_se_2d_data_send_': se_data_send_module.f:(.text+0xb77): undefined reference to mpi_send_’
se_data_send_module.o: In function se_data_send_module_mp_se_2de_data_send_': se_data_send_module.f:(.text+0x1210): undefined reference to mpi_send_’
se_data_send_module.o: In function se_data_send_module_mp_se_3d_data_send_': se_data_send_module.f:(.text+0x1b94): undefined reference to mpi_send_’
se_data_send_module.o: In function se_data_send_module_mp_se_3de_data_send_': se_data_send_module.f:(.text+0x23f4): undefined reference to mpi_send_’
se_data_send_module.o:se_data_send_module.f:(.text+0x2f0f): more undefined references to mpi_send_' follow se_internal_util_module.o: In function se_internal_util_module_mp_se_generate_map_‘:
se_internal_util_module.f:(.text+0xd4): undefined reference to mpi_gather_' se_internal_util_module.f:(.text+0x106): undefined reference to mpi_bcast_’
se_data_recv_module.o: In function se_data_recv_module_mp_se_1d_data_recv_': se_data_recv_module.f:(.text+0x225): undefined reference to mpi_recv_’
se_data_recv_module.o: In function se_data_recv_module_mp_se_2d_data_recv_': se_data_recv_module.f:(.text+0x6fd): undefined reference to mpi_recv_’
se_data_recv_module.o: In function se_data_recv_module_mp_se_2de_data_recv_': se_data_recv_module.f:(.text+0xda9): undefined reference to mpi_recv_’
se_data_recv_module.o: In function se_data_recv_module_mp_se_3d_data_recv_': se_data_recv_module.f:(.text+0x1449): undefined reference to mpi_recv_’
se_data_recv_module.o: In function se_data_recv_module_mp_se_3de_data_recv_': se_data_recv_module.f:(.text+0x1d28): undefined reference to mpi_recv_’
se_data_recv_module.o:se_data_recv_module.f:(.text+0x25f6): more undefined references to mpi_recv_' follow se_global_sum_module.o: In function se_global_sum_module_mp_se_global_isum_‘:
se_global_sum_module.f:(.text+0x3d): undefined reference to mpi_reduce_' se_global_sum_module.f:(.text+0x77): undefined reference to mpi_bcast_’
se_global_sum_module.o: In function se_global_sum_module_mp_se_global_rsum_': se_global_sum_module.f:(.text+0xbd): undefined reference to mpi_reduce_’
se_global_sum_module.f:(.text+0xf7): undefined reference to mpi_bcast_' se_global_sum_module.o: In function se_global_sum_module_mp_se_global_iasum_‘:
se_global_sum_module.f:(.text+0x285): undefined reference to mpi_reduce_' se_global_sum_module.f:(.text+0x2e5): undefined reference to mpi_bcast_’
se_global_sum_module.o: In function se_global_sum_module_mp_se_global_rasum_': se_global_sum_module.f:(.text+0x4d5): undefined reference to mpi_reduce_’
se_global_sum_module.f:(.text+0x535): undefined reference to mpi_bcast_' se_global_max_module.o: In function se_global_max_module_mp_se_global_imax_‘:
se_global_max_module.f:(.text+0x3a): undefined reference to mpi_reduce_' se_global_max_module.f:(.text+0x63): undefined reference to mpi_bcast_’
se_global_max_module.o: In function se_global_max_module_mp_se_global_rmax_': se_global_max_module.f:(.text+0x9a): undefined reference to mpi_reduce_’
se_global_max_module.f:(.text+0xc3): undefined reference to mpi_bcast_' se_global_min_module.o: In function se_global_min_module_mp_se_global_imin_‘:
se_global_min_module.f:(.text+0x3a): undefined reference to mpi_reduce_' se_global_min_module.f:(.text+0x63): undefined reference to mpi_bcast_’
se_global_min_module.o: In function se_global_min_module_mp_se_global_rmin_': se_global_min_module.f:(.text+0x9a): undefined reference to mpi_reduce_’
se_global_min_module.f:(.text+0xc3): undefined reference to mpi_bcast_' se_slice_module.o: In function se_slice_module_mp_se_slice1i_‘:
se_slice_module.f:(.text+0x7d): undefined reference to mpi_recv_' se_slice_module.f:(.text+0xba): undefined reference to mpi_send_’
se_slice_module.o: In function se_slice_module_mp_se_slice1r_': se_slice_module.f:(.text+0x13d): undefined reference to mpi_recv_’
se_slice_module.f:(.text+0x17a): undefined reference to mpi_send_' se_slice_module.o: In function se_slice_module_mp_se_slice2i_‘:
se_slice_module.f:(.text+0x673): undefined reference to mpi_send_' se_slice_module.f:(.text+0x6f6): undefined reference to mpi_recv_’
se_slice_module.o: In function se_slice_module_mp_se_slice2r_': se_slice_module.f:(.text+0xf93): undefined reference to mpi_send_’
se_slice_module.f:(.text+0x1016): undefined reference to mpi_recv_' se_slice_module.o: In function se_slice_module_mp_se_slice3i_‘:
se_slice_module.f:(.text+0x1a4e): undefined reference to mpi_send_' se_slice_module.f:(.text+0x1b07): undefined reference to mpi_recv_’
se_slice_module.o: In function se_slice_module_mp_se_slice3r_': se_slice_module.f:(.text+0x26ce): undefined reference to mpi_send_’
se_slice_module.f:(.text+0x2787): undefined reference to mpi_recv_' se_slice_module.o: In function se_slice_module_mp_se_slice4i_‘:
se_slice_module.f:(.text+0x3434): undefined reference to mpi_send_' se_slice_module.f:(.text+0x3525): undefined reference to mpi_recv_’
se_slice_module.o: In function se_slice_module_mp_se_slice4r_': se_slice_module.f:(.text+0x4294): undefined reference to mpi_send_’
se_slice_module.f:(.text+0x4385): undefined reference to mpi_recv_' se_init_module.o: In function se_init_module_mp_se_init_‘:
se_init_module.F:(.text+0x7ef): undefined reference to mpi_comm_dup_' se_init_module.F:(.text+0x818): undefined reference to mpi_comm_dup_’
se_init_module.F:(.text+0x1491): undefined reference to mpi_comm_group_' se_init_module.F:(.text+0x16e0): undefined reference to mpi_group_incl_’
se_init_module.F:(.text+0x1702): undefined reference to mpi_group_excl_' se_init_module.F:(.text+0x171d): undefined reference to mpi_comm_create_’
se_init_module.F:(.text+0x1738): undefined reference to mpi_comm_create_' se_comm_module.o: In function se_comm_module_mp_se_pe_comm3s_‘:
se_comm_module.f:(.text+0x748b): undefined reference to mpi_send_' se_comm_module.f:(.text+0x79a5): undefined reference to mpi_recv_’
se_gather_module.o: In function se_gather_module_mp_se_setup_gather_': se_gather_module.f:(.text+0x541): undefined reference to mpi_gather_’
se_gather_module.f:(.text+0x5a0): undefined reference to mpi_gather_' se_gather_module.o: In function se_gather_module_mp_se_gather1r_‘:
se_gather_module.f:(.text+0x150b): undefined reference to mpi_gather_' se_gather_module.f:(.text+0x156d): undefined reference to mpi_gather_’
se_gather_module.o: In function se_gather_module_mp_se_gather2r_': se_gather_module.f:(.text+0x222b): undefined reference to mpi_gather_’
se_gather_module.o:se_gather_module.f:(.text+0x228d): more undefined references to mpi_gather_' follow se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_wrf_cmaq_comm2_‘:
se_twoway_comm_module.f:(.text+0xe43): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x18f3): undefined reference to mpi_send_’
se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_wrf_cmaq_comm3_': se_twoway_comm_module.f:(.text+0x2aee): undefined reference to mpi_recv_’
se_twoway_comm_module.f:(.text+0x3792): undefined reference to mpi_send_' se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_wrf_cmaq_comm4_‘:
se_twoway_comm_module.f:(.text+0x48fd): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x56d3): undefined reference to mpi_send_’
se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_cmaq_wrf_comm2_': se_twoway_comm_module.f:(.text+0x65e3): undefined reference to mpi_recv_’
se_twoway_comm_module.f:(.text+0x7093): undefined reference to mpi_send_' se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_cmaq_wrf_comm3_‘:
se_twoway_comm_module.f:(.text+0x828e): undefined reference to mpi_recv_' se_twoway_comm_module.f:(.text+0x8f32): undefined reference to mpi_send_’
se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_cmaq_wrf_comm4_': se_twoway_comm_module.f:(.text+0xa09d): undefined reference to mpi_recv_’
se_twoway_comm_module.f:(.text+0xae73): undefined reference to mpi_send_' se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_twoway_send4_‘:
se_twoway_comm_module.f:(.text+0xb869): undefined reference to mpi_send_' se_twoway_comm_module.o: In function se_twoway_comm_module_mp_se_twoway_recv4_’:
se_twoway_comm_module.f:(.text+0xc676): undefined reference to mpi_recv_' se_util_module.o: In function se_util_module_mp_se_barrier_‘:
se_util_module.f:(.text+0xf7): undefined reference to mpi_comm_dup_' se_util_module.f:(.text+0x107): undefined reference to mpi_barrier_’
se_util_module.f:(.text+0x117): undefined reference to mpi_comm_free_' se_util_module.f:(.text+0x12a): undefined reference to mpi_barrier_’
se_util_module.o: In function se_util_module_mp_se_global_min_data_': se_util_module.f:(.text+0x32e): undefined reference to mpi_reduce_’
se_util_module.f:(.text+0x35f): undefined reference to mpi_bcast_' se_util_module.f:(.text+0x4d0): undefined reference to mpi_bcast_’
se_util_module.o: In function se_util_module_mp_se_isum_chk_': se_util_module.f:(.text+0x7b8): undefined reference to mpi_reduce_’
se_util_module.f:(.text+0x8b3): undefined reference to mpi_bcast_' se_util_module.o: In function se_util_module_mp_se_rsum_chk_‘:
se_util_module.f:(.text+0x908): undefined reference to mpi_reduce_' se_util_module.f:(.text+0xa0d): undefined reference to mpi_bcast_’
se_util_module.o: In function se_util_module_mp_se_my_region_': se_util_module.f:(.text+0xb05): undefined reference to mpi_reduce_’
se_util_module.f:(.text+0xb43): undefined reference to mpi_bcast_' se_util_module.o: In function se_util_module_mp_se_global_logical_‘:
se_util_module.f:(.text+0xbe6): undefined reference to mpi_reduce_' se_util_module.f:(.text+0xc59): undefined reference to mpi_reduce_’
se_util_module.f:(.text+0xc82): undefined reference to mpi_bcast_' se_util_module.o: In function se_util_module_mp_se_in_syn_‘:
se_util_module.f:(.text+0x4be0): undefined reference to mpi_bcast_' se_util_module.o: In function se_util_module_mp_se_collect_’:
se_util_module.f:(.text+0x4eb9): undefined reference to mpi_recv_' se_util_module.f:(.text+0x4f50): undefined reference to mpi_send_’
pm3err.o: In function pm3err_': pm3err.f:(.text+0x3d2): undefined reference to mpi_abort_’
pm3exit.o: In function pm3exit_': pm3exit.f:(.text+0x4d3): undefined reference to mpi_abort_’
pshut3.o: In function pshut3_': pshut3.F:(.text+0x3fd): undefined reference to mpi_bcast_’
ptrwrite3.o: In function ptrwrite3_': ptrwrite3.f:(.text+0x3ad): undefined reference to mpi_bcast_’
ptrwrite3.f:(.text+0x4c2): undefined reference to mpi_bcast_' ptrwrite3.f:(.text+0x4eb): undefined reference to mpi_bcast_’
ptrwrite3.f:(.text+0x74e): undefined reference to mpi_bcast_' ptrwrite3.o:ptrwrite3.f:(.text+0x1d78): more undefined references to mpi_bcast_’ follow
pwrgrdd.o: In function pwrgrdd_': pwrgrdd.f:(.text+0x93a): undefined reference to mpi_recv_’
pwrgrdd.f:(.text+0xa2c): undefined reference to mpi_recv_' pwrgrdd.f:(.text+0xdc3): undefined reference to mpi_send_’
pwrgrdd.f:(.text+0xe01): undefined reference to mpi_send_' pwrgrdd.f:(.text+0xe3c): undefined reference to mpi_allreduce_’
pwrite3.o: In function pwrite3_': pwrite3.f:(.text+0x376): undefined reference to mpi_bcast_’
pwrite3.f:(.text+0x48a): undefined reference to mpi_bcast_' pwrite3.f:(.text+0x4b3): undefined reference to mpi_bcast_’
pwrite3.f:(.text+0x6f6): undefined reference to mpi_bcast_' pwrite3.f:(.text+0x96b): undefined reference to mpi_bcast_’
RUNTIME_VARS.o: In function runtime_vars_mp_timing_split_': RUNTIME_VARS.F:(.text+0x6a4): undefined reference to mpi_wtime_’
RUNTIME_VARS.F:(.text+0xa2c): undefined reference to mpi_wtime_' RUNTIME_VARS.o: In function runtime_vars_mp_init_env_vars_‘:
RUNTIME_VARS.F:(.text+0x175f): undefined reference to mpi_comm_rank_' RUNTIME_VARS.F:(.text+0x1777): undefined reference to mpi_comm_size_’
vertext_module.o: In function vertext_module_mp_pwrvec_': vertext_module.F:(.text+0x54d4): undefined reference to mpi_recv_’
vertext_module.F:(.text+0x5eed): undefined reference to mpi_send_' vertext_module.F:(.text+0x5f2c): undefined reference to mpi_allreduce_’
distr_env.o: In function distr_env_': distr_env.c:(.text+0x163): undefined reference to MPI_Bcast’
distr_env.c:(.text+0x17f): undefined reference to MPI_Bcast' distr_env.c:(.text+0x28c): undefined reference to MPI_Barrier’
mpcomm_init.o: In function mpcomm_init_': mpcomm_init.F:(.text+0x25): undefined reference to mpi_init_’
mpcomm_init.F:(.text+0x38): undefined reference to mpi_wtime_' mpcomm_init.F:(.text+0x51): undefined reference to mpi_comm_size_’
mpcomm_init.F:(.text+0x71): undefined reference to mpi_comm_rank_' cmaq_main.o: In function MAIN__‘:
cmaq_main.F:(.text+0x234): undefined reference to mpi_wtime_' cmaq_main.F:(.text+0x37b): undefined reference to mpi_finalize_’
driver.o: In function cmaq_driver_': driver.F:(.text+0x4531): undefined reference to mpi_wtime_’
sciproc.o: In function sciproc_': sciproc.F:(.text+0x82): undefined reference to mpi_wtime_’
make: *** [CCTM_v532.exe] Error 1
ERROR while running make command
mv Makefile Makefile.intel
mv: ‘Makefile’ and ‘Makefile.intel’ are the same file
if ( -e Makefile.intel && -e Makefile ) rm Makefile
rm Makefile
ln -s Makefile.intel Makefile
if ( 0 != 0 ) then
if ( -e /work/users/weather/user/manh/cmaq/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_intel/CCTM_v532.cfg ) then
echo >>> previous CCTM_v532.cfg exists, re-naming to CCTM_v532.cfg.old <<<

previous CCTM_v532.cfg exists, re-naming to CCTM_v532.cfg.old <<<
mv /work/users/weather/user/manh/cmaq/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_intel/CCTM_v532.cfg /work/users/weather/user/manh/cmaq/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_intel/CCTM_v532.cfg.old
endif
mv CCTM_v532.cfg.bld /work/users/weather/user/manh/cmaq/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_intel/CCTM_v532.cfg
exit

My option of the config_cmaq.csh is

setenv myFC ifort
setenv myCC icc
setenv myFSTD “-O3 -fno-alias -mp1 -fp-model source -ftz -simd -align all -xHost -vec-guard-write -unroll-aggressive”
setenv myDBG “-O0 -g -check bounds -check uninit -fpe0 -fno-alias -ftrapuv -traceback”
setenv myLINK_FLAG “-qopenmp” #openMP not supported w/ CMAQ
setenv myFFLAGS “-fixed -132”
setenv myFRFLAGS “-free”
setenv myCFLAGS “-O2”
setenv extra_lib “-lcurl”
#setenv extra_lib “”
setenv mpi_lib “” #> No Library specification needed for mpiifort
#> -lmpich for mvapich
#> -lmpi for openmpi
breaksw

Please help me to address it.!
Thanks all,

2

Please follow the tutorial available here: CMAQ/CMAQ_UG_tutorial_benchmark.md at master · USEPA/CMAQ · GitHub

If you are using openmpi, you will need to add -l mpi in this section of your config_cmaq.csh

setenv mpi_lib “-lmpi” #> No Library specification needed for mpiifort
                                   #> -lmpich for mvapich
                                   #> -lmpi for openmpi

Please also be sure you are loading the correct module for openmpi.