Undefined references when compiling MCIP


I am trying to compile MCIP, however I keep getting the following undefined references:

/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(wrmpigrd.o): In function wrmpigrd_': wrmpigrd.F90:(.text+0x52): undefined reference tompi_bcast_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(modpdata.o): In function __modpdata_MOD_pn_flag': modpdata.F90:(.text+0x350): undefined reference tompi_allreduce_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(modpdata.o): In function __modpdata_MOD_pn_setup': modpdata.F90:(.text+0xeb0): undefined reference tompi_comm_rank_’
modpdata.F90:(.text+0xf0b): undefined reference to mpi_comm_group_' modpdata.F90:(.text+0xf35): undefined reference tompi_group_incl_’
modpdata.F90:(.text+0xf5a): undefined reference to mpi_comm_create_' modpdata.F90:(.text+0xf82): undefined reference tompi_group_excl_’
modpdata.F90:(.text+0xfa9): undefined reference to mpi_comm_create_' modpdata.F90:(.text+0xfc9): undefined reference tompi_comm_dup_’
modpdata.F90:(.text+0x1c3f): undefined reference to mpi_comm_dup_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In functionpn_wrint_’:
pn_wrvars.F90:(.text+0xc49): undefined reference to mpi_recv_' pn_wrvars.F90:(.text+0x1474): undefined reference tompi_send_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function pn_wrreal_': pn_wrvars.F90:(.text+0x2109): undefined reference tompi_recv_’
pn_wrvars.F90:(.text+0x2934): undefined reference to mpi_send_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In functionpn_wrdble_’:
pn_wrvars.F90:(.text+0x35c9): undefined reference to mpi_recv_' pn_wrvars.F90:(.text+0x3df4): undefined reference tompi_send_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function pn_wrint8_': pn_wrvars.F90:(.text+0x4a89): undefined reference tompi_recv_’
pn_wrvars.F90:(.text+0x52b4): undefined reference to `mpi_send_’
collect2: error: ld returned 1 exit status
Makefile:182: recipe for target ‘mcip.exe’ failed

This implies that the appropriate libraries are not being linked in the makefile. Here is the relevant portion of my makefile;

LIBS = -L$(IOAPI_ROOT)/Linux4_x86_64gfortmpi -lioapi
-L$(NETCDF)/lib -lnetcdff -lnetcdf -lpnetcdf -lmpich -lgomp

I added -lpnetcdf, -lmpich, and -lgomp which corrected a bunch of other undefined references. I’ve only included the remaining errors after linking those 3 libraries. My problem is that the undefined subroutines should be linked in -lmpich library, but they’re not.

Any suggestions?

Thanks in advance.


You don’t say what other undefined references you were getting, but since MCIP is an entirely serial code, there is no need for MPICH, openMP, or pnetCDF.


I had a similar problem with the ifort compiler.
I was getting the following
ld: /proj/ie/proj/CMAS/CMAQ/CMAQv5.3_branch_UNC3/mvapich2_2.3rc1_intel_17.2/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): in function m3msg2_': m3msg2.F:(.text+0x20): undefined reference to__kmpc_global_thread_num’
ld: m3msg2.F:(.text+0x37): undefined reference to __kmpc_critical' ld: m3msg2.F:(.text+0x9e): undefined reference to__kmpc_end_critical’

I solved it by adding -qopenmp to the LIBS section of the Makefile
LIBS = -L$(IOAPI_ROOT)/lib -lioapi
-L$(NETCDF)/lib -lnetcdff -lnetcdf -qopenmp

I think the problem is that whatever flags are used to build ioapi need to be used for compiling MCIP and CMAQ. In my case I took a look at whether -qopenmp was used in the Makeinclude.* file for ioapi build, and found:

>grep openmp Makeinclude.Linux2_x86_64ifort
OMPFLAGS  = -qopenmp      # for Intel compilers, version 16 or later
OMPLIBS   = -qopenmp

PnetCDF/MPI enabled I/O API should only be used with CMAQ; all others should use “vanilla” I/O API.
This is documented in the I/O API installation instructions, https://cjcoats.github.io/ioapi/AVAIL.html#build
though perhaps the wording might be improved (and strengthened).

In principle, a lot of effort has been expended to make it work anyway, but it does require substantial changes to compiler, compile-flags, and link-flags in Makefiles, etc.


@cothatch and @lizadams

As was noted by @cgnolte and @cjcoats, MCIP is a serial code, so there is no dependency on parallel processing. MCIP will not need a version of I/O API that is built for MPI or openMP. There are no plans to make MCIP into a parallelized code.

However, it seems like there is another disconnect somewhere in the build scripts for the entire CMAQ system, so this will need to be addressed. I don’t know if it’s possible to (at least temporarily) maintain both a parallel build of I/O API and a serial build of I/O API. I realize this is suboptimal.

I’m open to suggestions on how to make this work more seamlessly for the user community.



There are also dependencies on the netCDF build. You can use commands such as

nc-config --all
nf-config --all

To see how the C and Fortran versions of the netCDF library are built, respectively.


I have found that gfortran will not compile if there is a symbol that is unresolved in the *.a files, even if the code doesn’t use it. Try adding the following


For example:

LIBS    = -L$(IOAPI_ROOT) -lioapi  \
          -L$(NETCDF)/lib -lnetcdff -lnetcdf -lnetcdf -Wl,--unresolved-symbols=ignore-in-object-files

For more information see: https://stackoverflow.com/questions/5555632/can-gcc-not-complain-about-undefined-references

Bcon compilation

The solution here is that you have different $BIN-directories for the two different library-versions. And you use one $BIN for CMAQ, and the other for everything else.

– Carlie

1 Like

It looks like you are using IOAPI 3.2. Since MCIP is strictly a serial code (it can be parallelized easily), you can only link with regular IOAPI (@cjoats has mentioned it as vanilla IOAPI). In other words, when you build your IOAPI library, set BIN Linux2_x86_64ifort not Linux2_x86_64ifortmpi (assume you were using ifort compiler and same things applies to gcc or pgi compiler).


Thank you all for your help.

I made another a serial build of I/O API as many of you suggested this resulted in the many of the same undefined references. However, I then included the -fopenmp library as @lizadams suggested and MCIP compiled correctly.

Thanks again

1 Like