Undefined references when compiling MCIP

#1

I am trying to compile MCIP, however I keep getting the following undefined references:

/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(wrmpigrd.o): In function wrmpigrd_': wrmpigrd.F90:(.text+0x52): undefined reference tompi_bcast_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(modpdata.o): In function __modpdata_MOD_pn_flag': modpdata.F90:(.text+0x350): undefined reference tompi_allreduce_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(modpdata.o): In function __modpdata_MOD_pn_setup': modpdata.F90:(.text+0xeb0): undefined reference tompi_comm_rank_’
modpdata.F90:(.text+0xf0b): undefined reference to mpi_comm_group_' modpdata.F90:(.text+0xf35): undefined reference tompi_group_incl_’
modpdata.F90:(.text+0xf5a): undefined reference to mpi_comm_create_' modpdata.F90:(.text+0xf82): undefined reference tompi_group_excl_’
modpdata.F90:(.text+0xfa9): undefined reference to mpi_comm_create_' modpdata.F90:(.text+0xfc9): undefined reference tompi_comm_dup_’
modpdata.F90:(.text+0x1c3f): undefined reference to mpi_comm_dup_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In functionpn_wrint_’:
pn_wrvars.F90:(.text+0xc49): undefined reference to mpi_recv_' pn_wrvars.F90:(.text+0x1474): undefined reference tompi_send_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function pn_wrreal_': pn_wrvars.F90:(.text+0x2109): undefined reference tompi_recv_’
pn_wrvars.F90:(.text+0x2934): undefined reference to mpi_send_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In functionpn_wrdble_’:
pn_wrvars.F90:(.text+0x35c9): undefined reference to mpi_recv_' pn_wrvars.F90:(.text+0x3df4): undefined reference tompi_send_’
/home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function pn_wrint8_': pn_wrvars.F90:(.text+0x4a89): undefined reference tompi_recv_’
pn_wrvars.F90:(.text+0x52b4): undefined reference to `mpi_send_’
collect2: error: ld returned 1 exit status
Makefile:182: recipe for target ‘mcip.exe’ failed

This implies that the appropriate libraries are not being linked in the makefile. Here is the relevant portion of my makefile;

LIBS = -L$(IOAPI_ROOT)/Linux4_x86_64gfortmpi -lioapi
-L$(NETCDF)/lib -lnetcdff -lnetcdf -lpnetcdf -lmpich -lgomp

I added -lpnetcdf, -lmpich, and -lgomp which corrected a bunch of other undefined references. I’ve only included the remaining errors after linking those 3 libraries. My problem is that the undefined subroutines should be linked in -lmpich library, but they’re not.

Any suggestions?

Thanks in advance.

#2

You don’t say what other undefined references you were getting, but since MCIP is an entirely serial code, there is no need for MPICH, openMP, or pnetCDF.

#3

I had a similar problem with the ifort compiler.
I was getting the following
ld: /proj/ie/proj/CMAS/CMAQ/CMAQv5.3_branch_UNC3/mvapich2_2.3rc1_intel_17.2/lib/x86_64/intel/ioapi/lib/libioapi.a(m3msg2.o): in function m3msg2_': m3msg2.F:(.text+0x20): undefined reference to__kmpc_global_thread_num’
ld: m3msg2.F:(.text+0x37): undefined reference to __kmpc_critical' ld: m3msg2.F:(.text+0x9e): undefined reference to__kmpc_end_critical’

I solved it by adding -qopenmp to the LIBS section of the Makefile
LIBS = -L$(IOAPI_ROOT)/lib -lioapi
-L$(NETCDF)/lib -lnetcdff -lnetcdf -qopenmp

I think the problem is that whatever flags are used to build ioapi need to be used for compiling MCIP and CMAQ. In my case I took a look at whether -qopenmp was used in the Makeinclude.* file for ioapi build, and found:

>grep openmp Makeinclude.Linux2_x86_64ifort
OMPFLAGS  = -qopenmp      # for Intel compilers, version 16 or later
OMPLIBS   = -qopenmp
#4

PnetCDF/MPI enabled I/O API should only be used with CMAQ; all others should use “vanilla” I/O API.
This is documented in the I/O API installation instructions, https://cjcoats.github.io/ioapi/AVAIL.html#build
though perhaps the wording might be improved (and strengthened).

In principle, a lot of effort has been expended to make it work anyway, but it does require substantial changes to compiler, compile-flags, and link-flags in Makefiles, etc.

#5

@cothatch and @lizadams –

As was noted by @cgnolte and @cjcoats, MCIP is a serial code, so there is no dependency on parallel processing. MCIP will not need a version of I/O API that is built for MPI or openMP. There are no plans to make MCIP into a parallelized code.

However, it seems like there is another disconnect somewhere in the build scripts for the entire CMAQ system, so this will need to be addressed. I don’t know if it’s possible to (at least temporarily) maintain both a parallel build of I/O API and a serial build of I/O API. I realize this is suboptimal.

I’m open to suggestions on how to make this work more seamlessly for the user community.

Tanya

#6

There are also dependencies on the netCDF build. You can use commands such as

nc-config --all
nf-config --all

To see how the C and Fortran versions of the netCDF library are built, respectively.

#7

I have found that gfortran will not compile if there is a symbol that is unresolved in the *.a files, even if the code doesn’t use it. Try adding the following

-Wl,--unresolved-symbols=ignore-in-object-files

For example:

LIBS    = -L$(IOAPI_ROOT) -lioapi  \
          -L$(NETCDF)/lib -lnetcdff -lnetcdf -lnetcdf -Wl,--unresolved-symbols=ignore-in-object-files

For more information see: https://stackoverflow.com/questions/5555632/can-gcc-not-complain-about-undefined-references

Bcon compilation
#8

The solution here is that you have different $BIN-directories for the two different library-versions. And you use one $BIN for CMAQ, and the other for everything else.

– Carlie

1 Like
#9

It looks like you are using IOAPI 3.2. Since MCIP is strictly a serial code (it can be parallelized easily), you can only link with regular IOAPI (@cjoats has mentioned it as vanilla IOAPI). In other words, when you build your IOAPI library, set BIN Linux2_x86_64ifort not Linux2_x86_64ifortmpi (assume you were using ifort compiler and same things applies to gcc or pgi compiler).

#10

Thank you all for your help.

I made another a serial build of I/O API as many of you suggested this resulted in the many of the same undefined references. However, I then included the -fopenmp library as @lizadams suggested and MCIP compiled correctly.

Thanks again

1 Like