# Undefined references when compiling MCIP

**URL:** <https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630>\
**Category:** MCIP\
**Created:** [April 26, 2019, 5:17pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630 "2019-04-26T17:17:24Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![cothatch](https://avatars.discourse-cdn.com/v4/letter/c/51bf81/32.png) [@cothatch](https://forum.cmascenter.org/u/cothatch)\
**Post date:** [April 26, 2019, 5:17pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/1 "2019-04-26T17:17:24Z")

</div>

I am trying to compile MCIP, however I keep getting the following undefined references:

/home/colethatcher/ioapi-3.2/Linux4\_x86\_64gfortmpi/libioapi.a(wrmpigrd.o): In function `wrmpigrd_': wrmpigrd.F90:(.text+0x52): undefined reference to`mpi\_bcast\_’  
/home/colethatcher/ioapi-3.2/Linux4\_x86\_64gfortmpi/libioapi.a(modpdata.o): In function `__modpdata_MOD_pn_flag': modpdata.F90:(.text+0x350): undefined reference to`mpi\_allreduce\_’  
/home/colethatcher/ioapi-3.2/Linux4\_x86\_64gfortmpi/libioapi.a(modpdata.o): In function `__modpdata_MOD_pn_setup': modpdata.F90:(.text+0xeb0): undefined reference to`mpi\_comm\_rank\_’  
modpdata.F90:(.text+0xf0b): undefined reference to `mpi_comm_group_' modpdata.F90:(.text+0xf35): undefined reference to`mpi\_group\_incl\_’  
modpdata.F90:(.text+0xf5a): undefined reference to `mpi_comm_create_' modpdata.F90:(.text+0xf82): undefined reference to`mpi\_group\_excl\_’  
modpdata.F90:(.text+0xfa9): undefined reference to `mpi_comm_create_' modpdata.F90:(.text+0xfc9): undefined reference to`mpi\_comm\_dup\_’  
modpdata.F90:(.text+0x1c3f): undefined reference to `mpi_comm_dup_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function`pn\_wrint\_’:  
pn\_wrvars.F90:(.text+0xc49): undefined reference to `mpi_recv_' pn_wrvars.F90:(.text+0x1474): undefined reference to`mpi\_send\_’  
/home/colethatcher/ioapi-3.2/Linux4\_x86\_64gfortmpi/libioapi.a(pn\_wrvars.o): In function `pn_wrreal_': pn_wrvars.F90:(.text+0x2109): undefined reference to`mpi\_recv\_’  
pn\_wrvars.F90:(.text+0x2934): undefined reference to `mpi_send_' /home/colethatcher/ioapi-3.2/Linux4_x86_64gfortmpi/libioapi.a(pn_wrvars.o): In function`pn\_wrdble\_’:  
pn\_wrvars.F90:(.text+0x35c9): undefined reference to `mpi_recv_' pn_wrvars.F90:(.text+0x3df4): undefined reference to`mpi\_send\_’  
/home/colethatcher/ioapi-3.2/Linux4\_x86\_64gfortmpi/libioapi.a(pn\_wrvars.o): In function `pn_wrint8_': pn_wrvars.F90:(.text+0x4a89): undefined reference to`mpi\_recv\_’  
pn\_wrvars.F90:(.text+0x52b4): undefined reference to `mpi\_send\_’  
collect2: error: ld returned 1 exit status  
Makefile:182: recipe for target ‘mcip.exe’ failed

This implies that the appropriate libraries are not being linked in the makefile. Here is the relevant portion of my makefile;

LIBS = -L$(IOAPI\_ROOT)/Linux4\_x86\_64gfortmpi -lioapi   
-L$(NETCDF)/lib -lnetcdff -lnetcdf -lpnetcdf -lmpich -lgomp

I added -lpnetcdf, -lmpich, and -lgomp which corrected a bunch of other undefined references. I’ve only included the remaining errors after linking those 3 libraries. My problem is that the undefined subroutines should be linked in -lmpich library, but they’re not.

Any suggestions?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![cgnolte](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.cmascenter.org/cgnolte/32/53_2.png) [@cgnolte](https://forum.cmascenter.org/u/cgnolte)\
**Post date:** [April 26, 2019, 8:12pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/2 "2019-04-26T20:12:55Z")

</div>

You don’t say what other undefined references you were getting, but since MCIP is an entirely serial code, there is no need for MPICH, openMP, or pnetCDF.

---

<div class="post-metadata">

**Author:** ![lizadams](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@lizadams](https://forum.cmascenter.org/u/lizadams)\
**Post date:** [April 26, 2019, 9:52pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/3 "2019-04-26T21:52:38Z")

</div>

I had a similar problem with the ifort compiler.  
I was getting the following  
ld: /proj/ie/proj/CMAS/CMAQ/CMAQv5.3\_branch\_UNC3/mvapich2\_2.3rc1\_intel\_17.2/lib/x86\_64/intel/ioapi/lib/libioapi.a(m3msg2.o): in function `m3msg2_': m3msg2.F:(.text+0x20): undefined reference to`\_\_kmpc\_global\_thread\_num’  
ld: m3msg2.F:(.text+0x37): undefined reference to `__kmpc_critical' ld: m3msg2.F:(.text+0x9e): undefined reference to`\_\_kmpc\_end\_critical’

I solved it by adding -qopenmp to the LIBS section of the Makefile  
LIBS = -L$(IOAPI\_ROOT)/lib -lioapi   
-L$(NETCDF)/lib -lnetcdff -lnetcdf -qopenmp

I think the problem is that whatever flags are used to build ioapi need to be used for compiling MCIP and CMAQ. In my case I took a look at whether -qopenmp was used in the Makeinclude.\* file for ioapi build, and found:

```auto
>grep openmp Makeinclude.Linux2_x86_64ifort
OMPFLAGS = -qopenmp # for Intel compilers, version 16 or later
OMPLIBS = -qopenmp

```

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [April 27, 2019, 2:22pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/4 "2019-04-27T14:22:25Z")

</div>

PnetCDF/MPI enabled I/O API should _only be used with CMAQ_; all others should use “vanilla” I/O API.  
This is documented in the I/O API installation instructions, [https://cjcoats.github.io/ioapi/AVAIL.html#build](https://cjcoats.github.io/ioapi/AVAIL.html#build)  
though perhaps the wording might be improved (and strengthened).

In principle, a lot of effort has been expended to make it work anyway, but it does require substantial changes to compiler, compile-flags, and link-flags in `Makefile`s, etc.

---

<div class="post-metadata">

**Author:** ![tlspero](https://avatars.discourse-cdn.com/v4/letter/t/bb73d2/32.png) [@tlspero](https://forum.cmascenter.org/u/tlspero)\
**Post date:** [April 29, 2019, 12:58pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/5 "2019-04-29T12:58:24Z")

</div>

@cothatch and @lizadams –

As was noted by @cgnolte and @cjcoats, MCIP is a serial code, so there is no dependency on parallel processing. MCIP will not need a version of I/O API that is built for MPI or openMP. There are no plans to make MCIP into a parallelized code.

However, it seems like there is another disconnect somewhere in the build scripts for the entire CMAQ system, so this will need to be addressed. I don’t know if it’s possible to (at least temporarily) maintain both a parallel build of I/O API and a serial build of I/O API. I realize this is suboptimal.

I’m open to suggestions on how to make this work more seamlessly for the user community.

Tanya

---

<div class="post-metadata">

**Author:** ![lizadams](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@lizadams](https://forum.cmascenter.org/u/lizadams)\
**Post date:** [April 29, 2019, 5:26pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/6 "2019-04-29T17:26:50Z")

</div>

There are also dependencies on the netCDF build. You can use commands such as

```auto
nc-config --all
nf-config --all

```

To see how the C and Fortran versions of the netCDF library are built, respectively.

---

<div class="post-metadata">

**Author:** ![lizadams](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@lizadams](https://forum.cmascenter.org/u/lizadams)\
**Post date:** [April 30, 2019, 6:56pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/7 "2019-04-30T18:56:55Z")

</div>

I have found that gfortran will not compile if there is a symbol that is unresolved in the \*.a files, even if the code doesn’t use it. Try adding the following

```auto
-Wl,--unresolved-symbols=ignore-in-object-files

```

For example:

```auto
LIBS = -L$(IOAPI_ROOT) -lioapi \
          -L$(NETCDF)/lib -lnetcdff -lnetcdf -lnetcdf -Wl,--unresolved-symbols=ignore-in-object-files

```

For more information see: [https://stackoverflow.com/questions/5555632/can-gcc-not-complain-about-undefined-references](https://stackoverflow.com/questions/5555632/can-gcc-not-complain-about-undefined-references)

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [April 30, 2019, 9:29pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/8 "2019-04-30T21:29:07Z")

</div>

The solution here is that you have different $BIN-directories for the two different library-versions. And you use one $BIN for CMAQ, and the other for everything else.

– Carlie

---

<div class="post-metadata">

**Author:** ![wong.david-c](https://avatars.discourse-cdn.com/v4/letter/w/bcef8e/32.png) [@wong.david-c](https://forum.cmascenter.org/u/wong.david-c)\
**Post date:** [May 7, 2019, 2:24pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/9 "2019-05-07T14:24:02Z")

</div>

It looks like you are using IOAPI 3.2. Since MCIP is strictly a serial code (it can be parallelized easily), you can only link with regular IOAPI (@cjoats has mentioned it as vanilla IOAPI). In other words, when you build your IOAPI library, set BIN Linux2\_x86\_64ifort not Linux2\_x86\_64ifortmpi (assume you were using ifort compiler and same things applies to gcc or pgi compiler).

---

<div class="post-metadata">

**Author:** ![cothatch](https://avatars.discourse-cdn.com/v4/letter/c/51bf81/32.png) [@cothatch](https://forum.cmascenter.org/u/cothatch)\
**Post date:** [May 7, 2019, 11:25pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/10 "2019-05-07T23:25:15Z")

</div>

Thank you all for your help.

I made another a serial build of I/O API as many of you suggested this resulted in the many of the same undefined references. However, I then included the -fopenmp library as @lizadams suggested and MCIP compiled correctly.

Thanks again

---

<div class="post-metadata">

**Author:** ![pariksheet](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.cmascenter.org/pariksheet/32/211_2.png) [@pariksheet](https://forum.cmascenter.org/u/pariksheet)\
**Post date:** [July 25, 2019, 3:30pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/11 "2019-07-25T15:30:48Z")

</div>

Thanks to @cgnolte for pointing us in the correct direction that MCIP should not be pulling in parallel dependencies.

Unfortunately, trying to work around building MCIP with IOAPI compiled with parallel NetCDF using `-Wl,--unresolved-symbols=ignore-in-object-files` or `-fopenmp` doesn’t work for me either because I still get OpenMPI related errors:

```auto
gfortran -o mcip.exe -O3 -I/home/yul18051/CMAQ_Project/netcdf_both/include -I/home/yul18051/src/cmaq/5.2.1/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/ioapi-3.2-3emuycdwvyttrvy2i5fgnce26avrn7tm/include/fixed132 -I/home/yul18051/src/cmaq/5.2.1/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/ioapi-3.2-3emuycdwvyttrvy2i5fgnce26avrn7tm/lib -Wl,--unresolved-symbols=ignore-in-object-files mcip.o alloc_ctm.o alloc_met.o alloc_x.o bcldprc_ak.o blddesc.o chkwpshdr.o chkwrfhdr.o close_files.o collapx.o comheader.o comheader_lufrac.o comheader_mos.o comheader_soi.o cori.o dealloc_ctm.o dealloc_met.o dealloc_x.o detangle_soil_px.o e_aerk.o dynflds.o getgist.o getluse.o getmet.o getpblht.o getpsih.o getsdt.o getversion.o graceful_stop.o gridout.o init_io.o init_met.o init_x.o julian.o layht.o ll2xy_lam.o ll2xy_lam_sec.o ll2xy_lam_tan.o ll2xy_merc.o ll2xy_ps.o locate.o lucro.o mapfac_lam.o mapfac_merc.o mapfac_ps.o metcro.o metdot.o metgrid2ctm.o metvars2ctm.o mm5v3opts.o moscro.o pblsup.o ptemp.o pvs.o qsat.o rdmm5v3.o rdwrfem.o readnml.o readter.o refstate.o resistcalc.o setgriddefs.o setup.o setup_mm5v3.o setup_wrfem.o sfclayer.o soilcro.o statflds.o vertarys.o vertnhy.o vertnhy_wrf.o vstamp.o vtemp.o wind.o wrfemopts.o wrgdesc.o xy2ll_lam.o xy2ll_merc.o const_mod.o const_pbl_mod.o coord_mod.o date_time_mod.o date_pack_mod.o files_mod.o groutcom_mod.o luoutcom_mod.o luvars_mod.o mcipparm_mod.o mcoutcom_mod.o mdoutcom_mod.o metinfo_mod.o metvars_mod.o mosoutcom_mod.o soioutcom_mod.o vgrd_mod.o wrf_netcdf_mod.o xvars_mod.o sat2mcip_mod.o -L/home/yul18051/src/cmaq/5.2.1/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/ioapi-3.2-3emuycdwvyttrvy2i5fgnce26avrn7tm/lib -lioapi -L/home/yul18051/CMAQ_Project/netcdf_both/lib -lnetcdff -lnetcdf
/home/yul18051/src/cmaq/5.2.1/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/openmpi-3.1.4-kjjd2yn2ppk3ra6amsukrdtsen6tmubo/lib/libopen-pal.so.40: undefined reference to `hwloc_topology_dup'
collect2: error: ld returned 1 exit status
make[1]: *** [mcip.exe] Error 1
make[1]: Leaving directory `/home/yul18051/src/cmaq/5.2.1/CMAQ-5.2.1/PREP/mcip/src'
make: *** [all] Error 2
make: Leaving directory `/home/yul18051/src/cmaq/5.2.1/CMAQ-5.2.1/PREP/mcip/src'

```

To make people’s lives easier, I’ve contributed an [IOAPI package](https://spack.readthedocs.io/en/latest/package_list.html#ioapi) to [spack](https://spack.io) @cjcoats so it’s trivial to maintain two separate and consistent IOAPI dependency tree installations with and without `parallel-netcdf`, namely:

```auto
# Parallel IOAPI for CMAQ
spack install ioapi ^netcdf+parallel-netcdf
# Serial IOAPI for MCIP
spack install ioapi

```

I’m going to try compiling MCIP against non-parallel NETCDF dependent IOAPI and see if solves my MCIP linking failure.

---

<div class="post-metadata">

**Author:** ![CMAQ\_user](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@CMAQ\_user](https://forum.cmascenter.org/u/CMAQ_user)\
**Post date:** [July 29, 2019, 4:41am UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/12 "2019-07-29T04:41:44Z")

</div>

Not sure, what you mean by others in ’ PnetCDF/MPI enabled I/O API should _only be used with CMAQ_ ; all others should use “vanilla” I/O API.’, does that also include the following case?

I have built the ioapi3.2 without any mpi or parallel netcdf, and using it for making MCIP work to prepare CMAQ ready data from WRF. I am not using CMAQ (at this point), but only MCIP (v4.3) program on its own (git clone -b 4.3 [https://github.com/CMASCenter/MCIP.git](https://github.com/CMASCenter/MCIP.git)).

The mcip makefile for gfortran includes the lines [I am using gcc]  
#…gfortran  
FC = gfortran  
FFLAGS = -O3 -I$(NETCDF)/include -I$(IOAPI\_ROOT)/Linux2\_x86\_64gfort  
LIBS = -L$(IOAPI\_ROOT)/Linux2\_x86\_64gfort -lioapi   
-L$(NETCDF)/lib -lnetcdf -lnetcdff  
[note the order of lnetcdf and lnetcdff]

With this, issuing ‘make’, I get  
gfortran -o mcip.exe -O3 -I/home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf/include -I/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort mcip.o alloc\_ctm.o alloc\_met.o alloc\_x.o bcldprc\_ak.o blddesc.o chkwpshdr.o chkwrfhdr.o close\_files.o collapx.o comheader.o cori.o dealloc\_ctm.o dealloc\_met.o dealloc\_x.o e\_aerk.o dynflds.o getgist.o getluse.o getmet.o getpblht.o getsdt.o getversion.o graceful\_stop.o gridout.o init\_io.o init\_met.o init\_x.o julian.o layht.o ll2xy\_lam.o ll2xy\_lam\_sec.o ll2xy\_lam\_tan.o ll2xy\_merc.o ll2xy\_ps.o locate.o mapfac\_lam.o mapfac\_merc.o mapfac\_ps.o metcro.o metdot.o metgrid2ctm.o metvars2ctm.o mm5v3opts.o pblsup.o ptemp.o pvs.o qsat.o rdmm5v3.o rdwrfem.o readnml.o readter.o refstate.o resistcalc.o setgriddefs.o setup.o setup\_mm5v3.o setup\_wrfem.o sfclayer.o statflds.o vertarys.o vertnhy.o vertnhy\_wrf.o vstamp.o vtemp.o wind.o wrfemopts.o wrgdesc.o xy2ll\_lam.o xy2ll\_merc.o const\_mod.o const\_pbl\_mod.o coord\_mod.o date\_pack\_mod.o date\_time\_mod.o files\_mod.o groutcom\_mod.o luvars\_mod.o mcipparm\_mod.o mcoutcom\_mod.o mdoutcom\_mod.o metinfo\_mod.o metvars\_mod.o vgrd\_mod.o wrf\_netcdf\_mod.o xvars\_mod.o sat2mcip\_mod.o -L/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort -lioapi -L/home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf/lib -lnetcdf -lnetcdff  
/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort/libioapi.a(m3msg2.o): In function `m3msg2_': m3msg2.F:(.text+0x1e): undefined reference to`GOMP\_critical\_name\_start’  
m3msg2.F:(.text+0x8d): undefined reference to `GOMP_critical_name_end' /home/somedir/cmaq_user/model_libs/library_cmaq/ioapi3.2/Linux2_x86_64gfort/libioapi.a(m3msg2.o): In function`m3mesg\_’:  
m3msg2.F:(.text+0xd4): undefined reference to `GOMP_critical_name_start' m3msg2.F:(.text+0x14d): undefined reference to`GOMP\_critical\_name\_end’  
/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort/libioapi.a(m3msg2.o): In function `m3prompt_': m3msg2.F:(.text+0x1a4): undefined reference to`GOMP\_critical\_name\_start’  
m3msg2.F:(.text+0x29e): undefined reference to `GOMP_critical_name_end' . . . nf_varsio.F90:(.text+0x1714): undefined reference to`nc\_inq\_varndims’  
nf\_varsio.F90:(.text+0x1816): undefined reference to `nc_get_vars_double' /home/somedir/cmaq_user/model_libs/library_cmaq/ioapi3.2/Linux2_x86_64gfort/libnetcdff.a(nf_varsio.o): In function`nf\_get\_vars\_’:  
nf\_varsio.F90:(.text+0x18a5): undefined reference to `nc_inq_varndims' nf_varsio.F90:(.text+0x19a7): undefined reference to`nc\_get\_vars’  
collect2: error: ld returned 1 exit status  
make[1]: \*\*\* [mcip.exe] Error 1  
make[1]: Leaving directory `/home/somedir/cmaq_user/mcip_test/MCIP/src' make[1]: Leaving directory`/home/somedir/cmaq\_user/mcip\_test/MCIP/src’  
make: \*\*\* [all] Error 2

#### However, if the order of lnetcdf and lnetcdff is just reversed, the I am getting the following:

gfortran -o mcip.exe -O3 -I/home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf/include -I/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort mcip.o alloc\_ctm.o alloc\_met.o alloc\_x.o bcldprc\_ak.o blddesc.o chkwpshdr.o chkwrfhdr.o close\_files.o collapx.o comheader.o cori.o dealloc\_ctm.o dealloc\_met.o dealloc\_x.o e\_aerk.o dynflds.o getgist.o getluse.o getmet.o getpblht.o getsdt.o getversion.o graceful\_stop.o gridout.o init\_io.o init\_met.o init\_x.o julian.o layht.o ll2xy\_lam.o ll2xy\_lam\_sec.o ll2xy\_lam\_tan.o ll2xy\_merc.o ll2xy\_ps.o locate.o mapfac\_lam.o mapfac\_merc.o mapfac\_ps.o metcro.o metdot.o metgrid2ctm.o metvars2ctm.o mm5v3opts.o pblsup.o ptemp.o pvs.o qsat.o rdmm5v3.o rdwrfem.o readnml.o readter.o refstate.o resistcalc.o setgriddefs.o setup.o setup\_mm5v3.o setup\_wrfem.o sfclayer.o statflds.o vertarys.o vertnhy.o vertnhy\_wrf.o vstamp.o vtemp.o wind.o wrfemopts.o wrgdesc.o xy2ll\_lam.o xy2ll\_merc.o const\_mod.o const\_pbl\_mod.o coord\_mod.o date\_pack\_mod.o date\_time\_mod.o files\_mod.o groutcom\_mod.o luvars\_mod.o mcipparm\_mod.o mcoutcom\_mod.o mdoutcom\_mod.o metinfo\_mod.o metvars\_mod.o vgrd\_mod.o wrf\_netcdf\_mod.o xvars\_mod.o sat2mcip\_mod.o -L/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort -lioapi -L/home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf/lib -lnetcdff -lnetcdf  
/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort/libioapi.a(m3msg2.o): In function `m3msg2_': m3msg2.F:(.text+0x1e): undefined reference to`GOMP\_critical\_name\_start’  
m3msg2.F:(.text+0x8d): undefined reference to `GOMP_critical_name_end' /home/somedir/cmaq_user/model_libs/library_cmaq/ioapi3.2/Linux2_x86_64gfort/libioapi.a(m3msg2.o): In function`m3mesg\_’:  
m3msg2.F:(.text+0xd4): undefined reference to `GOMP_critical_name_start' m3msg2.F:(.text+0x14d): undefined reference to`GOMP\_critical\_name\_end’  
.  
.  
.  
/home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2/Linux2\_x86\_64gfort/libioapi.a(wratt3.o): In function `master.0.wratt3_': wratt3.F90:(.text+0x114): undefined reference to`GOMP\_critical\_name\_start’  
wratt3.F90:(.text+0x191): undefined reference to `GOMP_critical_name_end' wratt3.F90:(.text+0xa47): undefined reference to`GOMP\_critical\_name\_end’  
wratt3.F90:(.text+0xe0b): undefined reference to `GOMP_critical_name_end' collect2: error: ld returned 1 exit status make[1]: *** [mcip.exe] Error 1 make[1]: Leaving directory`/home/somedir/cmaq\_user/mcip\_test/MCIP/src’  
make: \*\*\* [all] Error 2

#### Any hints to get out of it?

Thank you.

---

<div class="post-metadata">

**Author:** ![lizadams](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@lizadams](https://forum.cmascenter.org/u/lizadams)\
**Post date:** [July 29, 2019, 10:10am UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/13 "2019-07-29T10:10:30Z")

</div>

You are correct to switch the order of the netcdf libraries to: lnetcdff and lnetcdf (note the order is dependent on what version of netcdf you are using)  
The next step is to add the flag -qopenmp to the link step, and that will resolve the GOMP\_critical\_name\_end and other undefined references.

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [July 29, 2019, 9:16pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/14 "2019-07-29T21:16:23Z")

</div>

That’s -`fopenmp` for _gfortran_; -qopenmp is for Intel _ifort_ version 16 or later.

---

<div class="post-metadata">

**Author:** ![pariksheet](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.cmascenter.org/pariksheet/32/211_2.png) [@pariksheet](https://forum.cmascenter.org/u/pariksheet)\
**Post date:** [July 30, 2019, 5:14pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/15 "2019-07-30T17:14:17Z")

</div>

In case it’s useful to anyone trying to compile MCIP, I’ve uploaded my 2 shell scripts here:

> **[UConn-HPC/cmaq-spack](https://github.com/UConn-HPC/cmaq-spack)**
>
> Compile CMAQ's cctm, bcon, icon and mcip with spack - UConn-HPC/cmaq-spack

---

<div class="post-metadata">

**Author:** ![CMAQ\_user](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@CMAQ\_user](https://forum.cmascenter.org/u/CMAQ_user)\
**Post date:** [July 30, 2019, 5:34pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/16 "2019-07-30T17:34:11Z")

</div>

Thank you, but where should I put the ‘-fopenmp’, There is no place withing MCIP for that? The Makefile from MCIP/src has the following main part

.SUFFIXES:  
.SUFFIXES: .o .f90 .F90

MODEL = mcip.exe  
#NETCDF = /home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf  
NETCDF = /home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf  
#NETCDF = /home/somedir/cmaq\_user/model\_libs/library\_cmaq/netcdf  
IOAPI\_ROOT = /home/somedir/cmaq\_user/model\_libs/library\_cmaq/ioapi3.2

#…gfortran  
FC = gfortran  
FFLAGS = -O3 -I$(NETCDF)/include -I$(IOAPI\_ROOT)/Linux2\_x86\_64gfort  
LIBS = -L$(IOAPI\_ROOT)/Linux2\_x86\_64gfort -lioapi   
-L$(NETCDF)/lib -lnetcdff -lnetcdf

# -L$(PVM)/lib -lpvm3

DEFS =

MODULES =  
const\_mod.o   
.  
.  
.  
sat2mcip\_mod.o

OBJS =  
mcip.o   
valloc\_ctm.o   
.  
.  
.  
xy2ll\_merc.o

all:  
@(MAKE) (MODULES)  
@(MAKE) (MODEL)

(MODEL): (OBJS)  
(FC) -o (MODEL) (FFLAGS) (OBJS) (MODULES) (LIBS)

(OBJS): (MODULES)

.f90.o:  
(FC) (FFLAGS) (INCLUDES) -c \<

.F90.o:  
(FC) (FFLAGS) (INCLUDES) -c \< $(DEFS)

clean:  
rm -f \*.o \*.mod \*.il $(MODEL)

#### So, where does that linker flag (-qopenmp or -fopenmp) go? [just a reminder, I am using MCIP on its own, no relation with CMAQ at this point, and I don’t see any place for mpi too!]

Thank you.

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [July 30, 2019, 5:36pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/17 "2019-07-30T17:36:34Z")

</div>

at the start of the right-hand-side for LIBS

---

<div class="post-metadata">

**Author:** ![CMAQ\_user](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@CMAQ\_user](https://forum.cmascenter.org/u/CMAQ_user)\
**Post date:** [July 30, 2019, 5:49pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/18 "2019-07-30T17:49:30Z")

</div>

Thanks for the info, but ‘module purge’ not working in our linux cluster. No command ‘module’, and not sure how to build it from source!

---

<div class="post-metadata">

**Author:** ![CMAQ\_user](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@CMAQ\_user](https://forum.cmascenter.org/u/CMAQ_user)\
**Post date:** [July 30, 2019, 5:53pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/19 "2019-07-30T17:53:17Z")

</div>

Thank you for the quick reply, really appreciate!! Seems working and will come back if any issue arises.

---

<div class="post-metadata">

**Author:** ![CMAQ\_user](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@CMAQ\_user](https://forum.cmascenter.org/u/CMAQ_user)\
**Post date:** [July 30, 2019, 8:11pm UTC](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630/20 "2019-07-30T20:11:34Z")

</div>

I got passed that step, but now stuck at running mcip; The error is

MCIP\>./run.mcip  
Tue Jul 30 20:07:37 UTC 2019  
rm: No match.

```
 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).                                 
 Copyright (C) 1992-2002 MCNC,                                   
 (C) 1992-2013 Carlie J. Coats, Jr.,                             
 (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and   
 (C) 2014-2019 UNC Institute for the Environment.                
 Released under the GNU LGPL License, version 2.1. See URL     
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.2: $Id: init3.F90 120 2019-06-21 14:18:20Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version 4.4.1 of Jul 28 2019 23:01:39 $
  
  
 EXECUTION_ID: mcip

```

==============================================================================

```
              US EPA COMMUNITY MULTISCALE AIR QUALITY MODEL
                METEOROLOGY-CHEMISTRY INTERFACE PROCESSOR

                       MCIP V4.3 FROZEN 11/06/2015

```

==============================================================================

* * *

\*\*\* SUBROUTINE: SETUP  
\*\*\* UNKNOWN OR UNSUPPORTED WRF OUTPUT VERSION  
\*\*\* VERSION = OUTPUT FROM WRF V4.0 MODEL

* * *

```
 *** ERROR ABORT in subroutine SETUP
 ABNORMAL TERMINATION IN SETUP
 Date and time 0:00:00 June 1, 2019 (2019152:000000)

```

Error running mcip

##### Can anybody hint? Is WRF v 4.0 not supported?

Thank you.

[Next page](https://forum.cmascenter.org/t/undefined-references-when-compiling-mcip/630.md?page=2)