Compilation Problem of Bidirectionally Coupled WRF-CMAQ Model

Hello, I have some questions about the compilation of coupled WRF-CMAQ. I would like to ask you for help. Because I cannot use git to download WRF normally during the ./bldit_cctm.csh process, so I tried to build WRF-CMAQ manually, but when I ran to the last step of compile em_real, wrf.exe and twoway_feedback.o were not generated. Can anyone tell me what the reason is? Thank you very much.
bldit_cctm.csh (31.9 KB)

wrf-cmaq_buildlog.log.txt (1.6 MB)

It looks like the error message that you received is:

time mpif90  -o wrf.exe  -O2 -ftree-vectorize -funroll-loops -w -ffree-form -ffree-line-length-none -fconvert=big-endian -frecord-marker=4     wrf.o ../main/module_wrf_top.o libwrflib.a libcmaqlib.a  -L/home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort -lioapi  /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/fftpack/fftpack5/libfftpack.a /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/io_grib1/libio_grib1.a /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/io_grib_share/libio_grib_share.a /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/io_int/libwrfio_int.a -L/home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/esmf_time_f90 -lesmf_time /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/RSL_LITE/librsl_lite.a /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/frame/module_internal_header_util.o /home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/frame/pack_utils.o -L/home/likeco/Build_WRF-CMAQ/WRF-CMAQ5.3/CMAQ-CMAQv5.3.3_17Aug2021/CCTM/scripts/BLD_WRFv4.3_CCTM_v533_gcc/external/io_netcdf -lwrfio_nf -L/home/likeco/Build_WRF/LIBRARIES/netcdf/lib -lnetcdff -lnetcdf
/bin/bash: /home/likeco/miniconda3/lib/libtinfo.so.6: no version information available (required by /bin/bash)
gfortran: error: libcmaqlib.a: 没有那个文件或目录
Command exited with non-zero status 1
0.03user 0.00system 0:00.04elapsed 104%CPU (0avgtext+0avgdata 3404maxresident)k
0inputs+0outputs (0major+4979minor)pagefaults 0swaps

I then looked to see if the libcmaqlib.a was successfully created earlier in the log file and found.

ar ru ../main/libcmaqlib.a se_bndy_copy_info_ext.o se_pe_info_ext.o se_comm_info_ext.o se_data_send_module.o se_domain_info_ext.o se_internal_util_module.o se_data_recv_module.o se_disp_info_ext.o se_subgrid_info_ext.o se_global_sum_module.o se_global_max_module.o se_global_min_module.o se_slice_module.o se_init_module.o se_bndy_copy_module.o se_comm_module.o se_reconfig_grid_info_ext.o se_reconfig_grid_module.o se_gather_module.o se_data_copy_module.o se_term_module.o se_twoway_comm_module.o se_util_module.o se_modules.o alloc_data_mod.o parutilio.o piomaps_mod.o get_write_map.o pio_init.o pm3err.o pm3exit.o pm3warn.o pshut3.o ptrwrite3.o pwrgrdd.o pwrite3.o subdmap.o wrsubdmap.o twoway_data_module.o RXNS_DATA_MODULE.o get_env_mod.o RUNTIME_VARS.o UTILIO_DEFN.o HGRD_DEFN.o twoway_sd_time_series.o PA_DEFN.o STM_VARS.o VGRD_DEFN.o LSM_MOD.o VDIFF_DATA.o GRID_CONF.o CGRID_SPCS.o BIDI_MOD.o STAGE_DATA.o MOSAIC_MOD.o depv_data_module.o biog_emis_param_module.o AEROMET_DATA.o AERO_DATA.o crop_data_module.o em_param_module.o EMIS_VARS.o lus_data_module.o UDTYPES.o LUS_DEFN.o BIOG_EMIS.o BEIS_DEFN.o stack_group_data_module.o STK_PRMS.o centralized_io_util_module.o PCGRID_DEFN.o PTBILIN.o centralized_io_module.o STK_EMIS.o ASX_DATA_MOD.o LTNG_DEFN.o PRECURSOR_DATA.o DUST_EMIS.o NH3_BIDI_MOD.o HGSIM.o STD_CONC.o twoway_rrtmg_aero_optics.o CSQY_DATA.o twoway_rrtmg_aero_optics.o CLOUD_OPTICS.o SOA_DEFN.o AERO_PHOTDATA.o AVG_CONC.o RXNS_FUNC_MODULE.o AERO_BUDGET.o STAGE_MOD.o DEPV_DEFN.o PTMET.o AEROSOL_CHEMISTRY.o SA_DEFN.o PHOT_MOD.o PAGRD_DEFN.o PA_IRR_CTL.o PA_IRR_module.o PISAM_DEFN.o MGEMIS.o SA_IRR_DEFN.o VDIFF_MAP.o WVEL_DEFN.o SSEMIS.o vertext_module.o STM_MODULE.o AERO_EMIS.o PT3D_DEFN.o EMIS_DEFN.o distr_env.o mpcomm_init.o twoway_cgrid_aerosol_spc_map_module.o twoway_header_data_module.o twoway_met_param_module.o twoway_util_module.o twoway_aqprep.o twoway_aqprep_util.o twoway_feedback.o twoway_init_env_vars.o advstep.o cmaq_main.o driver.o hveloc.o sciproc.o wr_aconc.o wr_cgrid.o wr_conc.o wr_init.o diffmsg.o flcheck.o grdcheck.o initscen.o load_cgrid.o opaconc.o opconc.o couplewrf.o advbc_map.o hadvppm.o hcontvel.o hppm.o rdbcon.o x_ppm.o y_ppm.o zfdbc.o vppm.o zadvppmwrf.o deform.o hcdiff3d.o hdiff.o rho_j.o VDIFF_DIAG.o SEDIMENTATION.o aero_sedv.o conv_cgrid.o eddyx.o matrix1.o opddep.o opddep_fst.o opddep_mos.o rddepv.o tri.o vdiffacmx.o vdiffproc.o cgrid_depv.o gas_depv_map.o opdepv_diag.o opdepv_fst.o opdepv_mos.o cropcal.o opemis.o tfabove.o tfbelow.o beis3.o checkmem.o chkgrid.o czangle.o getfline.o getparb.o hrno.o parsline.o tmpbeis.o wrdaymsg.o delta_zs.o fire_plmris.o openlayout.o oppt3d_diag.o plmris.o plsprd.o preplm.o write3_distr.o PHOT_MET_DATA.o PHOTOLYSIS_ALBEDO.o SEAS_STRAT_O3_MIN.o concld_prop_acm.o o3totcol.o opphot.o phot.o DEGRADE_SETUP_TOX.o hrdata_mod.o degrade_data.o degrade.o final_degrade.o find_degraded.o hrdriver.o hrg1.o hrg2.o hrg3.o hrg4.o hrinit.o hrprodloss.o hrrates.o hrsolver.o init_degrade.o PMDIAG_DATA.o aero_driver.o aero_subs.o coags.o getpar.o isocom.o isofwd.o isorev.o opapmdiag.o oppmdiag.o AQ_DATA.o acmcld.o aqchem.o aq_map.o cldproc_acm.o convcld_acm.o getalpha.o hlconst.o indexn.o opwdep.o rescld.o scavwdep.o PA_GLOBAL.o PA_IPRDEF.o PA_VARS.o PA_PARSE.o PA_IPRVARS.o pa_compmech.o pa_datagen.o pa_errcheck.o pa_getcoef.o pa_getcycle.o pa_getdesc.o pa_getfamily.o pa_getiprout.o pa_getirrout.o pa_getrxns.o pa_getrxnsum.o pa_init.o pa_mkhdr.o pa_molcloss.o pa_molcprod.o pa_output.o pa_read.o pa_report.o pa_setup_ipr.o pa_setup_irr.o pa_update.o pa_wrtpadefn.o op_sa.o sa_array_init.o sa_dim.o sa_matrix1.o sa_opwddep.o sa_tri.o SA_WRAP_AE.o wr_avg_sa.o wr_sa_cgrid.o wr_sa.o log_header.o cksummer.o findex.o lstepf.o setup_logdev.o subhdomain.o subhfile.o
ar: `u' 修饰符被忽略,因为 `D' 为默认(参见 `U')
ar: 正在创建 ../main/libcmaqlib.a
ar: twoway_feedback.o: 没有那个文件或目录
make[2]: [Makefile.twoway:419:cmaq] 错误 1 (已忽略)

If this was successful, you should see the message:

ar: creating ../main/libcmaqlib.a

I did notice that you are using an older version, so please consider using CMAQv5.4.0.4
and
WRF 4.5.2

Tutorial: CMAQ/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_WRF-CMAQ_Benchmark.md at main · USEPA/CMAQ · GitHub

hi lizadams,Thank you very much for your reply. I followed your instructions and upgraded WRF-CMAQ to the latest version for compilation and installation, but it still failed. This is my run log. But it is strange that I succeeded in compiling WRF or CMAQ separately. Can you tell me what the reason may be? Thank you very much.
wrf-cmaq_buildlog.txt (1.6 MB)

It seems based on the following error messages in your log file that you may have compiled I/O API with openmp but then did not provide an openmp link flag when building the twoway WRF/CMAQ model:

time mpif90 -o wrf.exe -O2 -ftree-vectorize -funroll-loops -w -ffree-form -ffree-line-length-none -fconvert=big-endian -frecord-marker=4 wrf.o …/main/module_wrf_top.o libwrflib.a libcmaqlib.a -L/home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort -lioapi /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/fftpack/fftpack5/libfftpack.a /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/io_grib1/libio_grib1.a /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/io_grib_share/libio_grib_share.a /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/io_int/libwrfio_int.a -L/home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/esmf_time_f90 -lesmf_time /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/RSL_LITE/librsl_lite.a /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/frame/module_internal_header_util.o /home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/frame/pack_utils.o -L/home/likeco/Build_WRF-CMAQ/CMAQ5.4.0/CMAQ-CMAQv5.4.0.4/CCTM/scripts/BLD_WRFv4.5_CCTM_v54_gcc/external/io_netcdf -lwrfio_nf -L/home/likeco/Build_WRF/LIBRARIES/netcdf/lib -lnetcdff -lnetcdf
/bin/bash: /home/likeco/miniconda3/lib/libtinfo.so.6: no version information available (required by /bin/bash)
/usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(name2fid.o): in function name2fid_': name2fid.f:(.text+0x1b): undefined reference to GOMP_critical_name_start’
/usr/bin/ld: name2fid.f:(.text+0x31): undefined reference to GOMP_critical_name_end' /usr/bin/ld: name2fid.f:(.text+0xb1): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(getefile.o): in function getefile_': getefile.F:(.text+0x27): undefined reference to GOMP_critical_name_start’
/usr/bin/ld: getefile.F:(.text+0x4b): undefined reference to GOMP_critical_name_end' /usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(interp3.o): in function master.0.interp3_‘:
interp3.F:(.text+0xdf): undefined reference to GOMP_critical_name_start' /usr/bin/ld: interp3.F:(.text+0xf9): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: interp3.F:(.text+0x17c): undefined reference to GOMP_critical_name_start' /usr/bin/ld: interp3.F:(.text+0x1a9): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: interp3.F:(.text+0x3bf): undefined reference to GOMP_critical_name_start' /usr/bin/ld: interp3.F:(.text+0x5ed): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: interp3.F:(.text+0x17f9): undefined reference to GOMP_critical_name_end' /usr/bin/ld: interp3.F:(.text+0x1cb1): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: interp3.F:(.text+0x20b4): undefined reference to GOMP_critical_name_end' /usr/bin/ld: interp3.F:(.text+0x2c4a): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(interp3.o):interp3.F:(.text+0x31d4): more undefined references to GOMP_critical_name_end' follow /usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(m3msg2.o): in function m3msg2_‘:
m3msg2.F:(.text+0x24): undefined reference to GOMP_critical_name_start' /usr/bin/ld: m3msg2.F:(.text+0xad): undefined reference to GOMP_critical_name_end’
/usr/bin/ld: /home/likeco/install_cmaq/libext/ioapi_3.1/Linux2_x86_64gfort/libioapi.a(m3msg2.o): in function `m3mesg_':

I don’t have firsthand experience with this, but it seems to be the same issue discussed here.

Based on that thread, the solution would seem to be to either rebuild I/O API without openmp or to provide the openmp link flag when building WRF/CMAQ. I admit that I don’t understand why this wouldn’t have been an issue when you successfully built offline CMAQ.

hi christian,Thank you for your reply. I guess the reason why I could compile CMAQ successfully before was that I set mylink-FLAG to -fopenmp in config_cmaq.csh. Then I recompiled the ioapi library as you said, and no longer used openmp to compile it. Finally, wrf.exe was successfully generated, but I did not find twoway_feedback.o in the main folder. Can you help me find out the reason? Thank you very much
wrf-cmaq_buildlog.txt (3.1 MB)

Hello Zhu, I’m glad to hear that using a consistent openmp / non-openmp setup between the I/O API library build and the twoway WRF-CMAQ model build resolved the link errors you were encountering.

Unfortunately, I’m not familiar enough with the twoway WRF-CMAQ model build process to answer your question about the absence of file twoway_feedback.o in the main folder. Hopefully others with more experience can weigh in.

hi christian Thank you for your guidance, which made me more familiar with the whole process. I also hope that someone can help me solve the remaining problems.