Icon and bcon get stuck while running for the single day benchmark

CMAQ Benchmark Issues and Errors
1

Hello, I’m using the CMAQ5.2. When I run the run_icon.csh and run_bcon.csh for the single day benchmark, the program get stuck with no specific error. Would you please help me with that problem? Last part of the run_icon.log is as follow:

/public/software/CMAQ5.2/new-bulid/PREP/icon/scripts/BLD_ICON_v52_profile_intel/ICON_v52_profile.exe

 >>--->> WARNING in subroutine NCPOPT
 notCDF call to NCPOPT

 
 >>--->> WARNING in subroutine NF_INQ_LIBVERS
 notCDF call to NF_INQ_LIBVERS
 
                                                                 
 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).  Copyright (C) 1992-2002 MCNC,  
 (C) 1992-2013 Carlie J. Coats, Jr., and (C) 2003-2012 Baron     
 Advanced Meteorological Systems, LLC and released under the     
 GNU LGPL  License, version 2.1.  See URL                        
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.1: $Id:: init3.F 321 2016-02-26 16:00:20Z coats                $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version notCDF version 1.0
  
  
 EXECUTION_ID: ICON_v52_profile.exe
 Value for GRID_NAME:  'SE52BENCH'
 Value for GRID_NAME:  'SE52BENCH'

 File "GRIDDESC" opened for input on unit:  99
 /public/software/CMAQ5.2/new-bulid/data/SE52BENCH/single_day/cctm_input/met/mcip/GRIDDESC

 Value for NOTCDF_FATAL not defined;returning default:   TRUE

The output of bcon is similar.

2

It looks like you have selected the “notCDF” version of netCDF, and the default value of the environment variable “NOTCDF_FATAL” is TRUE. That suggests to me the program is aborting, though there should be an error message indicating this.

I recommend you recompile IOAPI with a suitable version of the netCDF library.

3

thank you for your reply! I have recompiled the IOAPI and now the problem of NOTCDF seems to have been solved. However, another problem emerged, and the program is still stuck. Last part of the run_icon.log is as follow:

 ioapi-3.2: $Id: init3.F90 98 2018-04-05 14:35:07Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 MPI/PnetCDF parallel-I/O enabled
 PnetCDF Library Version "1.10.0 of  2 Jul 2018"
 netCDF version 4.6.1 of Sep 15 2018 15:08:23 $

 EXECUTION_ID: ICON_v52_profile.exe
 Value for GRID_NAME:  'SE52BENCH'
 Value for GRID_NAME:  'SE52BENCH'

 File "GRIDDESC" opened for input on unit:  99
 /public/software/CMAQ5.2/new-bulid/data/SE52BENCH/single_day/cctm_input/met/mcip/GRIDDESC

 "MODPDATA:PN_SETUP":  Version
 $Id: modpdata.F90 1 2017-06-10 18:05:20Z coats $
 Value for NPCOL_NPROW not defined; returning defaultval ':  '                '
 Value for NPCOL_NPROW not defined; returning defaultval ':  '                '

I add setenv NPCOL_NPROW “1 1” to the run script and try again, then the program runs and exits, with no data in the output directory. Last part of the run_icon.log is as follow:

ioapi-3.2: Id: init3.F90 98 2018-04-05 14:35:07Z coats
Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
MPI/PnetCDF parallel-I/O enabled
PnetCDF Library Version “1.10.0 of 2 Jul 2018”
netCDF version 4.6.1 of Sep 15 2018 15:08:23 $

 EXECUTION_ID: ICON_v52_profile.exe
 Value for GRID_NAME:  'SE52BENCH'
 Value for GRID_NAME:  'SE52BENCH'

 File "GRIDDESC" opened for input on unit:  99
 /public/software/CMAQ5.2/new-bulid/data/SE52BENCH/single_day/cctm_input/met/mcip/GRIDDESC

 "MODPDATA:PN_SETUP":  Version
 $Id: modpdata.F90 1 2017-06-10 18:05:20Z coats $
 Value for NPCOL_NPROW:  '1 1'
 Value for NPCOL_NPROW:  '1 1'
 Processor-grid dimensions:  COLS=      1  ROWS=      1
 Value for GRID_NAME:  'SE52BENCH'
 Value for GRID_NAME:  'SE52BENCH'
 MODPDATA:PN_SETUP  Model-grid dimensions:  NCOLS=    100  NROWS=     80

Attempting to use an MPI routine before initializing MPI
0.021u 0.031s 0:00.07 71.4% 0+0k 0+0io 1pf+0w
exit ( )

Both ICON and BCON are single-processor program, but the libraries I compiled them with, such as netCDF, IOAPI and HDF5, are all parallel-enabled, and even ICON and BCON themselves are compiled using mpiifort to keep consistent with the libraries. Does this cause the problem? If this is indeed the cause of the problem, then I have to recompile all the libraries, from hdf5 to IOAPI, with no MPI, but what about the CCTM? Do I have to compile two kind of libraries, with and without MPI, and compile ICON and BCON with seriel libraries and CCTM with parallel ones?
Thank you in advance for your help!

4

When you say netCDF is parallel-enabled, do you mean you are using pNetCDF and IOAPI compiled with pNetCDF.
I have not used that. I am not sure to what extent the CMAQ programs (ICON, BCON, CCTM) work with pNetCDF. I would think they are compatible, but relatively few people have done that.
To clarify: it is possible to run CMAQ in parallel using IOAPI and regular netCDF.

For a new user, I would recommend compiling a plain vanilla (“no coupling”) version of IOAPI. That can be challenging enough… pay special attention to the notes about compiling with netCDF4.
https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#build

1 Like
5

Thank you very much! I did use PnetCDF. I have recompiled all the relative libraries and now ICON and BCON work normally. There are still some error in the compilation of CCTM, I’d continue to try it tomorror.
Thank you again!

1 Like
6

From the I/O API manual (https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#build or https://cjcoats.github.io/ioapi/AVAIL.html#build):

  • I/O API 3.2 only: If you want to enable the PnetCDF distributed I/O configuration (for CMAQ), cd ioapi and then cp Makefile.pncf Makefile and setenv environment variable BIN to one of the supported distributed-I/O types Linux2_x86_64gfortmpi, Linux2_x86_64ifortmpi,Linux2_x86_64pgmpi, Linux2_x86_64sunmpi
    NOTES
  1. PnetCDF enabled I/O API libraries are not compatible with “normal” libraries. Make sure you keep them separate.
    You will want to build both, since the PnetCDF enabled libraries are for CMAQ only.
  2. What is called mpif90 is really a script that understands about an underlying compiler-set (e.g., Intel ifort/icc, PGI pgf90/pgcc, etc.), a set of (MPI-installation specific) INCLUDE files, and a set of $quot;.a" library-files. These are not compatible from MPI-implementation to MPI-implementation (OpenMPI, MVAPICH, etc.) nor from underlying compiler-set to underlying compiler-set .
  3. There may even be compatibility problems between different versions of the “same” implementation: This author has found that OpenMPI-1.4.x is not even source-code compatible with OpenMPI-1.6.x ;-(
  4. This part of the I/O API source code had to be hacked badly to deal with the above.

So you want separate MPI enabled libraries for CMAQ, and non-MPI libraries for everything else…

Carlie J. Coats, Jr., Ph.D. cjcoats@email.unc.edu
Senior Software Engineer I/O API Author/Maintainer
Center for Environmental Modeling for Policy Development,
UNC Institute for the Environment www.ie.unc.edu
100 Europa Dr., Suite 490 Rm 4051 / Campus Box 1105 919.843.5951
Chapel Hill, NC 27599-1105 Fax 919.966.9920

1 Like
7

Hi
I want to benchmark CMAQ 5.2.1. I just ran the CCTM. Is it necessary to run BCON and ICON?

8

For the MPI/PnetCDF enabled version of the I/O API, pnetcdf is invoked only for files with setenv’s of the form

setenv lname MPI:/some/path

and the code has been very carefully structured (unlike the original submissions by David Wong) to behave “normally” (e.g., using netCDF operations) for all other files. It was a major effort to ensure that the code did not “go off into never-never land” for things other than the cross-point-dimensioned GRIDDED files for which the pnetCDF operations are defined.

FWIW.

9

@zyz

Hello,

Please, I need a single day benchmark data for CMAQ. Can you please send it to me or how can I get it downloaded? Thanks

Catalyst

10

Hi Catalyst

You can download it from this link:
https://drive.google.com/drive/folders/0B3Xmjif6GtQZYmtkczZkM0dndnc

1 Like
11

Hello,
Thanks for your prompt response.

I want to install IOAPI so that I can compile and run CMAQ but I have been facing problems with the installation of IOAPI-3.2. Please, can you send me the installation procedures (the simplified one because the one I saw on this site is somehow complex and confusing) so that I can follow it? Thanks a lot, I sincerely appreciate your kind gesture.

12

Please take a look a the following instructions.

https://github.com/USEPA/CMAQ/blob/5.2.1/DOCS/Tutorials/CMAQ_GettingStarted.md
1 Like
13

Hi Catalyst

I had this problem too. I changed some codes, but now I can’t run CMAQ and I want to install CMAQ in another system.

14

Hello LIzadams,

This is what I got while installing the IOAPI, kindly help me out. Thanks. Check it below:

-Precision-Tower-3620:~/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/ioapi% make
cd /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort; ar rsv libioapi.a m3utilio.o modgctp.o modwrfio.o modmpasfio.o modatts3.o modncfio.o modpdata.o
/bin/sh: 1: cd: can’t cd to /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort
r - m3utilio.o
r - modgctp.o
r - modwrfio.o
r - modmpasfio.o
r - modatts3.o
r - modncfio.o
r - modpdata.o
cd /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort; ar rsv libioapi.a bilin.o bmatvec.o chkbuf3.o ckdesc3.o ckfile3.o ckgeom.o ckname.o crtbuf3.o currstep.o dble2real.o dbllist.o dmatvec.o dscgrid.o dt2str.o filchk3.o find1.o find2.o find3.o find4.o findc.o findr1.o findr2.o findr3.o findr4.o flush3.o gcd.o gctp.o getdble.o getmenu.o getnum.o getreal.o getstr.o getyn.o grdchk3.o gridops.o hhmmss.o index1.o initblk3.o intg2real.o intlist.o ioparms3.o lambert.o lblank.o len2.o ll2utm.o locat1.o locat2.o locat3.o locat4.o locatc.o locatr1.o locatr2.o locatr3.o locatr4.o lustr.o m3warn.o name2fid.o pcoef.o pgrdsum.o pmatvec.o poly.o promptdfile.o promptffile.o promptgrid.o promptmfile.o rdbndary.o rdbuf3.o rdcustom.o rdgrdded.o readsmet.o realist.o scanint.o setsphere.o sec2time.o str2dble.o skipl.o smatvec.o splitline.o str2int.o str2real.o strlist.o synchtao.o time2sec.o trimlen.o ungridb.o ungridi.o upcase.o utm2ll.o wrbndary.o wrbuf3.o wrcustom.o wrgrdded.o xtbuf3.o year4.o chkfil3.o cktflag3.o crdict3.o crtkf.o kfindx.o kfopen.o m3abort.o modgctp.o opnlist3.o rddict3.o rdiddata.o rdsmatrx.o runspec.o sync3.o wrdict3.o wriddata.o wrsmatrx.o xtract3.o
/bin/sh: 1: cd: can’t cd to /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort
r - bilin.o
r - bmatvec.o
r - chkbuf3.o
r - ckdesc3.o
r - ckfile3.o
r - ckgeom.o
r - ckname.o
r - crtbuf3.o
r - currstep.o
r - dble2real.o
r - dbllist.o
r - dmatvec.o
r - dscgrid.o
r - dt2str.o
r - filchk3.o
r - find1.o
r - find2.o
r - find3.o
r - find4.o
r - findc.o
r - findr1.o
r - findr2.o
r - findr3.o
r - findr4.o
r - flush3.o
r - gcd.o
r - gctp.o
r - getdble.o
r - getmenu.o
r - getnum.o
r - getreal.o
r - getstr.o
r - getyn.o
r - grdchk3.o
r - gridops.o
r - hhmmss.o
r - index1.o
r - initblk3.o
r - intg2real.o
r - intlist.o
r - ioparms3.o
r - lambert.o
r - lblank.o
r - len2.o
r - ll2utm.o
r - locat1.o
r - locat2.o
r - locat3.o
r - locat4.o
r - locatc.o
r - locatr1.o
r - locatr2.o
r - locatr3.o
r - locatr4.o
r - lustr.o
r - m3warn.o
r - name2fid.o
r - pcoef.o
r - pgrdsum.o
r - pmatvec.o
r - poly.o
r - promptdfile.o
r - promptffile.o
r - promptgrid.o
r - promptmfile.o
r - rdbndary.o
r - rdbuf3.o
r - rdcustom.o
r - rdgrdded.o
r - readsmet.o
r - realist.o
r - scanint.o
r - setsphere.o
r - sec2time.o
r - str2dble.o
r - skipl.o
r - smatvec.o
r - splitline.o
r - str2int.o
r - str2real.o
r - strlist.o
r - synchtao.o
r - time2sec.o
r - trimlen.o
r - ungridb.o
r - ungridi.o
r - upcase.o
r - utm2ll.o
r - wrbndary.o
r - wrbuf3.o
r - wrcustom.o
r - wrgrdded.o
r - xtbuf3.o
r - year4.o
r - chkfil3.o
r - cktflag3.o
r - crdict3.o
r - crtkf.o
r - kfindx.o
r - kfopen.o
r - m3abort.o
r - modgctp.o
r - opnlist3.o
r - rddict3.o
r - rdiddata.o
r - rdsmatrx.o
r - runspec.o
r - sync3.o
r - wrdict3.o
r - wriddata.o
r - wrsmatrx.o
r - xtract3.o
cd /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort; ar rsv libioapi.a cbarnes1.o cbarnesN.o check3.o crlf.o currec.o daymon.o ddtvar3.o ddtvar3v.o getdate.o getdfile.o getefile.o getffile.o initlog3.o interp3.o isdstime.o jstep3.o julian.o junit.o m3err.o m3exit.o m3msg2.o mmddyy.o nextime.o read3.o read4d.o secsdiff.o wkday.o write3.o write4d.o yr2day.o close3.o crtfil3.o desc3.o init3.o inqatt3.o kfread.o kfwrite.o open3.o opnfil3.o opnkf.o opnlog3.o pn_crtfil3.o pn_opnfil3.o pn_wrvars.o rdatt3.o rdgrnest.o rdprofil.o rdtflag.o rdvars.o shut3.o syncfid.o wratt3.o wrgrnest.o wrmpigrd.o wrpatch.o wrprofil.o wrtflag.o wrvars.o
/bin/sh: 1: cd: can’t cd to /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort
r - cbarnes1.o
r - cbarnesN.o
r - check3.o
r - crlf.o
r - currec.o
r - daymon.o
r - ddtvar3.o
r - ddtvar3v.o
r - getdate.o
r - getdfile.o
r - getefile.o
r - getffile.o
r - initlog3.o
r - interp3.o
r - isdstime.o
r - jstep3.o
r - julian.o
r - junit.o
r - m3err.o
r - m3exit.o
r - m3msg2.o
r - mmddyy.o
r - nextime.o
r - read3.o
r - read4d.o
r - secsdiff.o
r - wkday.o
r - write3.o
r - write4d.o
r - yr2day.o
r - close3.o
r - crtfil3.o
r - desc3.o
r - init3.o
r - inqatt3.o
r - kfread.o
r - kfwrite.o
r - open3.o
r - opnfil3.o
r - opnkf.o
r - opnlog3.o
r - pn_crtfil3.o
r - pn_opnfil3.o
r - pn_wrvars.o
r - rdatt3.o
r - rdgrnest.o
r - rdprofil.o
r - rdtflag.o
r - rdvars.o
r - shut3.o
r - syncfid.o
r - wratt3.o
r - wrgrnest.o
r - wrmpigrd.o
r - wrpatch.o
r - wrprofil.o
r - wrtflag.o
r - wrvars.o
cd /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort; ar rsv libioapi.a bufint3.o check3c.o close3c.o currstepc.o daymonc.o ddtvar3c.o desc3c.o dscgridc.o dt2strc.o envgets.o filchk3c.o findsc.o get_endian.o getdfilec.o getdttime.o getefilec.o hhmmssc.o init3c.o inqatt3c.o interp3c.o iobin3.o julianc.o locatsc.o m3errc.o m3exitc.o m3mesgc.o m3warnc.o mmddyyc.o nameval.o nextimec.o open3c.o rdatt3c.o read3c.o read4dc.o rmfile.o sec2timec.o secsdiffc.o shut3c.o sleep3.o sortic.o sortir.o sortis.o sync3c.o systemf.o time2secc.o wkdayc.o wratt3c.o write3c.o write4dc.o xtract3c.o iocpl.o iocplf2c.o
/bin/sh: 1: cd: can’t cd to /home/catalyst/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort
r - bufint3.o
r - check3c.o
r - close3c.o
r - currstepc.o
r - daymonc.o
r - ddtvar3c.o
r - desc3c.o
r - dscgridc.o
r - dt2strc.o
r - envgets.o
r - filchk3c.o
r - findsc.o
r - get_endian.o
r - getdfilec.o
r - getdttime.o
r - getefilec.o
r - hhmmssc.o
r - init3c.o
r - inqatt3c.o
r - interp3c.o
r - iobin3.o
r - julianc.o
r - locatsc.o
r - m3errc.o
r - m3exitc.o
r - m3mesgc.o
r - m3warnc.o
r - mmddyyc.o
r - nameval.o
r - nextimec.o
r - open3c.o
r - rdatt3c.o
r - read3c.o
r - read4dc.o
r - rmfile.o
r - sec2timec.o
r - secsdiffc.o
r - shut3c.o
r - sleep3.o
r - sortic.o
r - sortir.o
r - sortis.o
r - sync3c.o
r - systemf.o
r - time2secc.o
r - wkdayc.o
r - wratt3c.o
r - write3c.o
r - write4dc.o
r - xtract3c.o
r - iocpl.o
r - iocplf2c.o
catalyst-Precision-Tower-3620:~/Desktop/Build_WRF/LIBRARIES/ioapi-3.2/ioapi%

15

You seem to have missed the

make dir

part of the build-instructions. Go to the ioapi directory, and issue this command before you do your make.

Carlie J. Coats, Jr., Ph.D.                    cjcoats@email.unc.edu
Senior Software Engineer                   I/O API Author/Maintainer
Center for Environmental Modeling for Policy Development,
UNC Institute for the Environment                     www.ie.unc.edu
100 Europa Dr., Suite 490 Rm 4051 / Campus Box 1105     919.843.5951
Chapel Hill, NC 27599-1105                          Fax 919.966.9920
16

Thanks for your response. I have resolved it with the assistance of lizadams.

17

Hello,

Below is what I got while compiling MCIP:

collect2: error: ld returned 1 exit status
Makefile:131: recipe for target ‘mcip.exe’ failed
make[1]: *** [mcip.exe] Error 1
make[1]: Leaving directory ‘/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/PREP/mcip/src’
Makefile:127: recipe for target ‘all’ failed
make: *** [all] Error 2
catalyst-Precision-Tower-3620:~/Desktop/Build_WRF/CMAQ-5.2/PREP/mcip/src%

Kindly help me out. Thanks.

18

I have resolved this issue and mcip now compiled correctly. Thanks. However, in building CCTM, I got this error:
st/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/modules -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/ioapi/include_files -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/mpi -I. se_data_send_module.f
se_data_send_module.f:84: Error: Can’t open included file ‘mpif.h’
Makefile:414: recipe for target ‘se_data_send_module.o’ failed
make: *** [se_data_send_module.o] Error 1
ERROR while running make command

else if ( 0 ) then
endif
mv Makefile Makefile.gcc7.3.0
if ( -e Makefile.gcc7.3.0 && -e Makefile ) rm Makefile
ln -s Makefile.gcc7.3.0 Makefile
if ( 0 != 0 ) then
if ( -e /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/CCTM/scripts/BLD_CCTM_v52_gcc7.3.0/CCTM_v52.cfg ) then
mv CCTM_v52.cfg.bld /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/CCTM/scripts/BLD_CCTM_v52_gcc7.3.0/CCTM_v52.cfg
exit

Kindly help me out. Thanks.

19

Be sure that the path to the include files that you specify contains the mpi.h file
use the command

ls -lrt /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/mpi

to see if mpi.h is in that directory, if not, you need to find mpi.h and use that path after the -I command.

 -I /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/lib/x86_64/gcc7.3.0/mpi

When I load a module on our server using the following command

module load openmpi_3.0.0/gcc_6.3.0

It sets up the environment variables such as the LD_LIBRARY_PATH and INCLUDE to point to everything that is needed by the mpifort compiler.
if you use the env command after loading a module on your machine, you can see what environment variables have been set on your machine by the module load command.

Here is an example of what was set up on mine:

CPATH=/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/include
INCLUDE=/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/include
LD_LIBRARY_PATH=/nas/longleaf/apps/gcc/6.3.0/lib:/nas/longleaf/apps/gcc/6.3.0/lib64:/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/lib64
LMOD_FAMILY_COMPILER=gcc
LMOD_FAMILY_COMPILER_VERSION=6.3.0
LMOD_FAMILY_MPI=openmpi_3.0.0
LMOD_FAMILY_MPI_VERSION=gcc_6.3.0
LOADEDMODULES=gcc/6.3.0:openmpi_3.0.0/gcc_6.3.0
MPICH_HOME=/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0
MPI_HOME=/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0
MVAPICH_HOME=/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0

in my config_cmaq.csh file I have the following settings for the gcc/gfortran compiler

#>  gfortran compiler............................................................
    case gcc:

        #> I/O API, netCDF, and MPI library locations
        setenv IOAPI_MOD_DIR   /proj/ie/proj/staff/lizadams/ioapi-3.2/Linux3_x86_64_openmpi_3.0.0_gcc_6.3.0  #> I/O API precompiled modules
        setenv IOAPI_INCL_DIR  /proj/ie/proj/staff/lizadams/ioapi-3.2/ioapi/fixed_src  #> I/O API include header files
        setenv IOAPI_LIB_DIR   /proj/ie/proj/staff/lizadams/ioapi-3.2/Linux3_x86_64_openmpi_3.0.0_gcc_6.3.0  #> I/O API libraries
        setenv NETCDF_LIB_DIR  /proj/ie/proj/staff/lizadams/netcdf-c-4.6.2-rc2/openmpi_3.0.0_gcc_6.3.0/lib #> netCDF directory path
        setenv NETCDF_INCL_DIR /proj/ie/proj/staff/lizadams/netcdf-c-4.6.2-rc2/openmpi_3.0.0_gcc_6.3.0/include #> netCDF directory path
        setenv MPI_LIB_DIR     /nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/lib    #> MPI directory path

        #> Compiler Aliases and Flags
        #setenv myFC mpif90
        setenv myFC mpifort
        setenv myCC gcc
        setenv myFSTD "-fPIC -no-pie -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -mtune=native -march=native -mavx2 -ftree-loop-if-convert -finline-limit=512"
        setenv myDBG  "-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace"
        #setenv myDBG  "$myDBG -fimplicit-none"
        setenv myFFLAGS "-fPIC -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32"
        setenv myFRFLAGS "-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32"
        setenv myCFLAGS "-O2 -fPIC"
        setenv myLINK_FLAG #"-openmp"
       setenv extra_lib ""
        #setenv mpi_lib "-lmpi_mpifh"   #> -lmpich for mvapich or -lmpi for openmpi
        setenv mpi_lib ""   #> -lmpich for mvapich or -lmpi for openmpi

        breaksw

Note, anything with a # in front is commented out, and ignored.
I am using mpifort rather than gfortran, as that is the Open MPI Fortran wrapper compiler . When you use the command

mpifort --showme

It will show you the commands that will be used to compile and link the CMAQ program.

gfortran -I/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/include -pthread -I/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/lib -Wl,-rpath -Wl,/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/lib -Wl,–enable-new-dtags -L/nas/longleaf/apps-dogwood/mpi/gcc_6.3.0/openmpi_3.0.0/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

20

Hello,

Below is what I got while building ICON and BCON, are they rightly built like this?

Finish building bldmake_gcc7.3.0.exe
end if
set Blder = /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/UTIL/bldmake/bldmake_gcc.exe -serial -verbose
cd /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/PREP/icon/scripts/BLD_ICON_v52_profile_gcc
if ( 0 ) then
if ( 1 ) then
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/UTIL/bldmake/bldmake_gcc.exe -serial -verbose ICON_v52_profile.cfg.bld
/home/catalyst/Desktop/Build_WRF/CMAQ-5.2/UTIL/bldmake/bldmake_gcc.exe: Command not found.
else
endif
mv Makefile Makefile.gcc
mv: cannot stat ‘Makefile’: No such file or directory
ln -s Makefile.gcc Makefile
if ( 0 != 0 ) then
if ( -e /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/PREP/icon/scripts/BLD_ICON_v52_profile_gcc/ICON_v52_profile.cfg ) then
mv ICON_v52_profile.cfg.bld /home/catalyst/Desktop/Build_WRF/CMAQ-5.2/PREP/icon/scripts/BLD_ICON_v52_profile_gcc/ICON_v52_profile.cfg

Thanks a lot.

Hello,
I have solved this one.
But it remains that of CCTM Compilation.
Thanks