Unknow CMAQ issue when running "./bldit_bcon.csh intel" and "./bldit_icon.csh intel"

Hi CMAQ developers:

I am a graduate student, who has being using CMAQ for more than 2 years. During the past of two years, it was always successfully compiled and run for a plenty of time
However, since the Cluster of my university update the system setting and modules, which the CMAQ is run, updated the system recently. The same program, which I run successfully several times during the past two years, keeps failing recently
I have tried versions of intel and impi, and tried under both bash and csh. The IT Specialist of my department has tried to rebuild the ioapi, but couldn’t find out the appropriate way. I just pasted the error messages I got so far below, and attached the scripts with this post.

Could anyone give us some guidance when you have time?
Best
Huan

When running on bash

HGRD_DEFN.F(113): error #7002: Error in opening the compiled module file. Check INCLUDE paths. [UTILIO_DEFN]

     USE UTILIO_DEFN

-------------^

HGRD_DEFN.F(175): error #6404: This name does not have a type, and must have an explicit type. [DSCGRID]

        IF ( .NOT. DSCGRID ( GRID_NAME,

-----------------------^

compilation aborted for HGRD_DEFN.F (code 1)

make: *** [HGRD_DEFN.o] Error 1

ERROR while running make command

When running on csh

bcon.F(58): error #7002: Error in opening the compiled module file. Check INCLUDE paths. [CGRID_SPCS]
USE CGRID_SPCS ! CGRID mechanism species
----------^
bcon.F(94): error #6404: This name does not have a type, and must have an explicit type. [NSPCSD]
IF ( NSPCSD .LE. 0 ) THEN
-----------^
bcon.F(127): error #6404: This name does not have a type, and must have an explicit type. [CGRID_SPCS_INIT]
IF ( .NOT. CGRID_SPCS_INIT() ) THEN
-----------------^
bcon.F(135): error #6385: The highest data type rank permitted is INTEGER(KIND=8). [NSPCSD]
ALLOCATE ( NML_BC_NAME( NSPCSD ), STAT = STATUS )
------------------------------^
bcon.F(142): error #6404: This name does not have a type, and must have an explicit type. [N_GC_SPC]
DO N = 1, N_GC_SPC
----------------^
bcon.F(144): error #6404: This name does not have a type, and must have an explicit type. [GC_SPC]
NML_BC_NAME( ISP ) = GC_SPC( N )
------------------------------^
bcon.F(144): error #6054: A CHARACTER data type is required in this context. [GC_SPC]
NML_BC_NAME( ISP ) = GC_SPC( N )
------------------------------^
bcon.F(147): error #6404: This name does not have a type, and must have an explicit type. [N_AE_SPC]
DO N = 1, N_AE_SPC
----------------^
bcon.F(149): error #6404: This name does not have a type, and must have an explicit type. [AE_SPC]
NML_BC_NAME( ISP ) = AE_SPC( N )
------------------------------^
bcon.F(149): error #6054: A CHARACTER data type is required in this context. [AE_SPC]
NML_BC_NAME( ISP ) = AE_SPC( N )
------------------------------^
bcon.F(152): error #6404: This name does not have a type, and must have an explicit type. [N_NR_SPC]
DO N = 1, N_NR_SPC
----------------^
bcon.F(154): error #6404: This name does not have a type, and must have an explicit type. [NR_SPC]
NML_BC_NAME( ISP ) = NR_SPC( N )
------------------------------^
bcon.F(154): error #6054: A CHARACTER data type is required in this context. [NR_SPC]
NML_BC_NAME( ISP ) = NR_SPC( N )
------------------------------^
bcon.F(157): error #6404: This name does not have a type, and must have an explicit type. [N_TR_SPC]
DO N = 1, N_TR_SPC
----------------^
bcon.F(159): error #6404: This name does not have a type, and must have an explicit type. [TR_SPC]
NML_BC_NAME( ISP ) = TR_SPC( N )
------------------------------^
bcon.F(159): error #6054: A CHARACTER data type is required in this context. [TR_SPC]
NML_BC_NAME( ISP ) = TR_SPC( N )
------------------------------^
compilation aborted for bcon.F (code 1)
make: *** [bcon.o] Error 1
ERROR while running make command

bldit_bcon.csh (10.9 KB) bldit_icon.csh (10.5 KB) config_cmaq.csh (9.8 KB)

They’ve changed the compilers, and the new compilers can’t read the existing “.mod” module-files. The rest of the errors seem to be a cascade starting from that. Don’t you hate such gratuitous incompatibility ?!!

You need to re-build everything from scratch.

Build instructions for the I/O API are here: https://cjcoats.github.io/ioapi/AVAIL.html#build

Oh, thank you, we will have a try on this way

Hi Carlie:
We rebuilt ioapi 3.2, but still the same error messages, maybe we need to change the version of netcdf and intel accordingly? What is the ideal version of netcdf and intel is for ioapi 3.2?
Huan

When you quote text from a build problem, please include the text of the compiler command-line, not just the error messages.

What is clear is that the compiler is not successfully reading the module-files. If the compiler command were correct, this would suggest a mixed-versions-of-compiler problem. If it’s not that, it means you have not properly followed the installation instructions, so that either the module-files are not in the right place or else the compiler’s module-include directive is not correct.

Hi Carlie:
Could you check the private message I sent you earlier? I think it would be more effective if the IT specialist of my department could keep in touch with you
Huan

Sometimes when system administrators adopt a new version of a compiler, they will keep the old version around. On our system, we use modules to put the different compiler versions and libraries in our path. You might check with your sysadmin as to whether the old version you were using still exists on your system.

bldit_bcon.log.txt (13.0 KB) bldit_icon.log.txt (10.4 KB)
Hi Carlie:
I just attach the running log of bldit_bcon.csh and bldit_icon.csh. Hope it will be helpful to figure it out

Thank you for your time
Huan

From bldit.bcon.log: it’s not finding cgrid_spcs.mod. If that’s in the BLD directory, you need a
-I.
directive in your compiler flags to tell the compiler to look for it in the current BLD directory. If it’s somewhere else, say directory /foo/bar, then you need
-I/foo/bar

Will this be generated by the script “config_cmaq.csh”, thus it is caused by the improper setting of ioapi, netcdf, mpi, etc.?

So what program generates this module, and what’s the default location for it?

Huan

From the BCON log:

mpiifort -c -fixed -132 -O3 -fno-alias -mp1 -fp-model source -I /scratch/brown/fang63/cmaq/CMAQ-5.2.1/lib/x86_64/intel/ioapi_new/Linux2_x86_64ifort -I /scratch/brown/fang63/cmaq/CMAQ-5.2.1/lib/x86_64/intel/ioapi_new/Linux2_x86_64ifort   BC_PARMS.F

Do you see that you are including (via the -I flag) the same directory twice? One of those should probably be some other directory, perhaps the current directory (-I.).

So how shall I fix this problem? Did I compile the ioapi improperly? Or something in the config_cmaq.csh?

And Carlie says it is missing the file called “cgrid_spcs.mod”, do you have any idea where could I get this file?

This is not a problem with I/O API.
It is likely something in config_cmaq.csh. You might first try modifying the Makefile in the build directory. Change it so that there is an include directive for the current directory.

Hi Chris:
It is direct to the same place twice because we put the files for include, module, and netcdf dependencies all reside in this single folder. Will this cause the program unable to recognize?
Huan

Do you have a cgrid_spcs.mod file (name might be capitalized) in your build directory? If so, then the problem is the make program does not know to look in the build directory for the module files.
Edit the Makefile. Where you currently have two

-I /scratch/brown/fang63/cmaq/CMAQ-5.2.1/lib/x86_64/intel/ioapi_new/Linux2_x86_64ifort, 

change one of them to -I..

build directory?
I think that shall be created after running bldit_bcon.csh right?

Yes, among other things the build script creates a build directory (name begins with BLD), copies source files into the directory, determines the dependencies, creates a Makefile, and invokes make to compile the model. If you get an error during compilation, you can edit the Makefile, then invoke make again, until the compilation is successful.
This approach is sometimes easier (and faster) than debugging the build script.