ERROR running HDDM using CMAQV5.3.3: Max NVARS exceeded when building CTM_SWETDEP_1

Hi,
I am running DDM with CMAQv5.3.3. I’ve already run it successfully when I set 2 or 3 sensitivities. However, I got this error when I defined 44 first-order and second-order sensitivities with specific regions.


I tried to limit the species output for DDM and changed the AVG_CONC_SPCS in run_cctm.csh. But this did not change the output species in the CTM_SWETDEP_1. Could someone help me to solve this problem? Thank you for taking the time.
Here’s my sensinput.dat and log file.
sensinput_CN3GD.dat.txt (3.6 KB)
CTM_LOG_000.PRD_process_2_d03_mcip5.0_base_20210920.txt (1.9 MB)

Thanks,
Dara

You need to be using I/O API-large, which supports up to 16384 variables, at some considerable space and performance overhead: see https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#build:

For CMAQ-DDM, ioapi-3.2-large has expanded maximum-number-of-files and maximum-variables-per-file parameter-values. This version should be kept carefully isolated from the regular version of the I/O API, and should be used only for CMAQ-DDM and CMAQ-ISAM

You can download it from https://www.cmascenter.org/ioapi/download/ioapi-3.2-large.tar.gz

Note

2 Likes

Additionally, once you install the I/O API-large library and rebuild your executables, you can verify that the correct library is being used by paying attention to this portion at the beginning of the CTM_LOG file you posted:

 ioapi-3.2: $Id: init3.F90 185 2020-08-28 16:49:45Z coats $                           
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048                                     
 netCDF version 4.6.2 of Oct  7 2019 01:45:40 $      

As @cjcoats noted, with the I/O API-large library the message should then show that MXVARS is 16384 which should get you past the error you’re currently encountering.

1 Like

Dear Carlie,
Thanks for your kind reply. I have downloaded ioapi-3.2-large.tar.gz. I found no Makefile file under $BASEDIR. I tried to copy Makefile.template from ioapi-3.2. The air2m3 error occured when compiling m3tools.
The loaded modules are as follows:


I attach the relevant files here:
make.log.txt (59.9 KB)

$BASEDIR/Makefile
Makefile.txt (10.0 KB)
$BASEDIR/ioapi/Makefile
Makefile_ioapi.txt (15.7 KB)
Makeinclude.Linux2_x86_64ifort.txt (1.5 KB)
$BASEDIR/m3tools/Makefile
Makefile_m3tools.txt (13.3 KB)

I added extra flag -DIOAPI_NCF4=1 to Makeinclude.Linux2_x86_64ifort but still got same error.

What you’re seeing is a netCDF that requires (among other things) HDF5…

See https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#ncf4:

It is recommended that you disable these options by adding the command-line flags below to your netCDF configure command:

--disable-netcdf4 --disable-dap
or (depending upon netCDF version; don’t you love consistency and compatibility !?? )
--disable-netcdf-4 --disable-dap
If you build netCDF-4 without the --disable-netcdf4, you will also need to add -DIOAPI_NCF4=1 either to the ARCHFLAGS in your Makeinclude.${BIN} or to the DEFINEFLAGS in your Makefile, since defining netCDF-4 causes netCDF to change parts of the netCDF-3 interface.

Hi guys,

As far as I understand CMAQv5.3.3 does not fully support DDM3D, only CMAQv5.2 or CMAQv5.4 does.

You may move to CMAQv5.4 if you need to run CMAQ with DDM and HDDM.

Thanks.

Thanks for your suggestion. I would like to try running DDM. At the moment it seems that the issue is not the version, but the IOAPI version. I will consider upgrade CMAQ later.

Dear Carlie,
Sorry for the delay in the response. I have tried to recompile netcdf4 with command:
./configure --prefix=/BIGDATA1/app/netcdf/4.6.2-icc-18/disable-netcdf4 CPPFLAGS=-I/BIGDATA1/app/hdf5/1.8.21-icc-18/include LDFLAGS=-L/BIGDATA1/app/hdf5/1.8.21-icc-18/lib CC=icc --disable-dap --disable-dap-remote-tests --disable-netcdf4
I have compiled ioapi and m3tools separately. And I got same error when I compiled m3tools before and after recompiling netcdf. Here are log file and makefiles.


log.txt (51.1 KB)
Makefile.txt (9.9 KB)
Makefile.ioapi.txt (15.5 KB)
Makeinclude.Linux2_x86_64ifort.txt (1.5 KB)
Makefile.m3tools.txt (13.1 KB)

Did you update the I/O API without adjusting the m3tools/Makefile for the changes in the update?
The Makefile should have the following lines:


wrftom3.f90 mpasdiff.f90 mpasstat.f90 mpastom3.f90 randomstat.f90

vertimeproc vertintegral wndwdesc wrfgriddesc wrftom3 \
randomstat

randomstat: randomstat.o
cd ${OBJDIR}; $(FC) ${LFLAGS} $^ ${LIBS} -o $@

and make sure that “cd...” line begins with a tab; web cut-and-paste often replaces tabs with spaces (which don’t work in Makefiles).

Program randomstat is used by the make test command…

Dear Carlie,
I found a spelling mistake in line 91 in m3tools/Makefile and corrected it.


But the problem is that there is no randomstat.f90 in the directory.