I am trying to perform a WRFv4.1.1-CMAQv5.3.1 benchmark run but it cannot read the benchmark WRF input files. It appears they have been created by netcdf with hdf5. If correct, files created by netcdf without hdf5 would be helpful.
I guess you could directly contact David (Wong.David-C@epa.gov).
Maybe he has several advice for you.
Yes, the WRF input files posted for the WRFv4.1.1-CMAQv5.3.1 coupled model benchmark run were generated from a WRF system compiled with netcdf4/hdf5 which is the default configuration of WRF. Following this default configuration allows WRF (as well as the coupled WRFv4.1.1-CMAQv5.3.1 model) to reduce the size of the wrfout output files through netdf4 deflation (the CCTM output files created through ioapi do not make use of that deflation option).
To reduce the size of the wrfout files from the WRFv4.1.1-CMAQv5.3.1 coupled model, you may want to consider rebuilding your netcdf library with netcdf4/hdf5 support and then building WRF (and WRF-CMAQ) with the default netcdf4/hdf5 option.
In case this isn’t an option for you, we are also in the process of converting the benchmark WRF input files from netcdf4 back to netcdf3 classic and will post an updated tar file with all benchmark data in the near future.
Thank you for your messages. I can rebuild WRF-CMAQ with the netcdf library with hdf5 support. But I am just curious that the classic netcdf is recommended for CMAQ as mentioned in the manual. Is it no problem in the case of WRF-CMAQ?
The netcdf building recommendations in the users guide were written with the requirements of offline CMAQ in mind. Given that I/O API (at least as used in CMAQ) does not currently utilize any capabilities that are specific to netcdf4/hdf5, our recommendation to disable the additional libraries during the netcdf build process was meant to make this process easier for users.
For the coupled WRF-CMAQ model (as well as stand-alone WRF, if the WRF output files are meant to be processed through MCIP or combine), users will have to decide whether to change the default WRF configuration to not use netcdf4/hdf5 during the build process, or whether to create a netcdf library with netcdf4/hdf5 support and then use that library for building WRF, WRF-CMAQ, and CMAQ components like MCIP and combine that read WRF output files.
Be aware that the recommendations were made in large part to simplify the build process for all of the programs and tools that use it: otherwise, the Makefiles become complicated in many different ways (depending upon netCDF configuration), and there is no practical way to automate their construction, thus requiring much manual user intervention for “fix up” of the Makefiles.
Besides, for some of the use cases (identified even thirty years ago in the original systems requirements), HDF does not provide truly random access to the data, and the year-long (or especially multi-decade) HDF based data files have serious performance problems. These performance problems do not occur with “netCDF Classic” – I know; I’ve done 33-year model runs (and the relevant post-processing and analysis for the 1.2TB files) with it. As of basically netCDF-3.5, the only basic obstacle is available disk space.
Carlie J. Coats, Jr., Ph.D.
I/O API Author/Maintainer
Thank you for your messages.
I understand that we can use netcdf4/hdf5 if we can manage Makefile by ourselves to use it, and I will keep in mind possible performance problems.
We appreciate your kind supports.