Hi there,
I have a netcdf library compiled with hdf5 and zlib, and subsequently an ioapi 3.2 has been built on it.
Does this ioapi enable writing to netcdf4 in compressed mode? If not, will there be any plan in the near future?
Thanks!
No.
One of the requirements for the I/O API is direct access, and compression prevents this, since it requires sequential decompression of the data, or else potentially arbitrarily-long index structures.
Consider the (actual, I’ve-done-this) situation of having a decade-long 3000x2000 gridded hourly surface meteorology file, for dates 19910101 - 20000101, and you want to use it to drive a land-surface simulation from 19990701 through 19910801. Or suppose you want to look at that simulation-start date&time 1990701:000000. It is unacceptable to force you to wait for hours while the program is decompressing/reading nine-years-plus of data before you can actually start simulating or visualizing anything.
By the way, one of the reasons netCDF was chosen instead of HDF as the lower layer for the I/O API was the experience that HDF had very long access times for situations like the above, because of the time required to access those (randomly-stored) index structures that it maintains before it ever started reading the data.
Thank you Carlie for providing the insight I never perceived before.
Roger
Hi
I want to install netCDF 4, but I couldn’t. I install zlib-1.2.7 and hdf5-1.10.1 and netcdf-4.1.3. but when I configure netcdf, it gave Error.
checking for library containing H5Fflush… no
configure: error: Can’t find or link to the hdf5 library.
Could you help me, please?
Thank you.
Hello Ensiyeh,
Did you include the following options while configuring?
./configure --prefix=${NCDIR} --disable-dap --enable-static --disable-shared
Roger
Make that “… --disable-dap --disable-netcdf-4…”
1 Like
Thank you.
I want to install online coupled wrf-cmaq and I need netcdf4, so I can’t disable it; is that right?
IIRC, WRF uses the “classic” netCDF3 interface, so you should not need the new netCDF4 features and can disable them.
What about using –disable-hdf4 which will disable the HDF4 and HDF5 interfaces?
Alternatively, you need to run the netCDF commands
nc-config --libs
nf-config --libs
to find out what extra libraries you will need in all your Makefiles, etc.
Thank you. It did not work and it had the same error.
These two commands didn’t work. I wanted to install them but they gave errors.
Hi
I want to install IOAPI for online coupled wrf_cmaq. Is it necessary that I install PVM?
NO. Build and use the “nocpl” version of the I/O API and use that.
Hi Carlie
Thank you for your answer. But I think cpl is for coupled and nocpl for uncoupled. do you sure that I must use nocpl?
“Coupled” refers to cooperating-process coupled modeling, where, for example, WRF, SMOKE, and CMAQ run simultaneously, with SMOKE using PVM to read a WRF time step as soon as it is written and then execute that step; then CMAQ reading that time step of WRFand SMOKE data when it becomes available and then executing that time step, etc.
This mode was used, for example, for the world’s first air quality forecasts (with the met running on one computer, SMOKE running on another, and the CTM running on a third) in a joint Penn State/North Carolina Supercomputing Center project.
One advantage of doing it that way is that AQ forecast results for a given time become available sooner than they would in a “conventional” system, where the met runs, then the emissions, and only finally the CTM. Unfortunately, the current authors of SMOKE and CMAQ have (unnecessarily) broken this model of computing by checking that ALL time steps of all inputs are available at model start-up, so that the models are coded not to run at all in this mode of operation. It is a failure of modularity to confuse data/process management with modeling, but the current SMOKE and CMAQ do this.
1 Like
Hi Carlie
In IOAPI which makefiles do we modify?we have a makefile in main directory, and some makefiles in ioapi and some makesfile inm3tools. Is it necessary to change the name of modified makefile to just makefile?
and is it necessary to modify makeinclude?
for example: Makefile.nocpl.sed, what is sed?
OK, if the web based installation instructions don’t work for you, try following the README.txt in the top level I/O API directory.
Hi Carlie
I try it, too; but it gives error.
Works for me.
And you’re giving me nothing to work with here: I don’t have a clue what error you’re getting, in order to diagnose how to fix it.