Turn off CTM_LOG files

Dear all,
Is there a way to turn off the CTM_LOG files? Currently, we are testing the performance of CMAQ-model on the new HPC cluster and it turns out that I/O times are quite high. We are trying to test different options (using pnetcdf, netcdf3, netcdf4) but we are also wonder how this Ascii CTM_LOG files affect I/O times. Can you help us how to turn off the production of CTM_LOG files? (we perfom simulation with 900 processors = 900 log files)

thank you

Hi Dusan,

Even with 900 log files, I’d be surprised if this was a major drain on the runtime compared to writing out the data. But it would be an interesting test. Unfortunately, there is no way I’m aware of to turn off the log file output with an option. You should be able to modify the source code to mostly achieve this though.

In the file RUNTIME_VARS.F, modify the routines LOG_HEADING, LOG_SUBHEADING, and LOG_MESSAGE by either commenting out all write statements or just entering a RETURN after the variable definition section in each. This won’t eliminate all of the writes, but should take care of most of them.

Do you have access to a profiler that will run on 900 cores? If so, I would be very interested to know what uses most of the time in that configuration.

Best wishes,
Ben

You’re probably already doing this, but just in case. On many clusters, you can write much faster to local node disk than network/shared disk. Therefore, it is often much faster to do the run with all outputs directly to local storage and then move the results after each day. This is a minor change to the run scripts, but can be very helpful. As an alternative, you could edit the source as Ben says to output just the log to local disk and then move the logs.

Just a thought.