Dear Ben,
thank you for the help, but this elimination of the writes in Logs does not help. A person from HPC company who helped me to test and optimized the performance of the CMAQ model for the specific simulation wrote me finally:
I tested a variety of processor grids with the result that a quadratic one is optimal and that we are at the very end of MPI scalability.
Anyhow, any grid with less than 900 processors showed worse calculation step performances. The hourly steps on the other hand are more demanding with regards to memory bandwidth which limits the number of utilized cores per socket to maximize the performance.
This would be an ideal situation where shared memory parallelism (OpenMP) would pay off but unfortunately it is not implemented in CMAQ and it is not trivial to do so.
Also, there were no configurations found where the overhead of parallel I/O through pnetcdf was compensated by better read/write times or lower wait times.