I am tracking the TAG CLASS “PM_TOT” using CMAQ v5.4 ISAM. The run successfully finished and I see all the source apportioned PM components in the SA outputs.
With SOA source apportionment now supported, all components of the particulate and gas phases are represented by CMAQ-ISAM and the apportionment of bulk PM metrics may be quantified.
Does this mean that SOA source apportionment results from tracking PM_TOT (or OA_TOT) using CMAQ v5.4 ISAM are wrong? If so, source apportionment of total PM2.5 (from CMAQ v5.4 ISAM) would be also wrong, right?
Thank you for your interest in CMAQ ISAM. This is an excellent question! The code changes described in the release note you found for v5.5 were also included in v5.4, i.e., your CMAQv5.4 PM ISAM results are not wrong. At the time of the v5.4 release we were doing our own CMAQ application using the SOA source apportionment code to test the new capability before fully publicizing it.
After the v5.4 release there were several updates to the ISAM code that were released as bugfixes. Take a look at this table to see if the bugfixes impact your work in which case you could consider switching to the final tagged version, v5.4.0.5: CMAQ/DOCS/CMAQ-Bugfix-Branch.md at main · USEPA/CMAQ · GitHub (specficially PRs #186, #183)
Like other areas of the CMAQ system, there are periodic updates to ISAM that are released with new model versions. These updates add new features, improve robustness, and/or fix issues. New releases do not invalidate the results of previous versions but do offer information on how to interpret or caveat results from past versions. Version 5.5 included the bugfixes from v5.4.0.5 as well as other updates. Here is the full list of v5.5 CMAQ ISAM release notes:
Returning to your question. We have used v5.4 for SOA source apporitonment for our own ISAM analysis and found the results robust and meaningful. For example see the Hogrefe et al. presentation at the 2024 CMAS Conference in NC which used CMAQ ISAM v5.4 with the ISAM cloud processing update released in v5.5.
Therefore, you have multiple viable options for your study. You can certainly procede with your current results, simply acknowledging where updates to ISAM in later versions may by important and worth future study. If you have the time/resources you could consider updating to v5.4.0.5 or even v5.5.0.3.
CMAQ-ISAM is an active area of development and application. A significant part of this effort is CMAS community members using ISAM for new applications and sharing results and issues. Please feel free to reach out if you have further questions.
Thanks for the detailed and comprehensive answer!
After reviewing the release notes and bug fixes, I ended up migrating to v5.5. But I’m glad to hear that v5.4 will still provide valid ISAM results.
Btw, I noticed that my ISAM run’s memory usage jumped on the 2nd simulation day by about 15%. I periodically checked the memory usage (using the Linux “free” command) while running the simulation, and this plot shows the result:
The only difference between the first and subsequent simulation days is that the first-day simulation is a new start (from an ICON input) and the subsequent days are re-started from previous day’s CGRID outputs, and I wonder why that would require additional memory space. Can you or anyone (@Ben_Murphy, @sergey, @hogrefe.christian) shed light on this?
Very interesting observation about the memory usage on the second day. Have you noticed if the spike or its magnitude is dependent on the number of sources or number of species tags requested?
Thanks @sergey and @Ben_Murphy for your responses, and my apologies for not getting back to you ealier.
This surprised me, too. Normally I don’t pay attention to how much memory a run requires, so I never realized this until I began using ISAM and became more conscious of memory usage.
I did a quick test with a standard run (non-ISAM build) and noticed that there is a small but distinct increase in memory usage with a restart. In this case, the memory usage increased by only about 2%, but the jump is noticeable.
I wonder if this has anything to do with how my cluster is configured, so it would be helpful if someone else verifies this issue on their cluster (a different machine).
One random thought is that ISAM always tracks contributions of initial conditions. On the first day, those are read from the standard CMAQ initial conditions files. On subsequent days, those are read from another file that was created at the end of the previous day. 15% seems a bit harsh for opening 1 extra file, but it could be possible, because it contains the number of variables equaling to the number of tracked species multiplied by the number of tags.
Do you still experience a jump in memory usage when you begin from an existing restart set of files or only from “clean” initial conditions?
Thanks @sergey for your comment! I think that makes sense. As shown in the plot in my reply to Kristen above, the 2nd and 3rd simulation days use (almost) the same amount of memory.
If that’s the case, I guess the memory blocks used for the restart conditions are not released even after the data is read and the CGRID array is populated.
