# CMAQ-5.3.1 Benchmark difference

**URL:** <https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445>\
**Category:** CMAQ\
**Created:** [June 22, 2021, 5:00pm UTC](https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445 "2021-06-22T17:00:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lwu127](https://avatars.discourse-cdn.com/v4/letter/l/ea666f/32.png) [@lwu127](https://forum.cmascenter.org/u/lwu127)\
**Post date:** [June 22, 2021, 5:00pm UTC](https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445/1 "2021-06-22T17:00:07Z")

</div>

Hi everyone,  
I run the benchmark test case (2016,2day) of CMAQ-5.3.1 and use m3diff tool to calculate the difference. There are very large difference for variables NUMACC, NUMATKN and NUMCOR. I attached the run script and compile script here. What should i do to fix this?  
Thanks!  
Lin  
[bldit\_cctm.csh](https://forum.cmascenter.org/uploads/short-url/cdU3vj6Ju0FYpLP9p0Nt2BSknLl.csh) (28.6 KB)  
[run\_cctm\_Bench\_2016\_12SE1.csh](https://forum.cmascenter.org/uploads/short-url/vfIWJi8qpTPZceEfIBc0kDqYNUU.csh) (34.0 KB)

 ![2021-06-22 (8)](https://canada1.discourse-cdn.com/flex027/uploads/cmas/original/2X/8/8c21e54548e510c98cafd7ee4513c768f2a9a3ee.png)  
 ![2021-06-22 (9)](https://canada1.discourse-cdn.com/flex027/uploads/cmas/original/2X/e/e9540835b7d1dd9c4170bf68120085abbdc57fc2.png)  
 ![2021-06-22 (10)](https://canada1.discourse-cdn.com/flex027/uploads/cmas/original/2X/b/bb9a76e98b1e1195bc028514eeaaa3ff02f15284.png)

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [June 22, 2021, 6:12pm UTC](https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445/2 "2021-06-22T18:12:33Z")

</div>

Those negative values worry me (all three variables).

And you’re running with

> `setenv CTM_CKSUM Y #> checksum report [default: Y]`

which should trap all conc-variables to be at least `CMIN` (which **should** be non-negative)… but you’re still getting negative number-concentrations ;-(

Another thing that worries me is the advective time-step constraint you’re using:

> `setenv CTM_ADV_CFL 0.95 #> max CFL [default: 0.75]`

which is much larger than I’m comfortable with: are these negative number-concentrations the result of numeric-algorithm errors in the advection coming from too large an advective time-step?

---

<div class="post-metadata">

**Author:** ![lwu127](https://avatars.discourse-cdn.com/v4/letter/l/ea666f/32.png) [@lwu127](https://forum.cmascenter.org/u/lwu127)\
**Post date:** [June 23, 2021, 2:49am UTC](https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445/3 "2021-06-23T02:49:18Z")

</div>

Thanks for your suggestion, I changed the CTM\_ADV\_CFL variable to 0.75 and rerun the model. The differences are still this large for the three variables.  
Lin

---

<div class="post-metadata">

**Author:** ![hogrefe.christian](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@hogrefe.christian](https://forum.cmascenter.org/u/hogrefe.christian)\
**Post date:** [June 23, 2021, 3:52pm UTC](https://forum.cmascenter.org/t/cmaq-5-3-1-benchmark-difference/2445/4 "2021-06-23T15:52:58Z")

</div>

Best I can tell, what is shown in these plots are not negative negative number concentrations, but both positive and negative _differences_ in number concentrations between a simulation performed by @lwu127 and the reference CMAQ benchmark output dataset distributed by CMAS. Please correct me if this is not the case.

Absolute values for NUMATK, NUMACC, and NUMCOR for the benchmark case are on the order of 1E+06 to 1E+10, so the difference values shown above aren’t necessarily concerning. It would be more informative to compute relative rather than absolute differences to judge whether your simulations produce results that are reasonably close to the reference data sets.