Thanks for the questions. It seems you’re already aware of a lot of the information I would have provided. The answer to your question is yes, the value in the obs file simply called OC is what I now intend for people to use most often when comparing to modeled OC. That OC value represents the “best” available OC for a particular site for a particular time. As you mentioned there are many different OC values included in the obs file, which represent different measurement and post-adjustment techniques for estimating an observed OC value.
As you noted, the default AQS daily site compare run script included with CMAQv5.5 is incompatible with the latest obs files, as I no longer include an OC_blank column. This was oversite that will be corrected going forward. The obs files are primarily linked to the AMET workflow, which does create site compare scripts that are compatible with the latest obs files, but that update did not get translated into the CMAQv5.5 site compare scripts, unfortunately.
Current site compare scripts for AQS daily include matching the various different OC observed values to modeled OC, as below:
PM2.5 Sharp Cutoff Species
setenv AERO_30 "PM25,ug/m3,PM25_TOT,ug/m3,PM25_TOT" # PM2.5 Total Mass with sharp cutoff
setenv AERO_31 "SO4,ug/m3, PM25_SO4,,PM25_SO4" # sulfate (sharp cutoff)
setenv AERO_32 "NO3,ug/m3, PM25_NO3,,PM25_NO3" # nitrate (sharp cutoff)
setenv AERO_33 "NH4,ug/m3, PM25_NH4,,PM25_NH4" # ammonium (sharp cutoff)
setenv AERO_34 "OC,ug/m3, PM25_OC,,PM25_OC" # Organic Carbon (sharp cutoff)
setenv AERO_35 "EC,ug/m3, PM25_EC,,PM25_EC" # Elemental Carbon (sharp cutoff)
setenv AERO_36 "OC+EC,ug/m3,PM25_OC+PM25_EC,,PM25_TC" # Total Carbon (sharp cutoff)
setenv AERO_37 "OC_88305,ug/m3,PM25_OC,ug/m3,PM25_OC_88305" # OC blank adjusted Mass
setenv AERO_38 "EC_88307,ug/m3,PM25_EC,ug/m3,PM25_EC_88307" # EC Mass
setenv AERO_39 "OC_88305+EC_88307,ug/m3,AOCIJ+AECIJ,,PM25_TC_88305" # Total Carbon
setenv AERO_40 "OC_88370,ug/m3,PM25_OC,ug/m3,PM25_OC_88370" # OC blank adjusted Mass
setenv AERO_41 "EC_88380,ug/m3,PM25_EC,ug/m3,PM25_EC_88380" # EC Mass
setenv AERO_42 "OC_88370+EC_88380,ug/m3,AOCIJ+AECIJ,,PM25_TC_88370" # Total Carbon
setenv AERO_43 "OC_88320,ug/m3,PM25_OC,ug/m3,PM25_OC_88320" # OC blank adjusted Mass
setenv AERO_44 "EC_88321,ug/m3,PM25_EC,ug/m3,PM25_EC_88321" # EC Mass
setenv AERO_45 "OC_88320+EC_88380,ug/m3,AOCIJ+AECIJ,,PM25_TC_88320" # Total Carbon
You can see it matches many of the different OC (and EC) values to the modeled OC value, which is just intended to provide more information to the user. As I stated above, the intent is that the simple OC ob value column is the best match to the modeled value.
So, the quickest fix for your situation would be to simply remove the +OC_Blank from your site compare scripts so that you’re just matching to the OC value in the obs file. No further corrections to the OC obs value would be required. The effects from TOT and TOR are already accounted for in the pre-processing of the obs files (see the note below that I include with the obs files).
I know that was a lot, but hopefully it helped. If you still have questions, let me know.
Wyat
Important update for CSN and AQS Daily EC/OC data (10/28/2021):
There are three OC values from CSN that we need to be considered. Specifically, these are 88305, 88370, and 88320.
A quick description of each OC value:
88305 / 88307 OC / EC under the old CSN (TOT) protocol
88370 / 88380 OC / EC under the new CSN (TOR) protocol
88320 / 88321 OC / EC under the IMPROVE (TOR) protocol
Note that by default neither the 88305 or 88370 values are corrected for artifacts, while the 88320 value is corrected for artifacts.
From these three values, the 88320 is the preferred value, as it uses both the TOR IMPROVE protocol and is corrected for artifacts. Next best is 88370, as it also uses the TOR protocol but is not corrected for artifacts. The least preferred is 88305, as it uses the old TOT protocol and is not corrected for artifacts. From 2016 and beyond, 88320 is reported for all CSN sites (at least, it should be). Before 2016, there is a mix of 88320 and 88370. And before roughly 2007, 88305 is the primary OC value available.
As such, it is necessary to use a mix of three values when creating data files that cover 2000-present. Fortunately, there are artifact correction methods available for the 88305 and 88370 values to make them more comparable to the 88320 value across time.
For 88305, Joann Rice from EPA recommended this presentation, https://www.epa.gov/sites/default/files/2016-10/documents/statistical_relationship.pdf, where it is described how to convert the New CSN values (TOR; 88370) to Old CSN values (TOT; 88305) using this formula:
OC (Old) = 1.57432 + 1.15101*(NEW_CSN)
As the TOR method is preferred to the TOT method, this formula was inverted to correct the 88305 TOT OC values to be similar to the 88370 TOR OC values:
88305_adj = 0.8688*(88305) - 1.57432
The 88305_adj value is intended to mimic the 88320 value.
Similarly for EC:
EC (Old) = 0.105071 + 0.92462*(NEW_CSN)
88307_adj = 1.091525*(88307) - 0.104071
The 88307_adj value is intended to mimic the 88321 value.
Joann Rice also mentioned that previously the artifact correction for the OC 88370 value was 0.4, but a smaller value of 0.14 is now preferred. So, a value of -0.14 ug/m3 is subtracted from all the 88370 values to correct for artifacts. Applying the -0.14 ug/m3 correction is intended to make the 88370 value similar to the 88320 value. This artifact corrected 88370 value is called 88370_adj in the AMET AQS obs files.
This does not solve the issue that many CSN sites report different OC values based on the year, and some can report multiple OC values under the different OC codes. To deal with this situation, the 88320 value, if available, is the preferred OC value for a site. If 88320 is not available, the missing values are filled with 88370 (artifact corrected), again, if available. Finally, any remaining missing values are filled with 88305 (artifact corrected) if available. This should give the most complete OC record for CSN sites for any period using the best available OC value.
The result of implemented this approach is much better consistency in the OC values over time compared to the previous approach of providing OC as a hodge-podge mix of 88305, 88370, and 88320 with different correction methods.
***Starting with AMETv1.5 and all future versions of AMET, only the CSN observation files labeled AQS_CSN_data_XXXX.csv should be used as the source of CSN data in AMET. The other CSN data files are provided for legacy AMET versions, but are no longer updated as of October 2021. The AMETv1.5 code will only point to the files labeled AQS_CSN_data_XXXX.csv regardless of the year, and the AQ_species_list.input file has also been updated to use only the species in the AQS_CSN_data_XXXX.csv file.
