Different year for CEM/ptegu emissions

jschnell · May 21, 2019, 2:35pm

I’m using the EPA’s 2016 alpha platform to process emissions. My simulated time period is over the summer of 2018. I’d like to know how to download and format the 2018 CEM data from the EPA Air Markets as inputs instead. It seems I’ll need the FF10 egucems* files, a “cemsum” file, and HOUR_UNIT files for each month I want to simulate. What is the recommended approach?

eyth.alison · May 21, 2019, 3:28pm

Hi, the beta platform is available now and has more updated data for many sectors.

The inputs for that platform and documentation are here:
http://views.cira.colostate.edu/wiki/wiki/10197

The 2018 CEMS data are here:
https://ampd.epa.gov/ampd/

Go to the Prepackaged Data tab and go to the bottom to find the data in SMOKE format.

Keep in mind that the annual inventory is 2016 and there could be retirements and other changes in the operation of the sources between 2016 and 2018.

jschnell · May 21, 2019, 3:35pm

Thank you @eyth.alison!

lexiewilson · January 21, 2022, 4:36pm

I am trying to use the 2016 v2 platform ptegu scripts to process CEM emissions from 2013. I downloaded the 2013 files from the link you provided on this thread. I ran cemscan with this script:
run_cemscan_2013.csh (525 Bytes)
When I run the ptegu daily script, I get this error:

SCRIPT ERROR: Some file patterns not matched for style Hourly
              According to MULTIFILE_NAMEBREAK, the file prefixes are:
                 /uufs/chpc.utah.edu/common/home/daq-group2/SMOKE4_7/2017by_13winter/inputs/cem/HOUR_UNIT
              Check file names and rerun
ERROR: Could not run combine_data.csh for hourly list file in Annual_area

I suspect the files I downloaded form AMPD are not quite in the right format for use with this script.
I see in the 2016v2 platform documentation that “we’ve split
the data into months and days as needed for our scripts, and run the data
through a “CEM correct” program.” I found this cem correct program in the 2016 v1 folder:
https://gaftp.epa.gov/Air/emismod/2016/v1/2016emissions/cemcorrect_2.1_Nov12.zip
But I’m unsure how to use the “cem correct” program. Is there guidance for this program?

eyth.alison · January 24, 2022, 2:22pm

You can try changing the ptegu daily run script to increase the number of the NAMEBREAK_HOURLY variable by 1.

If that doesn’t work, please send your list of hourly CEMS files, i.e.:

/uufs/chpc.utah.edu/common/home/daq-group2/SMOKE4_7/2017by_13winter/inputs/cem/HOUR_UNIT*

lexiewilson · January 24, 2022, 4:17pm

I changed the NAMBREAK_HOURLY variable from 4 to 5 and am still getting the same error as before. Here is my file list that I input to cemscan:
filelist2013.txt (1.2 KB)

eyth.alison · January 24, 2022, 5:09pm

Did you download the files from the SMOKE-ready files area? There are a couple areas of the CAMD site…

lexiewilson · January 24, 2022, 5:19pm

Here are the first few lines of my january HOUR_UNIT file:

3,"1","130101",0,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",1,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",2,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",3,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",4,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",5,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",6,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",7,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",8,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",9,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",10,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",11,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",12,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",13,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",14,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",15,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",16,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",17,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",18,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",19,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",20,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",21,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",22,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",23,-9,-9,-9,0,-9,-9,-9,,,,,-9

Here is a screenshot of where I downloaded the files:

eyth.alison · January 24, 2022, 6:16pm

Do they match the format of the files for the 2016/2017 platforms?

lexiewilson · January 24, 2022, 6:32pm

It looks like it - but maybe I am missing something! Here are the first few lines from the january HOUR_UNIT file from the 2017 platform:

3,"1","170101",0,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",1,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",2,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",3,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",4,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",5,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",6,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",7,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",8,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",9,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",10,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",11,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",12,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",13,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",14,-9,-9,-9,0,-9,-9,-9,,,,,-9

I notice that the 2017 platform files have 2 HOUR_UNIT files for each month:

while the 2013 files I downloaded only have 1 per month:

Is this where the cem correct script might come into play?

eyth.alison · January 24, 2022, 6:39pm

So there is an additional step involved in using the files. The SMOKE scripts expect to find two files per month, one with the last day of the month and one with all of the other days of the month, plus an additional file for December 31 of the previous year. A sample script to create these files from the full-month CEMs is pasted below. This script (after modifying for 2013) should be run in the directory with the HOUR_UNIT_2013_* files. The outputs will go in a subdirectory called SMOKE/, and the SMOKE run scripts should then point to the SMOKE/ files.

A sample script is provided for 2018 as an example. Save this text to an executable .csh file

#!/bin/csh -f

grep 180131 HOUR_UNIT_2018_01.txt > SMOKE/HOUR_UNIT_2018_01_31jan_2018.txt

sed ‘/180131/d’ HOUR_UNIT_2018_01.txt > SMOKE/HOUR_UNIT_2018_01_1-30jan_2018.txt

grep 180228 HOUR_UNIT_2018_02.txt > SMOKE/HOUR_UNIT_2018_02_28feb_2018.txt

sed ‘/180228/d’ HOUR_UNIT_2018_02.txt > SMOKE/HOUR_UNIT_2018_02_1-27feb_2018.txt

grep 180331 HOUR_UNIT_2018_03.txt > SMOKE/HOUR_UNIT_2018_03_31mar_2018.txt

sed ‘/180331/d’ HOUR_UNIT_2018_03.txt > SMOKE/HOUR_UNIT_2018_03_1-30mar_2018.txt

grep 180430 HOUR_UNIT_2018_04.txt > SMOKE/HOUR_UNIT_2018_04_30apr_2018.txt

sed ‘/180430/d’ HOUR_UNIT_2018_04.txt > SMOKE/HOUR_UNIT_2018_04_1-29apr_2018.txt

grep 180531 HOUR_UNIT_2018_05.txt > SMOKE/HOUR_UNIT_2018_05_31may_2018.txt

sed ‘/180531/d’ HOUR_UNIT_2018_05.txt > SMOKE/HOUR_UNIT_2018_05_1-30may_2018.txt

grep 180630 HOUR_UNIT_2018_06.txt > SMOKE/HOUR_UNIT_2018_06_30jun_2018.txt

sed ‘/180630/d’ HOUR_UNIT_2018_06.txt > SMOKE/HOUR_UNIT_2018_06_1-29jun_2018.txt

grep 180731 HOUR_UNIT_2018_07.txt > SMOKE/HOUR_UNIT_2018_07_31jul_2018.txt

sed ‘/180731/d’ HOUR_UNIT_2018_07.txt > SMOKE/HOUR_UNIT_2018_07_1-30jul_2018.txt

grep 180831 HOUR_UNIT_2018_08.txt > SMOKE/HOUR_UNIT_2018_08_31aug_2018.txt

sed ‘/180831/d’ HOUR_UNIT_2018_08.txt > SMOKE/HOUR_UNIT_2018_08_1-30aug_2018.txt

grep 180930 HOUR_UNIT_2018_09.txt > SMOKE/HOUR_UNIT_2018_09_30sep_2018.txt

sed ‘/180930/d’ HOUR_UNIT_2018_09.txt > SMOKE/HOUR_UNIT_2018_09_1-29sep_2018.txt

grep 181031 HOUR_UNIT_2018_10.txt > SMOKE/HOUR_UNIT_2018_10_31oct_2018.txt

sed ‘/181031/d’ HOUR_UNIT_2018_10.txt > SMOKE/HOUR_UNIT_2018_10_1-30oct_2018.txt

grep 181130 HOUR_UNIT_2018_11.txt > SMOKE/HOUR_UNIT_2018_11_30nov_2018.txt

sed ‘/181130/d’ HOUR_UNIT_2018_11.txt > SMOKE/HOUR_UNIT_2018_11_1-29nov_2018.txt

grep 181231 HOUR_UNIT_2018_12.txt > SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt

sed ‘/181231/d’ HOUR_UNIT_2018_12.txt > SMOKE/HOUR_UNIT_2018_12_1-30dec_2018.txt

sed ‘s/181231/171231/g’ SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt > SMOKE/HOUR_UNIT_2017_12_31dec_2018.txt

Here is a list of the resulting files that come out for 2018:

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2017_12_31dec_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_01_1-30jan_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_01_31jan_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_02_1-27feb_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_02_28feb_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_03_1-30mar_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_03_31mar_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_04_1-29apr_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_04_30apr_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_05_1-30may_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_05_31may_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_06_1-29jun_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_06_30jun_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_07_1-30jul_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_07_31jul_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_08_1-30aug_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_08_31aug_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_09_1-29sep_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_09_30sep_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_10_1-30oct_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_10_31oct_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_11_1-29nov_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_11_30nov_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_12_1-30dec_2018.txt

/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt

eyth.alison · January 24, 2022, 6:59pm

Yes I just sent another post on this topic

Topic		Replies	Views
A question about ptegu SMOKE script 2016v2 Emissions Inventory smoke , cmaqsmoke	5	442	September 15, 2022
Create CEM file for 2022? SMOKE	1	326	May 19, 2022
SMOKE input file format _EPA 2016v1 platform NEI SMOKE smoke , smoke-481 , cmaqsmoke	6	385	August 16, 2021
EPA 2016v1 SMOKE run output for CMAQ model SPIN UP period SMOKE smoke	1	378	April 21, 2021
Creating CEMSUM file SMOKE	3	430	November 9, 2021

Different year for CEM/ptegu emissions

Related topics