# Cmaq make BCON files that take more than a day

**URL:** <https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948>\
**Category:** CMAQ\
**Created:** [February 28, 2023, 8:24am UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948 "2023-02-28T08:24:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Xin\_jiayi](https://avatars.discourse-cdn.com/v4/letter/x/b9bd4f/32.png) [@Xin\_jiayi](https://forum.cmascenter.org/u/Xin_jiayi)\
**Post date:** [February 28, 2023, 8:24am UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/1 "2023-02-28T08:24:44Z")

</div>

I’m running a three-level nested case of CMAQ, and the first domain is already running using profile.Now, When I make a second level nested domain bcon fileWhen I make a second-level nested domain bcon file, I can only generate bcon for one day at a time. But the time of my simulation is 1 year, which is obviously not recommended. Is there any better way to batch processing please, I would appreciate it.

---

<div class="post-metadata">

**Author:** ![hogrefe.christian](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@hogrefe.christian](https://forum.cmascenter.org/u/hogrefe.christian)\
**Post date:** [February 28, 2023, 12:17pm UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/2 "2023-02-28T12:17:14Z")

</div>

While the example `run_bcon.csh` script released with the code is set up to process a single day, you can easily modify it to create a loop over an extended time period and then call the BCON executable for each day. This is the way the example CCTM run scripts are set up (e.g. `run_cctm_Bench_2018_12NE3.csh`), so you can adapt the relevant script segments from there. Something like this should work:

```
diff run_bcon.csh run_bcon_loop_example.csh
116,120c116,132
<
< set DATE = "2016-07-01"
< set YYYYJJJ = `date -ud "${DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ
< set YYMMDD = `date -ud "${DATE}" +%y%m%d` #> Convert YYYY-MM-DD to YYMMDD
< set YYYYMMDD = `date -ud "${DATE}" +%Y%m%d` #> Convert YYYY-MM-DD to YYYYMMDD
---
>
> set START_DATE = "2016-07-01" #> beginning date (July 1, 2016)
> set END_DATE = "2016-07-15" #> ending date (July 1, 2016)
>
> set TODAYG = ${START_DATE}
> set TODAYJ = `date -ud "${START_DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ
> set START_DAY = ${TODAYJ}
> set STOP_DAY = `date -ud "${END_DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ
>
> while ($TODAYJ <= $STOP_DAY ) #>Compare dates in terms of YYYYJJJ
>
> #> Retrieve Calendar day Information
> set YYYYJJJ = $TODAYJ
> set YYMMDD = `date -ud "${TODAYG}" +%y%m%d` #> Convert YYYY-MM-DD to YYMMDD
> set YYYYMMDD = `date -ud "${TODAYG}" +%Y%m%d` #> Convert YYYY-MM-DD to YYYYMMDD
> set YYYYMM = `date -ud "${TODAYG}" +%Y%m` #> Convert YYYY-MM-DD to YYYYMM
>
157a170,176
>
> #> Increment both Gregorian and Julian Days
> set TODAYG = `date -ud "${TODAYG}+1days" +%Y-%m-%d` #> Add a day for tomorrow
> set TODAYJ = `date -ud "${TODAYG}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ
>
> end #Loop to the next day
>

```

Also note that BCON opens the `BNDY_CONC_1` output file in FSUNKN3 mode which means a new BCON run can add time steps to an existing output file. So, if you prefer to create (for example) one large BCON file with hourly data per month rather than multiple smaller BCON files with hourly data for each day in the month, you could change

` setenv BNDY_CONC_1 "$OUTDIR/BCON_${VRSN}_${APPL}_${BCON_TYPE}_${YYYYMMDD} -v"`

to

` setenv BNDY_CONC_1 "$OUTDIR/BCON_${VRSN}_${APPL}_${BCON_TYPE}_${YYYYMM} -v"`

---

<div class="post-metadata">

**Author:** ![Xin\_jiayi](https://avatars.discourse-cdn.com/v4/letter/x/b9bd4f/32.png) [@Xin\_jiayi](https://forum.cmascenter.org/u/Xin_jiayi)\
**Post date:** [February 28, 2023, 2:27pm UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/3 "2023-02-28T14:27:19Z")

</div>

Thank you very much for your advice, you are an angel. Due to my lack of experience, there was still an error. I hope you can check it out for me, best wishes.  
Below are  
[run\_bcon\_xie\_re\_while2.csh](https://forum.cmascenter.org/uploads/short-url/n25f3COWOwzttQgqo7fbCl9ygqu.csh) (6.1 KB)  
[bcon\_while.txt](https://forum.cmascenter.org/uploads/short-url/c9aeDvtKf74k9PmS2PXMCirNTcV.txt) (87.1 KB)  
my related documents.

---

<div class="post-metadata">

**Author:** ![Xin\_jiayi](https://avatars.discourse-cdn.com/v4/letter/x/b9bd4f/32.png) [@Xin\_jiayi](https://forum.cmascenter.org/u/Xin_jiayi)\
**Post date:** [February 28, 2023, 2:33pm UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/4 "2023-02-28T14:33:56Z")

</div>

Value for IOAPI\_CHECK\_HEADERS not defined;returning default: FALSE

```
 "BNDY_CONC_1" opened as UNKNOWN(R-W)    
 File name "/home/u220220948411/wrf/cmaq532/data/xie_cr/d02/01_03/BCON_v532_2016_BJM_regrid_5_"
 File type BNDARY3 
 Execution ID "BCON_v532.exe"
 Grid name "2016_BJM"
 Dimensions: 148 rows, 178 cols, 42 lays, 220 vbles, 1 cells thick
 NetCDF ID: 196608 opened as VOLATILE READWRITE  
 Starting date and time 2017001:000000 (0:00:00 Jan. 1, 2017)
 Timestep 010000 (1:00:00 hh:mm:ss)
 Maximum current record number 25
 Value for IOAPI_LOG_WRITE: F returning FALSE

```

```auto
Timestep written to BNDY_CONC_1 for date and time 2017001:000000

     Timestep written to BNDY_CONC_1 for date and time 2017001:010000

     Timestep written to BNDY_CONC_1 for date and time 2017001:020000

     Timestep written to BNDY_CONC_1 for date and time 2017001:030000

     Timestep written to BNDY_CONC_1 for date and time 2017001:040000

     Timestep written to BNDY_CONC_1 for date and time 2017001:050000

     Timestep written to BNDY_CONC_1 for date and time 2017001:060000

     Timestep written to BNDY_CONC_1 for date and time 2017001:070000

     Timestep written to BNDY_CONC_1 for date and time 2017001:080000

     Timestep written to BNDY_CONC_1 for date and time 2017001:090000

     Timestep written to BNDY_CONC_1 for date and time 2017001:100000

     Timestep written to BNDY_CONC_1 for date and time 2017001:110000

     Timestep written to BNDY_CONC_1 for date and time 2017001:120000

     Timestep written to BNDY_CONC_1 for date and time 2017001:130000

     Timestep written to BNDY_CONC_1 for date and time 2017001:140000

     Timestep written to BNDY_CONC_1 for date and time 2017001:150000

     Timestep written to BNDY_CONC_1 for date and time 2017001:160000

     Timestep written to BNDY_CONC_1 for date and time 2017001:170000

     Timestep written to BNDY_CONC_1 for date and time 2017001:180000

     Timestep written to BNDY_CONC_1 for date and time 2017001:190000

     Timestep written to BNDY_CONC_1 for date and time 2017001:200000

     Timestep written to BNDY_CONC_1 for date and time 2017001:210000

     Timestep written to BNDY_CONC_1 for date and time 2017001:220000

     Timestep written to BNDY_CONC_1 for date and time 2017001:230000

     Timestep written to BNDY_CONC_1 for date and time 2017002:000000
     netCDF error number -40
     
     >>--->> WARNING in subroutine RDTFLAG
     Error reading netCDF time step flag for CTM_CONC_1
     M3WARN: DTBUF 1:00:00 Jan. 2, 2017 (2017002:010000)
     
     >>--->> WARNING in subroutine M3_BCOUT:INTERP3
     Time step not available for "NO2" from CTM_CONC_1
     M3WARN: DTBUF 1:00:00 Jan. 2, 2017 (2017002:010000)
     
     *** ERROR ABORT in subroutine M3_BCOUT
     Could not read input CTM Conc file CTM_CONC_1
     Date and time 1:00:00 Jan. 2, 2017 (2017002:010000)
     

```

---

<div class="post-metadata">

**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\
**Post date:** [February 28, 2023, 2:51pm UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/5 "2023-02-28T14:51:54Z")

</div>

To output a multi-day `BCON` file, the program needs the matching multi-day input `CONC` file (or else needs to be making multiple runs, one for each input day, but with the same `BCON` output file.

The “usual” way to do this is to modify the shell-script to loop over all the days executing single-day `BCON`-program runs, as mentioned above.

An alternative is to use [I/O API File-Set **File Lists**](https://cjcoats.github.io/ioapi/BUFFERED.html#list) to have the I/O API treat the entire sequence of single-day `CONC` files as though it were a single input file, and script a multi-day run that way, using a single list as the input file, and generating a single (multi-day) output file. (Note that programs like VERDI that treat the I/O Applications **Programming** Interface (API) as generating just a data format, rather than as the programming interface it is, are unable to use File-Set file lists ;-( )

---

<div class="post-metadata">

**Author:** ![hogrefe.christian](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@hogrefe.christian](https://forum.cmascenter.org/u/hogrefe.christian)\
**Post date:** [February 28, 2023, 2:58pm UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/6 "2023-02-28T14:58:48Z")

</div>

A few observations:

1. YYYYJJJ, YYMMDD, YYYYMMDD, and YYYYMM all need to be set inside the “while” loop for this to work as intended, and need to work off of {TODAYG}. In the script you posted, you kept the original script setting of defining these variables prior to the start of the “while” loop, and working off of the static {DATE} variable rather than the {TODAYG} variable incremented in the loop.

2. the loop needs to encompass the call to the BCON executable.

3. moreover, in your situation you seem to be working with MCIP files containing three months of hourly data, and CCTM\_CONC files just containing a single day. Therefore, you’ll need to define the optional SDATE, STIME, and RUNLEN environment variables to restrict the processing of each day to just the hours present in the CCTM\_CONC files.

Something like this might work:

```
# set DATE = "2017-01-01"
# set YYYY = `date -ud "${TODAYG}" +%Y`
# set YYYYJJJ = `date -ud "${DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ

# set YYMMDD = `date -ud "${DATE}" +%y%m%d` #> Convert YYYY-MM-DD to YYMMDD

# set YYYYMMDD = `date -ud "${DATE}" +%Y%m%d` #> Convert YYYY-MM-DD to YYYYMMDD
# setenv SDATE ${YYYYJJJ}
# setenv STIME 000000
# setenv RUNLEN 240000

    set START_DATE = "2017-01-01" #> beginning date (July 1, 2016)
    set END_DATE = "2017-03-31" #> ending date (July 1, 2016)
    set TODAYG = ${START_DATE}
    set TODAYJ = `date -ud "${START_DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ
    set START_DAY = ${TODAYJ}
    set STOP_DAY = `date -ud "${END_DATE}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ

    while ($TODAYJ <= $STOP_DAY ) #>Compare dates in terms of YYYYJJJ

 #> Retrieve Calendar day Information
  set YYYYJJJ = $TODAYJ
  set YYMMDD = `date -ud "${TODAYG}" +%y%m%d` #> Convert YYYY-MM-DD to YYMMDD
  set YYYYMMDD = `date -ud "${TODAYG}" +%Y%m%d` #> Convert YYYY-MM-DD to YYYYMMDD
  set YYYYMM = `date -ud "${TODAYG}" +%Y%m` #> Convert YYYY-MM-DD to YYYYMM

  setenv SDATE ${TODAYJ}
  setenv STIME 000000
  setenv RUNLEN 250000

  if ( $BCON_TYPE == regrid ) then 

    setenv CTM_CONC_1 /home/u220220948411/wrf/cmaq532/data/xie_cr/d01/normal2/output_CCTM_v532_intel_d01_0103_normal/CCTM_CONC_v532_intel_d01_0103_normal_${YYYYMMDD}.nc

    setenv MET_CRO_3D_CRS /home/u220220948411/wrf/cmaq532/data/xie_cr/d01/01_03/METCRO3D_01.nc
    setenv MET_BDY_3D_FIN /home/u220220948411/wrf/cmaq532/data/xie_cr/d02/01_03/METBDY3D_01.nc
    setenv BNDY_CONC_1 "$OUTDIR/BCON_${VRSN}_${APPL}_${BCON_TYPE}_5_${YYYY} -v"

 endif

  if ( $BCON_TYPE == profile ) then
    setenv BC_PROFILE $BLD/avprofile_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
    setenv MET_BDY_3D_FIN /home/u220220948411/wrf/cmaq532/data/xie_cr/d01/01_03/METBDY3D_01.nc    
setenv BNDY_CONC_1 "$OUTDIR/BCON_${VRSN}_${APPL}_${BCON_TYPE}_${YYYYMMDD} -v"
 endif

# =====================================================================
#> Output File
# =====================================================================

#>- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

 if ( ! -d "$OUTDIR" ) mkdir -p $OUTDIR

 ls -l $BLD/$EXEC; size $BLD/$EXEC
 unlimit
 limit

#> Executable call:
 time $BLD/$EXEC

#Increment both Gregorian and Julian Days
    set TODAYG = `date -ud "${TODAYG}+1days" +%Y-%m-%d` #> Add a day for tomorrow

    set TODAYJ = `date -ud "${TODAYG}" +%Y%j` #> Convert YYYY-MM-DD to YYYYJJJ

 end #Loop to the next day

```

---

<div class="post-metadata">

**Author:** ![Xin\_jiayi](https://avatars.discourse-cdn.com/v4/letter/x/b9bd4f/32.png) [@Xin\_jiayi](https://forum.cmascenter.org/u/Xin_jiayi)\
**Post date:** [March 1, 2023, 2:50am UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/8 "2023-03-01T02:50:51Z")

</div>

Good morning, I’m glad to see your reply at first glance today. Thank you very much for your patience and posting the code. I have successfully run the BCON script from 2017-01-01 to 2017-03-31 and CCTM is currently running normally.

Best wishes.

---

<div class="post-metadata">

**Author:** ![Xin\_jiayi](https://avatars.discourse-cdn.com/v4/letter/x/b9bd4f/32.png) [@Xin\_jiayi](https://forum.cmascenter.org/u/Xin_jiayi)\
**Post date:** [March 1, 2023, 2:52am UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/9 "2023-03-01T02:52:17Z")

</div>

Thank you for your advice and guidance, the problem has been solved.  
Best wishes

---

<div class="post-metadata">

**Author:** ![lizadams](https://avatars.discourse-cdn.com/v4/letter/l/49beb7/32.png) [@lizadams](https://forum.cmascenter.org/u/lizadams)\
**Post date:** [March 1, 2023, 3:04am UTC](https://forum.cmascenter.org/t/cmaq-make-bcon-files-that-take-more-than-a-day/3948/10 "2023-03-01T03:04:02Z")

</div>