Problems on Create Initial and Boundary Conditions from Seasonal Average Hemispheric CMAQ Output

Hi,
i`ve been trying to create IC and BC file from “Seasonal Average Hemispheric CMAQ Output” which is available at https://drive.google.com/file/d/15Vt6f5WuyN8RiLRjTlKeQUHjYbZ6QCrA/view?usp=sharing.But my run aborted with the following error:

“MET_CRO_3D_CRS” opened as OLD:READ-ONLY
File name “/public/home/cmaq/zjr/two_way/CMAQ-5.3/data/hemisphere/ICON_HEMIS_CMAQ.nc”
File type GRDDED3
Execution ID “???”
Grid name “WRF_CMAQ_2WAY”
Dimensions: 187 rows, 187 cols, 44 lays, 254 vbles
NetCDF ID: 196608 opened as READONLY
Starting date and time 2015001:000000 (0:00:00 Jan. 1, 2015)
Timestep 010000 (1:00:00 hh:mm:ss)
Maximum current record number 2
Differences found between files CTM_CONC_1 and MET_CRO_3D_CRS :
NONE
Checking MET_BDY_3D_FIN File for consistent header data
No file header inconsistencies for MET_BDY_3D_FIN
Input SDATE equals zero; SDATE reset to MET_BDY_3D_FIN file start date
Input STIME equals zero; STIME reset to MET_BDY_3D_FIN file start time

 Input RUNLEN not set or equal to zero.                                          
 Resetting RUNLEN to correspond to MET_BDY_3D_FIN file ending date & time.       

 NCOLS_IN:          187
 NROWS_IN:          187
 NLAYS_IN:           44
 NSPCS_IN:          254
 NBNDY:             688
 NLAYS:              27

 Value for IOAPI_ISPH:  '20'
 INITSPHERES:  input sphere Normal Sphere (MM5 & WRF-ARW) R=6370000
 
 >>--->> WARNING in subroutine LAMBERT/SETLAM
 Bad origin latitude Y =    90.000
 
 
 *** ERROR ABORT in subroutine LAT_LON
 Lambert projection setup error for CTM CONC file

Is there someting wrong with projection mapping?whether BCON con not convert the polar projection to Lambert projection?
I would be grateful if anyone can help me with this issue.

Hello,

could you please provide some additional details on what you did prior to running BCON and could you please also post your BCON run script?

Looking at the portion of the log file you posted above, it appears you may have downloaded the seasonal average H-CMAQ file “CCTM_CONC_v53beta2_intel17.0_HEMIS_cb6r3m_ae7_kmtbr_m3dry_2016_quarterly_av.nc”, renamed it as ICON_HEMIS_CMAQ.nc, and then set MET_CRO_3D_CRS in the BCON run script to point to that file. However, the start date/time and number of time steps reported in your log file don’t exactly match those in the original file, so I’m wondering if you may have modified the downloaded file in some fashion before attempting to run BCON. If you did, what did you attempt to modify and which tools did you use?

Thanks,

Christian

Dear Christian,

Yes i downloaded the seasonal average H-CMAQ file,and used ‘m3tshift’ tool shifted time stamps in the downloaded file,which means the shifted file only contain one-time stamps(2015001), otherwise bcon run would aborted with the following error messages:
*** ERROR ABORT in subroutine M3_INBNDY
Requested starting time is not in the CTM_CONC_1 file
Date and time 1:00:00 Jan. 1, 2015 (2015001:010000)

Attached are the BCON run script, and c-shell script for using m3tshift to shift time stamps.
run_bcon.csh (5.1 KB) timeshift.csh (1010 Bytes)

Thanks a lot !
zunyir

Thanks for this additional information.

Could you please post the header attributes, particularly the projection information, for your CTM_CONC_1 (i.e. $CMAQ_DATA/hemisphere/ICON_HEMIS_CMAQ.nc) and MET_BDY_3D_FIN (i.e. $CMAQ_DATA/mcip/output01/METBDY3D_d01) files?

In addition, while this shouldn’t be related to the projection problem, I suggest that you update your timeshift.csh script to exactly follow the sample script posted at https://github.com/USEPA/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_HCMAQ_IC_BC.md, only updating “set TARGET_YEAR = 2014” to “set TARGET_YEAR = 2015”. The values in this file are six quarterly averages, and you want BCON to take care of the temporal interpolation to a particular day rather than handling this via m3tshift. The only thing m3tshift should be used for in this approach is to shift all six quarterly averages forward or backward by one or more years to cover the intended modeling period.

Dear Christian,

Thanks for your suggestion and it will be great helpful!
Attached are the header file for CTM_CONC_1 and MET_BDY_3D_FIN.
header_attributes_BDY.txt (6.0 KB) header_attributes_CONC.txt (9.9 KB)
Thanks a lot !
zunyir

Do you really want a projection relevant to Mongolia? the XCENT=105 seems to suggest that.

It is customary to follow the ISO 6709 Standard, according to which Western Hemisphere longitudes are negative, between 0 and -180. -105 cuts across the western Great Plains in the US…

[BTW, beware, of course, of WMO data, for which longitudes are between 0 and 360 (and which is a PITA to manipulate in standard geographic tools ;-( ) ]

Thanks for posting the projection parameters for CTM_CONC_1 and MET_BDY_3D_FIN.

It seems the underlying problem is that GDTYP is set to 2 (Lambert conformal) in your CTM_CONC_1 file while it was 6 (polar stereographic) in the file you downloaded and ran through m3tshift. All the other projection parameters are still unchanged from the downloaded file, in particular YCENT is still set to 90 which causes the error in SETLAM. Put differently, the BCON code should be calling SETPOL rather than SETLAM when processing CTM_CONC_1 but gets tripped up by the incorrect setting of GDTYP.

Did you do any manual editing of this attribute? Processing the downloaded file through m3tshift should not have caused this, so I’m wondering what other processing might have been done to this file.

Thanks,

Christian

Dear Christian,

As you suggested, I manually revised the ‘GDTYP=2’ to ‘GDTYP=6’ and it worked !
Thanks for your useful detailed feedback,i will re-download it from google drive and doublecheck the file.But i did not do other modify to seasonal average H-CMAQ file before attempting to run BCON.

Thanks a lot !
Zunyir

Hi Zunyir,

I’m glad this worked. I’m still puzzled how GDTYP was changed to 2 in the H-CMAQ seasonal average file you initially tried to use as input to BCON. I downloaded the H-CMAQ file from the CMAS data warehouse yesterday and confirmed that in that file the value of GDTYP is correctly set to 6 (polar stereographic). If you are able to trace back what happened, maybe you can post your experience in this thread to help other users avoid this potential pitfall.

Christian

Dear Christian,

I think the only one questionable step is the way to download the H-CMAQ file from the CMAS data warehouse, in China mainland we can not directly access the google drive so i have to download it via mobile VPN application, then copy it from phone memory card to computer, and i also find that raw data only contain three-time stamps (10/16/2015 12:00 (2015289, 120000), 1/16/2016 0:00 (2016016, 0), 4/16/2016 12:00 (2016107, 120000)), the other three-time stamps( 7/17/2016 0:00, 10/16/2016 12:00, and 1/16/2017 0:00)turned ito ( 0, 0).Maybe an exception might occur when i transfer the data from phone to PC ……?
I swear did not do anyother modify to this file after i downloaded,i will re-download it from google drive again and doublecheck it.

Zunyir

This is really interesting, thank you for posting your experience.

Hi Zunyir,
I successfully created ICBC files for my target year from seasonal average hemispheric CMAQ output. My experience is

  1. download the seasonal average hemispheric CMAQ output from the website posted at CMAQ_UG_tutorial_HCMAQ_IC_BC.md at https://github.com/USEPA/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_HCMAQ_IC_BC.md.
  2. If you need shift the downloaded data to your target year (e.g.2014), as Christian Hogrefe suggested that you only need to update “set TARGET_YEAR = 2014” in the script posted at the same website.
  3. You do not need to update GDTYP which is 6 and correct in original and downloaded file.
  4. The error you have may be caused by wrong MET_BDY_3D_FIN File. Make sure the MET_BDY_3D_FIN is METBDY3D_d0x_YYYYMMDD.nc (dx is domain id, it is d01 in your case. It should be the coarse domain you want create boundary conditions for). For ICON, the input file MET_CRO_3D_FIN should be METCRO3D_d0x_YYYYMMDD.nc. In addition, you may use MCIP v5.0 to precess WRF outputs to generate METBDY3D_d0x_YYYYMMDD.nc and METCRO3D_d0x_YYYYMMDD.nc.
  5. I do not think you have any troubles to create ICBC files from seasonal average hemispheric CMAQ output if you follow above steps.
    Thanks
2 Likes

Hi all,

I met a problem running m3tshift. What I did were:

  1. download the nc file from google drive
  2. create a timeshift.csh file by copy the sample scripts provided in https://github.com/USEPA/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_HCMAQ_IC_BC.md and change the target year to 2015
  3. run the script by typing ./timeshift.csh

The error I got was “Command not found” and “@: Badly formed number.”

Anyone can help with it? Do I need to do anything else before I run the script?

Thanks,
Qian

Did you specify the path to the m3tshift executable?

set EXEC = /path/to/m3tshift

Hi Qian,

Please use the following command on your machine to find the path to m3tshift

which m3tshift
If m3tshift is found, that means that the path will be recognized when you run the timeshift.csh script.

Then replace
set EXEC = /path/to/m3tshift
with
set EXEC = m3tshift

If the m3tshift command isn’t available in your path, then you will need to find the ioapi-3.2 library. In the directory where you have the libioapi.a library, you should also find the m3tshift executable.

The m3tshift executable is likely available under a subdirectory of ioapi-3.2 that corresponds to the mpi and compiler version that was used to build it:
/ioapi-3.2/Linux2_x86_64gfort_openmpi_4.0.1_gcc_9.1.0/m3tshift

Liz

I downloaded the dataset and tried to set the year to 2015.
I did see an error.

[lizadams@dogwood-login2 timeshift]$ ./timeshift.csh |& tee ./timeshift.log

 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).                                 
 Copyright (C) 1992-2002 MCNC,                                   
 (C) 1992-2013 Carlie J. Coats, Jr.,                             
 (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and   
 (C) 2014-2019 UNC Institute for the Environment.                
 Released under the GNU LGPL  License, version 2.1.  See URL     
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.2: $Id: init3.F90 120 2019-06-21 14:18:20Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version 4.1.3 of Jun 19 2018 15:50:22 $
  
  
 Missing environment variable EXECUTION_ID
                 
 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 
                 
 Program M3TSHIFT to copy a selected time period from a Models-3 file
 to a different time period in a different Models-3 file.
 
 You need to have set environment variables for the input and output
 file logical names.  You will be asked to select the time period to be
 copied and the start of the time period to receive the results.
 Note that RUNLEN=0 for single-step runs (a "fencepost" problem)
 
 USAGE:  m3tproc [INFILE OUTFILE]     (and then answer the prompts).
 
 See URL
 
    https://www.cmascenter.org/ioapi/documentation/3.1/html/AA.html#tools
 
 Program copyright (C) 1992-2002 MCNC, (C) 1995-2013 Carlie J. Coats, Jr.
 (C) 2002-2010 Baron Advanced Meteorological Systems, LLC., and
 (C) 2014-2016 UNC Institute for the Environment.
 Released under Version 2 of the GNU General Public License. See
 enclosed GPL.txt, or URL
 
     https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
 
 Comments and questions are welcome and can be sent to
 
     Carlie J. Coats, Jr.    carlie@jyarborough.com
 or
     UNC Institute for the Environment
     100 Europa Dr., Suite 490 Rm 405
     Campus Box 1105
     Chapel Hill, NC 27599-1105
 
 Program version: 
 $Id: m3tshift.f90 117 2019-06-15 14:56:29Z coats $
  
 
 Value for PROMPTFLAG not defined;returning default:   TRUE
 Enter INPUT FILE name [INFILE] >>     Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "INFILE" opened as OLD:READ-ONLY   
 File name "/proj/ie/proj/staff/lizadams/timeshift/download_data/CCTM_CONC_v53beta2_intel17.0_HEMIS_cb6r3m_ae7_kmtbr_m3dry_2016_quarterly_av.nc"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name "WRF_CMAQ_2WAY"
 Dimensions: 187 rows, 187 cols, 44 lays, 254 vbles
 NetCDF ID:     65536  opened as READONLY            
 Starting date and time  2015289:120000 (12:00:00  Oct. 16, 2015)
 Timestep                        21960000 (2196:00:00 hh:mm:ss)
 Maximum current record number         6
 Value for PROMPTFLAG not defined;returning default:   TRUE
 Enter starting (source) date (YYYYDDD) for run [2015289] >>     Using response   2015289
 Enter starting (source) time  (HHMMSS) for run [120000] >>     Using response    120000
 Enter target date (YYYYDDD) for run [2015289] >>     Using response   2014289
 Enter target time  (HHMMSS) for run [120000] >>     Using response    120000
 Enter output time step (HHMMSS) for run [21960000] >>     Using response  21960000
 Enter duration (HHMMSS) for run [109800000] >>     Using response 131760000
 Enter OUTPUT FILE name [OUTFILE] >>     
 Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "OUTFILE" opened as UNKNOWN(R-W)    
 File name "/proj/ie/proj/staff/lizadams/timeshift/download_data/CCTM_CONC_v53beta2_intel17.0_HEMIS_cb6r3m_ae7_kmtbr_m3dry_2015_quarterly_av.nc"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name "WRF_CMAQ_2WAY"
 Dimensions: 187 rows, 187 cols, 44 lays, 254 vbles
 NetCDF ID:    131072  opened as READWRITE           
 Starting date and time  2014289:120000 (12:00:00  Oct. 16, 2014)
 Timestep                        21960000 (2196:00:00 hh:mm:ss)
 Maximum current record number         6
 Value for IOAPI_LOG_WRITE not defined;returning default:   TRUE
 Timestep written to OUTFILE for date and time   2014289:120000
 Timestep written to OUTFILE for date and time   2015016:000000
 Timestep written to OUTFILE for date and time   2015107:120000
 Timestep written to OUTFILE for date and time   2015199:000000
 Timestep written to OUTFILE for date and time   2015290:120000
 Timestep written to OUTFILE for date and time   2016017:000000
 netCDF error number  -40
 
 >>--->> WARNING in subroutine RDTFLAG
 Error reading netCDF time step flag for INFILE
 M3WARN:  DTBUF 12:00:00  April 17, 2017(2017107:120000)
 
 >>--->> WARNING in subroutine M3TSHIFT
 Read failure:  file  INFILE
 M3WARN:  DTBUF 12:00:00  April 17, 2017(2017107:120000)
 
 *** ERROR ABORT in subroutine M3TSHIFT

I think the duration requested in the timeshift.csh example is too long.
Enter duration (HHMMSS) for run [109800000] >> Using response 131760000

Changing 131760000 to 109800000 seems to get rid of this error, and results in the following output after running.
—>> Normal Completion of program M3TSHIFT
Success in program

I am attaching a corrected script.timeshift.csh (1.1 KB)

Yes, and I also ran the executable directly to check if it worked.

Hi Liz,

I did what you recommended. m3tshift command was not available in my path. I had included the path in the script but still got the same issue.

Qian

Hi Qiuo,
Can you upload your script so we can take a look at it.
Thanks,
Liz

The script is attached here.timeshift.csh (1.2 KB)

Thank you so much!