Custom CMAQ File from Python

PYTHON
1

Several different groups have reached out to me to find out how to quickly make an IOAPI-like file from custom data using Python. There are lots of ways to get this done – I am putting one online for reference.

The script below pulls grid metadata from a GRIDDESC text file. It then creates a file-like object in memory for a TMP variable. The vals array is currently just a placeholder. You’d need to define your own vals array however you like.

from datetime import datetime
import numpy as np
import PseudoNetCDF as pnc


# Grid definition must be selected by GDNAM from gdpath, which should
# to your GRIDDESC file (currently assuming GRIDDESC in the same folder)
gdpath = './GRIDDESC'
GDNAM = '12US1'

# Output path and metadata
outpath = 'test.nc'
fdesc = f"""author: Your Name
institution: Your Institution
title: Short Description
version: 1.0
date: {datetime.utcnow():%Y-%m-%dT%H:%M:%S}+0000
description: Additional notes about what is in the file
"""
# SDATE, STIME and TSTEP should almost always be defined
# any other metadata property can be added
file_kw = dict(SDATE=2023001, STIME=0, TSTEP=10000)

# Define variables with units (one or many)
var_kw = dict(TMP=dict(units='m'))

# Open a template
gf = pnc.pncopen(
    gdpath, format='griddesc', GDNAM=GDNAM, withcf=False,
    var_kwds=var_kw, **file_kw
)

# define your own values however you like here I am using RRRCCC as an integer
# and reshaping to (1, 1, NROWS, NCOLS
vals = np.arange(gf.NROWS)[:, None] * 1000 + np.arange(gf.NCOLS)[None, :]

# Copy values into variable (or variables)
gf.variables['TMP'][:] = vals.reshape(-1, gf.NLAYS, gf.NROWS, gf.NCOLS)

# Add description to the file
gf.FILEDESC = ''.join([l.ljust(80) for l in fdesc.split('\n')])[:60*80]

# Save to disk
gf.save(outpath, format='NETCDF3_CLASSIC', verbose=0)

If you save this code to customfile.py, you run it from a command line with python customfile.py Or you can paste the text into a Jupyter Notebook and run it.

Either way, the script requires Python3 and PseudoNetCDF and netcdf4 (and maybe pyproj). To install, use the command below.

python -m pip install --user numpy netcdf4 pyproj pseudonetcdf

I have tested the output with m3hdr and m3xtract tools to assure output compatibility with IOAPI.

4 Likes