4 km oceanfile?

Spatial Allocator
1

Hello everyone,

I have been trying to create an oceanfile using the spatial allocator tool. I’ve tried running the script for 10 days twice (my max run time on our cluster) and it never completes running. I’ve successfully created the dummy_oceanfile, but I would like to integrate the real ocean file. My domain is huge, so I understand that this will take time. I’m using the surfzone_poly_st that is downloaded with the spatial allocator tool. My questions are as follows:

  1. Can I use mpirun/srun with the allocator.exe script? Currently I’ve tried this, and it’s running on 6 nodes, but am I simply wasting memory? I’ve read through the user guide and it never mentions parallel computing but perhaps that functionality exists?

  2. if this never completes, is there another tool I can use to create the ocean file?

I’ve attached the runscript I’ve used to create the file and my griddesc.

alloc_surf_zone_to_oceanfile.csh (1.6 KB)

GRIDDESC:

' '
'LAM_40N97W'
  2        33.000        45.000       -97.000       -97.000        40.000
' '
'CONUS4K_d02'
'LAM_40N97W'  -2292000.000  -1584000.000      4000.000      4000.000 1155 726   1
' '
2

As an update, I tried running the script interactively as the log file gives me nothing when I fail, this is what the interactive output looks like:

Spatial Allocator Version 3.6, 03/10/2009

WARNING: Environment variable: MAX_INPUT_FILE_SHAPES, not set

EV: MIMS_PROCESSING=ALLOCATE

EV: ALLOCATE_ATTRS=TYPE

EV: OUTPUT_FILE_NAME=/projects/b1045/montgomery/ocean_file_CONUS4K_d02_try3.ncf

Using ALLOCATE mode

EV: OUTPUT_FILE_TYPE=IoapiFile

EV: OUTPUT_FILE_TYPE=IoapiFile

WARNING: Environment variable: OVERLAY_TYPE, not set

EV: OUTPUT_FILE_TYPE=IoapiFile

Reading Regular Grid

EV: OUTPUT_GRID_NAME=CONUS4K_d02

WARNING: Environment variable: MAX_LINE_SEG, not set

MAX_LINE_SEG not set, discretization intervals disabled

griddesc file name = /projects/b1045/wrf-cmaq/input/land/CONUS4K_d02/GRIDDESC

Ellipsoid var = OUTPUT_FILE_ELLIPSOID

EV: OUTPUT_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0

Ellipsoid=+a=6370000.0,+b=6370000.0

EV: OUTPUT_GRID_NAME=CONUS4K_d02

Not using BB optimization

Finished reading the output file…

EV: INPUT_FILE_TYPE=ShapeFile

EV: INPUT_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0

Ellipsoid=+a=6370000.0,+b=6370000.0

EV: INPUT_FILE_MAP_PRJN=+proj=lcc,+lat_1=33,+lat_2=45,+lat_0=40,+lon_0=-97

EV: INPUT_FILE_TYPE=ShapeFile

EV: INPUT_FILE_NAME=/projects/b1045/cmaq/Spatial-Allocator/data/Spatial-Allocator/data/surfzone/surfzone_poly_st.shp

WARNING: Environment variable: MAX_LINE_SEG, not set

MAX_LINE_SEG not set, discretization intervals disabled

max_line_seg=

Reading Shapefile /projects/b1045/cmaq/Spatial-Allocator/data/Spatial-Allocator/data/surfzone/surfzone_poly_st.shp

Shapefile Type: 5 Polygon # of Shapes: 20183

Input projection:

args[1]=+a=6370000.0

args[2]=+b=6370000.0

param 0 = +proj=lcc,

param 3 = +lat_1=33,

param 4 = +lat_2=45,

param 5 = +lat_0=40,

param 6 = +lon_0=-97,

PROJ args=+proj=lcc+a=6370000.0+b=6370000.0+lat_1=33+lat_2=45+lat_0=40+lon_0=-97

Output projection:

args[1]=+a=6370000.0

args[2]=+b=6370000.0

setting Lambert Conic Conformal – 2SP parameters

PROJ args=+proj=lcc+a=6370000.0+b=6370000.0+lat_1=33.000000+lat_2=45.000000+lon_0=-97.000000+lat_0=40.000000+units=m

Total area = 1.6543e+13

Skipped 0 polygons and 0 vertices

File bounding box:

xmin = -2423753.500 xmax = 3235765.000 ymin = -2882042.250 ymax = 2841043.750

Finished reading the input shape file…

EV: ALLOCATE_ATTRS=TYPE

EV: ALLOCATE_ATTRS=TYPE

Not using function for weights

Using attribute TYPE

Finished reading the input attribute file…

xmin = -2423753.500 xmax = 3235765.000 ymin = -2882042.250 ymax = 2841043.750

xmin = -2292000.000 xmax = 2328000.000 ymin = -1584000.000 ymax = 1320000.000

numpoly1=20183, numpoly2 = 838530

xmin = -2423753.500 xmax = 3235765.000 ymin = -2882042.250 ymax = 2841043.750

(and then it hangs)

Is there some indication that I’ve got some mistake I’ve overlooked? Are the domains the wrong size?

Thanks,
Anastasia

3

Are you trying to create this 4km file for the entire continental US? What is your grid description?

4

Yes I am, this is my griddesc

5

This is a very large grid and your system may not be able to handle running it all at once.

Have you tried running the 12km version of this grid to see if it completes successfully to make sure the other parts of your configuration are correct?

image001.jpg

image004.jpg

6

I’m running the 12 km version now. With respect to my system not being able to handle it all at once, do you think there would be an overflow/bus error to indicate that? I’ve also resubmitted these jobs with 128GB mem per cpu, I’ve also run the allocator script across multiple nodes (but I don’t think that actually did anything).

7

Good idea. I’m not sure that there would be an overflow – it might just take a really long time to finish.

It has to intersect every grid cell with the Shapefile boundaries, so the more grid cells there are the longer it will take.

I don’t know that multiple nodes will help as I don’t know that the software is configured to make use of multiple nodes.

8

Yes the 12 km domain finished in 18 hours so the script/inputs are working, it just must take too much time to do. Is there a tool to downscale this 12 km oceanfile and put it on my 4 km grid?

9

I have two thoughts.

From surfzone_poly_st.prj, it has “SPHEROID[“Sphere_ARC_INFO”,6370997.0,0.0]” in it. So should we set "setenv INPUT_FILE_ELLIPSOID “+a=6370997.0,+b=6370997.0” in the run script?

Would it work if we divide the huge 4km grid into say 9 pieces and process them individually and piece the 9 output files into one?

1 Like
10

That INPUT ELLIPSOID should match the Shapefile as you mention.

Yes, I think it could be done in pieces at 4km and then “stitched together” using appropriate tools.

Perhaps the grids could be non-overlapping and cover the coastline areas.

We don’t really need for the tool to intersect all the 4km grid cells in the interior of the country.

The idea of taking the 12km and proportioning that to overlapping 4km grid cells is “lossy” in that the coastline will not be reflected at the 4km resolution.