Hello,
I’m trying to run an ozone-only simulation (sort of a semi dry-run). I have deactivated most of science options (CTM_OCEAN_CHEM=‘N’ … CTM_BIOGEMIS_BE=‘N’ … ) with no streams (N_EMIS_GR=0 & N_EMIS_PT=0). However, and when I try to run the case, I get a seg fault immediately after CMAQ acknowledges that no emission streams are to be read:
================================
|>--- TIME INTEGRATION ---<|
================================
Processing Day/Time [YYYYDDD:HHMMSS]: 2023069:000000
Which is Equivalent to (UTC): 0:00:00 Friday, March 10, 2023
Time-Step Length (HHMMSS): 000500
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0xffffb8ac5e9b in ???
#1 0xffffb8ac4f27 in ???
#2 0xffffb913c78b in ???
#3 0xffffb88bbf74 in ???
at …/sysdeps/aarch64/memset.S:53
#4 0x56645b in ???
#5 0x5aef03 in ???
#6 0x572823 in ???
#7 0x56c7ab in ???
#8 0x56ba57 in ???
#9 0x404a93 in ???
#10 0xffffb8856e0f in __libc_start_main
at …/csu/libc-start.c:308
#11 0x404ad3 in ???
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
mpirun noticed that process rank 0 with PID 0 on node ip-172-31-58-210 exited on signal 11 (Segmentation fault).
Command exited with non-zero status 139
with the log showing
===========================================
|>--- INITIALIZE EMISSIONS MODULE ---<|
===========================================
|> Initialize Emissions Input Files:
+=====================================
>>--->> WARNING in subroutine DESID_INIT on PE 000
No Emissions Streams Have Been Selected.
M3WARN: DTBUF 0:00:00 March 10, 2023(2023069:000000)
The 2018_12NE3 case runs smoothly so I’m not sure which is the problem. I was wondering if CMAQ_Control_DESID.nml would require some modification or what else might be the problem.
Thanks.
(I took the liberty of changing the title of this thread to describe more precisely what it is you are doing with this test.)
You don’t say what version of CMAQ you are using, but I was able to run CMAQv5.4 in the manner you describe:
setenv N_EMIS_GR 0
setenv N_EMIS_PT 0
setenv CTM_WB_DUST N
setenv CTM_ BIOGEMIS_BE N
setenv CTM_BIOGEMIS_MG N
setenv CTM_OCEAN_CHEM N
and the model runs. The log file produces this appropriate message:
===========================================
|>--- INITIALIZE EMISSIONS MODULE ---<|
===========================================
|> Initialize Emissions Input Files:
+=====================================
>>--->> WARNING in subroutine DESID_INIT on PE 000
No Emissions Streams Have Been Selected.
Hi Chris,
Totally cool with the title change. If you’re not getting the seg fault, and my icbc & met files are there, it probably means that something in my compilation is not quite right and needs some debugging.
Thanks,
Arturo
Hello Chris,
I apologize for the delay in providing further feedback but this is kind of an obscure issue (the previous 2 weeks were hectic and I was only been able to sit down and focus on solving the problem over the long weekend). The version is 5.4 and this is critical because the component causing the crash seems to be new. The subroutine causing the crash is DESID_DRIVER. If I comment out the call to this subroutine and set EMIS_AP equal to 0.0, CMAQ runs smoothly (as it does with the 12NE3 &12US1 benchmarks).
Thanks,
Arturo
P.S. The code is using acm2_m3dry not acm2_stage.
Arturo,
We will need a few more details to diagnose this issue.
What case are you running? If it is your own case, please upload your run script, the main log file, and the CTM_LOG_000 file. If you modified the bldit_cctm.csh script, please provide the CCTM_v54.cfg file also.
Another thing to try: recompile the model in debug mode. One way to do this is to modify bldit_cctm.csh and uncomment this line:
set Debug_CCTM.
I recommend also modifying the line:
set VRSN = v54
to something like:
set VRSN = v54_dbg
Rerun the build script, then run the model, making sure that your run script points to the debug version of the executable.
The log file should have a more helpful error message pinpointing where the seg fault is occurring.
Hi Chris,
This has become more of a potential bug report than a help request as the cases are running (w/o the previous problem) just by following the steps previously mentioned. This is a custom case so I cannot share the full scripts (they are in bash -not Cshell- anyway). Any potential issue might be in the desid_module but I still haven’t had time to track down where the wrong memory reference is happening.
Arturo