Hi teachers,
I submitted job to run smk_edgar_HEMI108k.csh but it stuck in TEMPORAL program and no error returned in log files. It just run without stop for hours and not into SMKMERGE program. I checked invtable, gsref and gspro files and didnt find any issues.
temporal.edgar.HEMI_108k.log：
It could be an out-of-space issue, either related to lack of RAM or disk space. If the you delete the intermediate files and rerun temporal, does it get hung up at the same exact timestep?
Thank you! I changed serial ./smk_edgar.csh to mpi run it to get more RAM and it didnt stuck any more. But but now I want to run multiple SMOKE programs simultaneously (with each program processing a different source) to improve efficiency. I copied the original SMOKE folder and then started running several SMOKE programs separately. However, when running them at the same time, some of the programs always terminate unexpectedly, and in the end, only one program remains running. What could be causing this issue?
