How to have a post-processing job in condor dagman, running at the end of a batch, regardless of parent job failures?
Issue
Having a very simple DAG, with 2 levels:
JOB A a.condor
RETRY A 5
JOB B a.condor
RETRY B 5
JOB C a.condor
RETRY C 5
JOB PostPROCESS pp.condor
PARENT A B C CHILD PostPROCESS
- Jobs A - C are processing jobs: each does something which sometimes fails, therefore having
RETRY 5 - a post-processing job that aggregates results from A - C and produces a report.
It is possible that some of the jobs A - C fail eventually (even after the 5 retries). It is desired the PostPROCESS job to run, even in this case.
Otherwise said, the PostPROCESS should be be just a synchronization point, and executed regardless of the result of the jobs A-C, but only after each of A-C has been given its chance to run.
How can one achieve this?
Environment
- MRG Grid (any version)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
