How to have a post-processing job in condor dagman, running at the end of a batch, regardless of parent job failures?
Issue
Having a very simple DAG, with 2 levels:
JOB A a.condor
RETRY A 5
JOB B a.condor
RETRY B 5
JOB C a.condor
RETRY C 5
JOB PostPROCESS pp.condor
PARENT A B C CHILD PostPROCESS
- Jobs A - C are processing jobs: each does something which sometimes fails, therefore having
RETRY 5 - a post-processing job that aggregates results from A - C and produces a report.
It is possible that some of the jobs A - C fail eventually (even after the 5 retries). It is desired the PostPROCESS job to run, even in this case.
Otherwise said, the PostPROCESS should be be just a synchronization point, and executed regardless of the result of the jobs A-C, but only after each of A-C has been given its chance to run.
How can one achieve this?
Environment
- MRG Grid (any version)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
