batch processing

Latest response

I am running RHEL 5.6.

I have many concurrent interactive users working on the system and when they too many batch jobs run concurrently, they cause file contention. Several concurrent batch jobs can impact hundreds of interactive users.
Is there an add-on to Linux to restrict the number of concurrent batch jobs?

Responses

You can provide process limits for individual users using /etc/security/limits.conf. You can adjust job priority at start time (or during execution) using nice/renice. You can also adjust scheduling via cron to run important and long-running jobs at different times to improve efficiency.

If the batch jobs are regularly scheduled, consider modifying them to use a file-based lock to prevent concurrency (look at the 'flock' command.)

Finally, a more featureful scheduler (like 'whenjobs', written by an employee at Red Hat) might be useful.