-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
In the past, our policy on SGE (batch job scheduling) was that it should only be used for jobs which run once then exit, and that it should not be used for jobs which run continuously.
This policy has now changed, and SGE can be used for all jobs, including long-running jobs. This effectively obsoletes Phoenix and similar tools, since SGE with cronsub provides the same functionality.
Because SGE makes it easier for us to scale the Toolserver cluster, as well as making your tools run more efficiently (and therefore quicker), we recommend that all tools be converted to SGE where possible. The wiki page now has a quick-start section explaining how to do this:
https://wiki.toolserver.org/view/Batch_job_scheduling#Quick_start
(While SGE has a lot of features and can be complicated, we've tried to make it easy to use for the typical cases; if something is not clear, please let us know.)
- river.
2010/11/16 River Tarnell river.tarnell@wikimedia.de
This policy has now changed, and SGE can be used for all jobs, including long-running jobs. This effectively obsoletes Phoenix and similar tools, since SGE with cronsub provides the same functionality.
Thanks. A question. I'm a new toolserver user, presently working at a long list of personal "Hello world" tests. Next scheduled "Hello world" was cron test. Is your suggestion to step over cron, and to try to learn SGE from the beginning?
Alex brollo
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Alex Brollo:
Thanks. A question. I'm a new toolserver user, presently working at a long list of personal "Hello world" tests. Next scheduled "Hello world" was cron test. Is your suggestion to step over cron, and to try to learn SGE from the beginning?
If you're new to Unix, I would suggesting learning cron first, then adding SGE later. You'll need to use cron to run your SGE jobs anyway.
Converting an existing cron job to SGE is quite simple: https://wiki.toolserver.org/view/Batch_job_scheduling#Running_jobs_from_cron
- river.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Sorry for the flood of announcements about SGE today. I think this should be the last one.
This policy has now changed, and SGE can be used for all jobs, including long-running jobs. This effectively obsoletes Phoenix and similar tools, since SGE with cronsub provides the same functionality.
To make this work better, there is now a 'longrun' queue that should be used instead of the default 'all.q' for long-running jobs:
% qsub -q longrun -N myjob myjob.sh or: % cronsub -l myjob myjob.sh
- river.
toolserver-l@lists.wikimedia.org