In article AANLkTi=vXRKLQLs9QJ1Byj25NTAhmKQjXYXcCMGyMK3M@mail.gmail.com, Magnus Manske magnusmanske@googlemail.com wrote:
Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo meaning from my scripts (which predate qsub availability) anyway?
Because it means you can include directives in your script indicating how it should run:
#! /bin/ksh #$ -o $HOME/mytool.out #$ -l h_rt=3:0:0 #$ -t 1-10 #$ -tc 3 # rest of script...
This is easier than having a separate script just to start that one.
If that is desired for some cases, why is that activated by default?
With SGE, it's usually desirable for *all* cases (or at least the vast majority). The cronsub script makes some of the most common uses unnecessary (e.g. options controlling output file), so it's used less there. However, turning it off for cronsub while it was enabled for qsub seemed like it would be more confusing for users.
Is there at least a qsub option to turn it off?
Yes.
Will that option work with cronsub?
No. But you can invoke qsub yourself if you prefer that. (This is a known issue but it's somewhat difficult to fix because of how cronsub works.)
In my "day job", I work with a large compute farm (lfs, ~10K cores), so I'm more than happy to work with, and migrate existing cronjobs to, this system, but it would help if it weren't a deliberate minefield...
I didn't write SGE. I considered several existing batch scheduling systems and SGE seemed to be the best/most useful. Of course that doesn't mean every user will like every feature it has.
- river.