On Fri, Nov 12, 2021 at 6:58 PM Huji Lee huji.huji@gmail.com wrote:
I had the chance to try out these options.
YiFei was right in that the scripts, when invoked by crontab after zsh was activated, were being invoked by sh and that was why the source command was not working.
Using . (dot) would resolve the issue on command line, but I cannot tell jsub to use dot. My current crontab entries look like this:
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err ~/grid/jobs/hourly.sh
But neither of these are allowed:
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err . ~/grid/jobs/hourly.sh
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err ". ~/grid/jobs/hourly.sh"
The first one causes jsub to find two arguments and crash; the second one causes jsub to complain that ". ~/grid/jobs/hourly.sh" is not a program.
Apologies, I missed the emails.
By "." I mean to use it instead of "source". i.e. replace the "source" with "." in the scripts such as daily.sh [1], so that it looks like ". ~/venv/bin/activate".
[1] https://github.com/PersianWikipedia/fawikibot/blob/master/HujiBot/grid/jobs/...
I did a bit of testing it out and if the script does not have a shebang and is not an ELF executable, it will be executed as "-bash -c /path/to/scriptfile.sh". Since you have "exec zsh" in your .bash_profile, bash will run it as startup as a login shell, which in theory would immediately replace itself with zsh with no arguments. zsh will then see it has no arguments, attempts to read script from stdin and get nothing, and immediately exit, stopping the job in grid. Where dash gets involved in this process I do not know.
Honestly, I think you should not depend on the behavior of shebang-less scripts as the executable. You should either put "bash /path/to/scriptfile.sh" or add a shebang to top of the script.
YiFei Zhu
Roy Smith offered a trick to explore what environment was used when my script was run. So I create a small script with content "/usr/bin/env > ~/myenv.jsub.txt" and then asked jsub to run it. The job was successfully submitted, but myenv.jsub.txt file remained empty! It is as if no environment was passed, which is confusing.
Anyway, for now, I have given up on `exec zsh` and am going to just invoke zsh manually after I run `become`.
On Fri, Nov 12, 2021 at 1:28 PM Roy Smith roy@panix.com wrote:
Actually, now that I think about it more, in a remote execution environment, it's possible that by the time the script gets run, it's not even running on the same machine, which means it'll have its own /tmp. If that's the case, then my next thought is a NFS path to your home directory which is valid on any possible execution machine. And pre-creating the file mode 0666 becomes more likely to be necessary.
On Nov 12, 2021, at 1:16 PM, Roy Smith roy@panix.com wrote:
No matter how screwed up your env, path, choice of shell, output redirection, etc are, that's pretty much guaranteed to dump some useful information into someplace where you can find it.
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/