On Fri, Nov 12, 2021 at 6:58 PM Huji Lee <huji.huji(a)gmail.com> wrote:
I had the chance to try out these options.
YiFei was right in that the scripts, when invoked by crontab after zsh was activated,
were being invoked by sh and that was why the source command was not working.
Using . (dot) would resolve the issue on command line, but I cannot tell jsub to use dot.
My current crontab entries look like this:
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err
~/grid/jobs/hourly.sh
But neither of these are allowed:
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err .
~/grid/jobs/hourly.sh
jsub -N "h" -once -o ~/err/hourly.out -e ~/err/hourly.err ".
~/grid/jobs/hourly.sh"
The first one causes jsub to find two arguments and crash; the second one causes jsub to
complain that ". ~/grid/jobs/hourly.sh" is not a program.
Apologies, I missed the emails.
By "." I mean to use it instead of "source". i.e. replace the
"source"
with "." in the scripts such as daily.sh [1], so that it looks like ".
~/venv/bin/activate".
[1]
https://github.com/PersianWikipedia/fawikibot/blob/master/HujiBot/grid/jobs…
I did a bit of testing it out and if the script does not have a
shebang and is not an ELF executable, it will be executed as "-bash -c
/path/to/scriptfile.sh". Since you have "exec zsh" in your
.bash_profile, bash will run it as startup as a login shell, which in
theory would immediately replace itself with zsh with no arguments.
zsh will then see it has no arguments, attempts to read script from
stdin and get nothing, and immediately exit, stopping the job in grid.
Where dash gets involved in this process I do not know.
Honestly, I think you should not depend on the behavior of
shebang-less scripts as the executable. You should either put "bash
/path/to/scriptfile.sh" or add a shebang to top of the script.
YiFei Zhu
> Roy Smith offered a trick to explore what environment was used when my script was
run. So I create a small script with content "/usr/bin/env >
~/myenv.jsub.txt" and then asked jsub to run it. The job was successfully submitted,
but myenv.jsub.txt file remained empty! It is as if no environment was passed, which is
confusing.
>
> Anyway, for now, I have given up on `exec zsh` and am going to just invoke zsh
manually after I run `become`.
>
>
>
> On Fri, Nov 12, 2021 at 1:28 PM Roy Smith <roy(a)panix.com> wrote:
>>
>> Actually, now that I think about it more, in a remote execution environment,
it's possible that by the time the script gets run, it's not even running on the
same machine, which means it'll have its own /tmp. If that's the case, then my
next thought is a NFS path to your home directory which is valid on any possible execution
machine. And pre-creating the file mode 0666 becomes more likely to be necessary.
>>
>> On Nov 12, 2021, at 1:16 PM, Roy Smith <roy(a)panix.com> wrote:
>>
>> No matter how screwed up your env, path, choice of shell, output redirection, etc
are, that's pretty much guaranteed to dump some useful information into someplace
where you can find it.
>>
>>
>> _______________________________________________
>> Cloud mailing list -- cloud(a)lists.wikimedia.org
>> List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
> _______________________________________________
> Cloud mailing list -- cloud(a)lists.wikimedia.org
> List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/