Hello Everyone,
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
UAA.out says:
newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed: No such file or directory
. My files are set for CHMOD 774
. They all exist
. It always returns the error
What am I doing wrong?
--------------------------
DeltaQuad
What does this produce:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/set.out set
-Jeremy On Apr 13, 2011 11:54 AM, "DeltaQuad" deltaquad@live.ca wrote:
Hello Everyone,
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
UAA.out says:
newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed: No such file or directory
. My files are set for CHMOD 774
. They all exist
. It always returns the error
What am I doing wrong?
DeltaQuad
Can’t say right now as my school blocks FTP/SSH to the toolserver, will let you know later today.
--------------------------
DeltaQuad
http://enwp.org/User_Talk:DeltaQuad http://enwp.org/User_Talk:DeltaQuad
From: toolserver-l-bounces@lists.wikimedia.org [mailto:toolserver-l-bounces@lists.wikimedia.org] On Behalf Of Jeremy Baron Sent: April-13-11 12:01 PM To: toolserver-l@lists.wikimedia.org Subject: Re: [Toolserver-l] Python/Job issues
What does this produce:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/set.out set
-Jeremy
On Apr 13, 2011 11:54 AM, "DeltaQuad" deltaquad@live.ca wrote:
Hello Everyone,
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
UAA.out says:
newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed: No such file or directory
. My files are set for CHMOD 774
. They all exist
. It always returns the error
What am I doing wrong?
DeltaQuad
:-/
Maybe someone should set up a TS login server sshd to listen on port 443 aka https. (if it's not already claimed by a web service)
-Jeremy On Apr 13, 2011 12:06 PM, "DeltaQuad" deltaquad@live.ca wrote:
Can’t say right now as my school blocks FTP/SSH to the toolserver, will
let you know later today.
DeltaQuad
http://enwp.org/User_Talk:DeltaQuad http://enwp.org/User_Talk:DeltaQuad
From: toolserver-l-bounces@lists.wikimedia.org [mailto:
toolserver-l-bounces@lists.wikimedia.org] On Behalf Of Jeremy Baron
Sent: April-13-11 12:01 PM To: toolserver-l@lists.wikimedia.org Subject: Re: [Toolserver-l] Python/Job issues
What does this produce:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/set.out set
-Jeremy
On Apr 13, 2011 11:54 AM, "DeltaQuad" deltaquad@live.ca wrote:
Hello Everyone,
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
UAA.out says:
newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed:
No
such file or directory
. My files are set for CHMOD 774
. They all exist
. It always returns the error
What am I doing wrong?
DeltaQuad
On 13 April 2011 17:14, Jeremy Baron jeremy@tuxmachine.com wrote:
Maybe someone should set up a TS login server sshd to listen on port 443
aka https. (if it's not already claimed by a web service)
An alternate SSH port would be trivial to set up, but 443 maybe isn't the best port, as this would prevent possible future access to the toolserver over https (unless it was changed)
On Apr 13, 2011 12:06 PM, "DeltaQuad" deltaquad@live.ca wrote:
Can’t say right now as my school blocks FTP/SSH to the toolserver, will
let you know later today.
Heh, I can't say I blame them. SSH was always my way around the filtering at school, combined with portable firefox. (socks5 proxy using dynamic port forwarding if you want to know :P)
Simon
On 04/13/2011 08:17 PM, Simon Walker wrote:
On 13 April 2011 17:14, Jeremy Baron <jeremy@tuxmachine.com mailto:jeremy@tuxmachine.com> wrote:
Maybe someone should set up a TS login server sshd to listen on port
443 aka https. (if it's not already claimed by a web service)
An alternate SSH port would be trivial to set up, but 443 maybe isn't the best port, as this would prevent possible future access to the toolserver over https (unless it was changed)
If it's only on a login server, I don't see any problem. Those servers shouldn't be serving web pages anyway.
Not sure if there's really that much need for it, though.
On Apr 13, 2011 12:06 PM, "DeltaQuad" <deltaquad@live.ca
mailto:deltaquad@live.ca> wrote:
Can’t say right now as my school blocks FTP/SSH to the toolserver,
will let you know later today.
Heh, I can't say I blame them. SSH was always my way around the filtering at school, combined with portable firefox. (socks5 proxy using dynamic port forwarding if you want to know :P)
BTW, if you have access to an HTTP(S) proxy, you may be able to tunnel SSH over that. Just Google for "ssh over https".
It produces the command line error:
Unable to read script file because of error: error opening set: No such file or directory
Instead of the text file.
--------------------------
DeltaQuad
http://enwp.org/User_Talk:DeltaQuad http://enwp.org/User_Talk:DeltaQuad
From: toolserver-l-bounces@lists.wikimedia.org [mailto:toolserver-l-bounces@lists.wikimedia.org] On Behalf Of Jeremy Baron Sent: April-13-11 12:14 PM To: toolserver-l@lists.wikimedia.org Subject: Re: [Toolserver-l] Python/Job issues
:-/
Maybe someone should set up a TS login server sshd to listen on port 443 aka https. (if it's not already claimed by a web service)
-Jeremy
On Apr 13, 2011 12:06 PM, "DeltaQuad" deltaquad@live.ca wrote:
Can’t say right now as my school blocks FTP/SSH to the toolserver, will let you know later today.
DeltaQuad
http://enwp.org/User_Talk:DeltaQuad http://enwp.org/User_Talk:DeltaQuad
From: toolserver-l-bounces@lists.wikimedia.org [mailto:toolserver-l-bounces@lists.wikimedia.org] On Behalf Of Jeremy Baron Sent: April-13-11 12:01 PM To: toolserver-l@lists.wikimedia.org Subject: Re: [Toolserver-l] Python/Job issues
What does this produce:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/set.out set
-Jeremy
On Apr 13, 2011 11:54 AM, "DeltaQuad" deltaquad@live.ca wrote:
Hello Everyone,
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
UAA.out says:
newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed: No such file or directory
. My files are set for CHMOD 774
. They all exist
. It always returns the error
What am I doing wrong?
DeltaQuad
DeltaQuad wrote:
I am having some issues with Job Scheduling with python. I always get:
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py Your job 314397 ("UAA.py") has been submitted
UAA.out says: newtask: exec of /sge62/default/spool/yarrow/job_scripts/314397 failed: No such file or directory
As far as I can see, this is due to /home/deltaquad/UAA/UAA.py using Windows line endings (CRLF) instead of Unix line endings (LF).
MZMcBride
deltaquad@willow:~$ qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
Your job 314397 ("UAA.py") has been submitted
Can you try this:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -b y -m e -j y -o $HOME/UAA.out $HOME/UAA/UAA.py
(I added the "-b y" option and it seems to work here).
//Marcin
You should read https://wiki.toolserver.org/view/Mailing_list_etiquette
when you get a chance.
Sorry about that. Read it now.
As far as I can see, this is due to /home/deltaquad/UAA/UAA.py using
Windows line endings (CRLF) instead of Unix line endings (LF).
I've ran this on my Linux Mint (with the change of the pywikipedia directory only) and it runs fine. (Not sure if that helps)
Can you try this:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -b y -m e -j y -o $HOME/UAA.out
$HOME/UAA/UAA.py
(I added the "-b y" option and it seems to work here).
I get this instead in the output file now: " newtask: exec of /home/deltaquad/UAA/UAA.py failed: No such file or directory"
-------------------------- DeltaQuad http://enwp.org/User_Talk:DeltaQuad
DeltaQuad deltaquad@live.ca wrote: You should read https://wiki.toolserver.org/view/Mailing_list_etiquette
when you get a chance.
Sorry about that. Read it now.
As far as I can see, this is due to /home/deltaquad/UAA/UAA.py using
Windows line endings (CRLF) instead of Unix line endings (LF).
I've ran this on my Linux Mint (with the change of the pywikipedia directory only) and it runs fine. (Not sure if that helps)
Can you try this:
qsub -l h_rt=0:05:00 -l s_rt=0:04:30 -b y -m e -j y -o $HOME/UAA.out
$HOME/UAA/UAA.py
(I added the "-b y" option and it seems to work here).
I get this instead in the output file now: " newtask: exec of /home/deltaquad/UAA/UAA.py failed: No such file or directory"
This is because your CRLF Windows line endings in your file. I copied your file to my $HOME and got the same problem.
SGE is not exactly the same as running your script from shell.
Besides, you don't need #! /usr/bin/python twice and if you use "-b y" your #$ declarations won't work and should be removed.
//Marcin
Marcin Cieslak wrote:
This is because your CRLF Windows line endings in your file. I copied your file to my $HOME and got the same problem.
Right. I said this already. ;-)
SGE is not exactly the same as running your script from shell.
From a bit of testing, it seems like Python doesn't care if a script uses
CRLF line endings or LF line endings, but Unix's exec() does. When SGE calls exec() on the Python script with the CRLF line endings, it can't get past the first line and then dies with a mostly unhelpful error message.
DeltaQuad: I tested this pretty extensively last night. Seriously, just fix your line endings and you should be good to go. :-)
MZMcBride
toolserver-l@lists.wikimedia.org