The problem with redirecting in the shell is that my script can't access it's own output, which I sometimes need. Also, unlike my solution, which lists all output in the order in which it were produced regardless of whether it was sent to stdout or stderr, shell redirection (at least when I tried it yesterday) produces a file that list everything sent to stdout followed by everything sent to stderr, so things aren't in the proper order.

It also means that to email it to myself, I have to run the main script, redirecting output to a file, then run a second Python script to read that file and send the email. Running two scripts sequentially with one command requires a batch file, which produces an ugly console window that interrupts whatever I'm doing when the Task Scheduler triggers the script.

All of those are why I prefer to redirect output within the Python environment.

Jonathan Goble

On Tue, Jun 24, 2014 at 6:05 PM, John <> wrote:
Or if your going to do mass redirecting dont do it in the program, use terminal redirecting like > to do it in a sane manor

On Tue, Jun 24, 2014 at 6:02 PM, Jonathan Goble <> wrote:
I found the cause. I have several functions that are used by multiple scripts, so I have those in a custom module that each of my scripts imports. I recently added a subclass of catlib.Category to that module for use in a new script that will only run a couple times per month. To do that, I had to import pywikibot and catlib to that module, and it was the addition of those imports that broke things.

The reason is that that module also contained a subclass of StringIO that handles mixed str/unicode input better, and therefore I had to import it before I could redirect stdout and stderr to an instance of that class. In performing that import, the code in the module was run, which caused pywikibot to be imported, and pywikibot assigned the then-current stdout and stderr to pywikibot.ui. The subsequent redirection of stdout and stderr was then not reflected in pywikibot.ui, resulting in pywikibot.output() sending everything to the console instead of my StringIO subclass instance.

To fix it, I copied my StringIO subclass into one of my scripts proper, and made sure to redirect stdout and stderr before importing pywikibot or any other module that would import pywikibot. Subsequent tests successfully sent all output to my email.

Moral of the story: if you're going to redirect stdout and stderr by simply importing sys and reassigning sys.stdout and sys.stderr to something else, make sure to do it BEFORE you import pywikibot or any module that would import pywikibot. (Although it does kinda seem like a bug that pywikibot doesn't respond to future changes to sys.stdout and sys.stderr; maybe the output function in the interface class should check whether either of those has changed, and adjust its setup as needed?)

Jonathan Goble

On Tue, Jun 24, 2014 at 5:25 AM, Merlijn van Deen <> wrote:
Could you provide a minimal example that would allow us to reproduce the issue? With core, the following works for me (I don't have compat installed):

import sys, StringIO
oldstderr = sys.stderr
oldstdout = sys.stdout

sys.stderr = sys.stdout = StringIO.StringIO()
sys.stderr.fileno = lambda: 10

import pywikibot
oldstdout.write("Before output\n")

oldstdout.write("After output; this is in our buffer:\n")

$ PYWIKIBOT2_DIR=. python
Before output
After output; this is in our buffer:
'WARNING: Running on Windows and transliteration_target is not set.\nPlease see

Nonetheless, I would suggest to use normal shell redirection techniques, instead of monkey-patching python's sys module. If you want to use python, the following should work (untested):

import subprocess
stdout, stderr = subprocess.Popen(["c:\python27\python", ""], stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
# do something with stdout and stderr

I *think* it should also be possible to do this with the logging module, but I'm not sure how to do that in a quick-and-easy way.


On 24 June 2014 09:52, Jonathan Goble <> wrote:
When the error began occurring, I was using the Pywikibot nightly dump from February 25 of this year. It continues to occur with the nightly dump from yesterday (June 23). I just tried the fileno = lambda: 10 trick and it had no effect.

As for core, it's been four months since I last tried it so I don't recall the issues. It's past bedtime for me right now, but I'll look into giving it another try within the next day or two when I have some time.

On Tue, Jun 24, 2014 at 3:17 AM, Merlijn van Deen <> wrote:
On 24 June 2014 06:17, Jonathan Goble <> wrote:

I've given up trying to solve a bug that popped up in my scripts a couple days ago. I run a bot for Wookieepedia, over at Wikia, and run three simple scripts on a daily basis. They are set up to run automatically through Windows Task Scheduler. Since they run automatically, they run in the background through pythonw.exe, i.e. without a console, and therefore I need a means of getting the output. My solution for the past two months has been to redirect sys.stdout and sys.stderr to the same StringIO() instance, then at the end call getvalue() on that and email it to myself.

Could it be you haven't updated in a few years? It sounds like it's related to a feature that I added two or three years ago, that allowed Windows users to get full unicode out- and input. However, that does mean sys.stdin and sys.stdout are no longer being used. We /do/ check whether the user has redirected the output using normal shell redirection, but your method doesn't do that.

There are a two options I can think of.

  - trick the code into thinking you're doing 'regular' redirection by adding a fileno function to your streams:

    you'd need something like x = StringIO.StringIO(); x.fileno = lambda: 10
    and the same for stdout.

(By the way, the answer is NOT "switch to core". I have tried to get core to run on my system and failed miserably after two hours of repeated attempts without even getting it to talk to the wiki. Compat worked perfectly on the first try. Until such time as core can be installed by a beginner, it is not for me.)

I would appreciate it if you could clarify what the issues were you ran into.


Pywikipedia-l mailing list

Pywikipedia-l mailing list

Pywikipedia-l mailing list

Pywikipedia-l mailing list

Pywikipedia-l mailing list