On Fri, Sep 30, 2011 at 12:56 PM, Andre Engels andreengels@gmail.comwrote:
The interwiki links are retrieved from page content. The page content has been received through a call to Special:Export.
I.e. would receiving no content (from the bot POV) produce that behavior?
Yes, the only reasonable explanation seems to be that the bot interprets what it gets from the server as an empty page.
So you screen-scrape? No surprise it breaks. Why? For example, due to protocol-relative URLs. Or some other changes to HTML output. Why not just use API?