[Toolserver-l] question about performance/getting webpage content

seth email_metawiki_138 at wg-karlsruhe.de
Mon Nov 24 20:31:28 UTC 2008


Hi!

I wrote a perl script, which works on some HTML content of some 
wikipedia-webpages. Some of those pages are >300kB and perls LWP-mirror 
hangs up.

Two questions:
1. Is there a better/faster way to get the HTML content of e.g. 
http://meta.wikimedia.org/wiki/Spam_blacklist/Log
than
   my $ua = LWP::UserAgent->new;
   $ua->mirror($url, $filename);
?
2. If I've questions about such stuff, am I right here? Otherwise, sorry 
for bothering you. :-)

Cheers
seth



More information about the Toolserver-l mailing list