yes the system is still used. the shards of the index are available for
download @
however that is only partial and
(afaik) relatively out of date. the better way would be to query the api of
the system.
also yeah changes in robots.txt would be nice :-p
regards
mark
On Sun, Aug 24, 2008 at 6:14 PM, Platonides <Platonides(a)gmail.com> wrote:
Anthony wrote:
On Sun, Aug 24, 2008 at 4:59 AM, wrote:
If the tool should be popular Yahoo! should
probably be contacted as the
load generated could be pretty huge.
Does anyone use Grub any more? Some kind of open source, distributed
system
would be a good long term solution. I hate to
give Jimbo a plug, but...
They're supposed to provide their full index for download, so they won't
probably have any problem. You could always fork it... if you had enough
space and computer power, that is.
However, before asking them something like that, we should lift the ban
on Grub, so it can crawl wikipedia ;)
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/foundation-l