[Mediawiki-l] Help with crawling Special:AllPages for small proprietary wiki

[jidanni at jidanni.org]

This is https://bugzilla.wikimedia.org/show_bug.cgi?id=8473

The obvious solution is to make $wgArticleRobotPolicies work as
advertized and not be overpowered by hardwired code.

Also having users maintain private copies of includes/* usually lasts
until the next upgrade only, when different staff don't know about
previous tweaks.

Yes the user could also choose to maintain a sitemap, but that is
beside the point.