On 1/17/06, Brock Batsell wikipedia@theskeptik.com wrote:
On Jan 17, 2006, at 3:59 PM, Sam Korn wrote:
Can robots.txt use wildcards?
Not according to the Robots Exclusion Standard, so no.
Regards, [[en:User:Bbatsell]]
Isn't there something you can put in the head of a html page that makes search engines not index that page. It would be trivial to change the mediawiki code to include that in all articles starting with "Wikipedia:Articles for Deletion", wouldn't it?
The whole blanking it thing and linking to the history, I'm not sure that would work, wouldn't the robot simply follow the link and index that page? It would probably be lower in the scores, but it would still show up in searches.
- Oskar