I was pointed to this section of the HTML 4.0 spec ("Notes on helping search engines index your Web site") at W3C recently:
http://www.w3.org/TR/REC-html40/appendix/notes.html#h-B.4
I was specifically interested in this section, which I quote in full:
Specify language variants of this document
If you have prepared translations of this document into other languages, you should use the LINK element to reference these. This allows an indexing engine to offer users search results in the user's preferred language, regardless of how the query was written. For instance, the following links offer French and German alternatives to a search engine:
<LINK rel="alternate" type="text/html" href="mydoc-fr.html" hreflang="fr" lang="fr" title="La vie souterraine"> <LINK rel="alternate" type="text/html" href="mydoc-de.html" hreflang="de" lang="de" title="Das Leben im Untergrund">
I think it'd be useful for most multilingual MediaWiki installations that use interlanguage links to have such hidden <link> elements. <link> elements aren't rendered in most browsers (Mozilla 1.5+, I think, will show links in a menu on the toolbar), but as mentioned they do provide some guidance to bots and spiders.
The downside is that they still take up network bandwidth, and for oft-interwikied pages on big sites (e.g. Wikipedia) this section could run to the 5-10kB size range. (My off-the-cuff estimate for, say, articles where each <link> is about 100B, and there are 50-100 interwiki links.)
Barring objections I'm going to add a feature to HEAD to render these <link> elements, controlled by a variable $wgInterLanguageLinkElement , default to false.
~Evan