Brion Vibber wrote:
A text base without the unnecessary UI elements could improve search results, and I suppose can be kept more complete more easily than constant spidering of a 200k+ page site. *shrug* If that's the data format they want, hey fine, though having to download the entire set of a couple hundred megabytes for every update doesn't sound ideal.
If the problem is that each article has navigational elements on the top, bottom, and left, which multiplies the time required to crawl things, then how about allowing something like &raw=1 which would output only the parsed page, without a skin or anything, with all the links in it also pointing to &raw=1, and then arranging with Yahoo to have them spider these pages, but still send people to the URL without &raw=1?