On 5 Apr 2005, at 16:33, Ira Abramov wrote:
when you are a bot that has to slurp up millions of pages a day, it's safe to assume in 99.99% of the cases, that a jpg suffix will indeed lead you to an image. requesting that URL just to see that the header indeed gives one MIME type or the other means adding a considderable overhead.
What overhead? If you're loading it anyway, you should look at the MIME type. Otherwise, it's just lazy, sloppy programming.
A problem of greater concern is links that send an image with the proper MIME type *without* putting ".jpg" at the end of the URI. In that case, you do make more work for spiders, if you expect them to index your "hidden" images.
:::: Getting a personal computer is sorta like getting married so you'll have someone to help you with all the problems you never would have had if you had never gotten married in the first place. :::: Jan Steinman http://www.Bytesmiths.com/Item/794637