On Jan 15, 2004, at 09:54, Sascha Noyes wrote:
How do things like google's image search work. I believe they also load images from other's servers. Is there anything specified in robots.txt about this?
Google's image search loads the images (as it does the text) while spidering, then they create a thumbnail image and keep it locally. Browsing the thumbnailed results doesn't impinge on the servers where the images came from at all.
When you click a thumbnail, it sends you to a framed document which loads the page where it came from off the original server, and also in the top frame the image alone.
-- brion vibber (brion @ pobox.com)