We now have squid logs again, after a long absence. We're using the UDP
logging patch I describe here:
http://www.squid-cache.org/mail-archive/squid-dev/200701/0042.html
The log collection program is running on henbane. The log format is our
own custom format described here:
https://wikitech.leuksman.com/view/Squid_log_format
Only a sampled log will be stored, the full log will only be available as
a real-time stream.
I'd like to invite submissions at this stage for log analysis programs,
which can process either the full stream or a sampled stream, aggregate
the data and present statistics on the web. There are lots of log analysis
packages already available, but I haven't found one yet that is designed
to work at such a high request rate.
Currently we only have a raw log, which contains duplicates due to
requests forwarded between squids. These should probably be filtered out
by the collection daemon before the log is handed out to the analysers.
-- Tim Starling