Rob Church wrote:
On 05/07/06, Neil Harris neil@tonal.clara.co.uk wrote:
If this would not be practical, please let me know why.
The fact that we don't retain the logs is perhaps a wee bit prohibitive.
Rob Church _______________________________________________ Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
Aha. Then how about adding a patch to squid that goes something like:
if (do_throttled_logging && ((throttled_logging_count++ % throttled_logging_ratio) == 0)) generate_normal_squid_logfile_line(...);
that then generates a logfile output every throttled_logging_ratio hits? If this ratio is 1000, then logs one-thousandth of the normal size will be generated at one-thousandth of the normal load that might be expected from full-on logging.
Assuming N=1000, and 100 hits/second/squid, that would be one log entry per ten seconds on each machine, or a total of 10,000 log entries per day per machine, which would take up perhaps 1 Mbyte: rather different from the Gbyte of logfile which would be generated daily if full logging was turned on.
The logs can then be aggregated and analyzed in the usual ways.
-- Neil