On Mar 31, 2021, at 2:20 PM, Roy Smith
<roy(a)panix.com> wrote:
Is it feasible to do a log analysis of the database servers to find out what tools are
(were?) using cross-wiki joins? At least that would ensure that all the tool owners could
be contacted directly to make sure they know this is happening.
It’s not feasible to log all the queries, unfortunately. That grows log too fast to keep
on a disk (since we tried it). However, I am running a service to sample the queries at
random intervals, which doesn’t provide a complete list, but it’s been running for
something like a month now. As a result, it should be a pretty good list. I just did a
pull from that tool and can try to script a concept of who is doing cross-wiki joins and
let you know. It’s possible it would be quite doable once I’ve got the list parsed out.
Brooke Storm
Staff SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org