On Mar 31, 2021, at 2:20 PM, Roy Smith roy@panix.com wrote:
Is it feasible to do a log analysis of the database servers to find out what tools are (were?) using cross-wiki joins? At least that would ensure that all the tool owners could be contacted directly to make sure they know this is happening.
It’s not feasible to log all the queries, unfortunately. That grows log too fast to keep on a disk (since we tried it). However, I am running a service to sample the queries at random intervals, which doesn’t provide a complete list, but it’s been running for something like a month now. As a result, it should be a pretty good list. I just did a pull from that tool and can try to script a concept of who is doing cross-wiki joins and let you know. It’s possible it would be quite doable once I’ve got the list parsed out.
Brooke Storm Staff SRE Wikimedia Cloud Services bstorm@wikimedia.org