Hey Aaron,
(removing E3 and adding wmfresearch)
can you recap the main use case for the revert tables generated by reverts.py? We've been thinking of moving them to the prod DB but now that we have SHA1 population completed in enwiki AND revert rates implemented in the metrics API I am curious about what you use this for. If we were to make this a permanent table in prod we should definitely have the script in a public repo as a starter.
Dario
On Mar 22, 2013, at 10:42 AM, Ori Livneh ori@wikimedia.org wrote:
It's useful to have such things in a public repo so people can take a peek at your code if it goes crazy and suggest improvements :)
-- Ori Livneh
On Friday, March 22, 2013 at 9:05 AM, Aaron Halfaker wrote:
Hey folks,
I was running a script to update the revert tables on db1047 with stat1 two days ago that had some bad disk access patterns. (FYI, don't use python shelve as an on-disk cache of a dict().) As soon as I saw the load come up, I killed the script. For any difficulty that occurred in the meantime, I'm very sorry. I've since re-written things to behave much better.
I currently have two processes running on the machine: sessions.py - Updating session table on db1047. Useful for measuring editor labor hours. reverts.py - Updating revert tables on db1047. Fixed to not need a disk cache.
Both of these processes are nice'd, so they should wait in line for CPU access behind any non-nice'd processes you have running. If the processes cause any trouble, please feel free to kill them or let me know and I'll kill them.
For Science, -Aaron
E3-team mailing list E3-team@lists.wikimedia.org (mailto:E3-team@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/e3-team
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics