I note that MNT-1225, which was originally projected to take over 24 hours, is now closing in on 72 hours and there doesn't seem to be any meaningful way to project a completion time. Nothing we can do about that now, of course.
But, I am wondering whether it was necessary to run these updates on *both* replicas of s1 (s1-sql-rr and s1-sql-user) at the same time? This is really a naive question, as I don't know enough about mysql administration to even guess at the answer. Perhaps someone more knowledgeable can enlighten us. (I do observe, however, that WMF seems to have managed to update all of its database slaves in some kind of sequential fashion that didn't impact access to enwiki.)
If, in fact, there is no technical requirement for updating both replicas at the same time, I would suggest that the next time a situation like this arises, it would make more sense to do the updates sequentially so that users (both toolserver users and tool users) are not deprived of access to this resource for such a long time.
Russ
Hello, At Thursday 22 March 2012 21:36:01 DaB. wrote:
But, I am wondering whether it was necessary to run these updates on both replicas of s1 (s1-sql-rr and s1-sql-user) at the same time? This is really a naive question, as I don't know enough about mysql administration to even guess at the answer. Perhaps someone more knowledgeable can enlighten us. (I do observe, however, that WMF seems to have managed to update all of its database slaves in some kind of sequential fashion that didn't impact access to enwiki.)
If, in fact, there is no technical requirement for updating both replicas at the same time, I would suggest that the next time a situation like this arises, it would make more sense to do the updates sequentially so that users (both toolserver users and tool users) are not deprived of access to this resource for such a long time.
yes, for normal you would be right. There are 2 buts here: First: There is only 1 s1-server (rosemary) at the moment (sql-s1-user and sql-s1-rr points to the same server) because the other server (thyme) is corrupt and we wait for a dump by the foundation for a re-import. Second: I THOUGHT that the schema-change on s1 were already done before; otherwise I would had applied the change on rosemary while the user-databases were still on thyme. If for any reason thyme is faster done as rosemary, I will move the user-database back to thyme to speed up the change on rosemary and give you a writable database.
Sincerely, DaB.
Hello,
On Thu, 22 Mar 2012, Russell Blau wrote:
I note that MNT-1225, which was originally projected to take over 24 hours, is now closing in on 72 hours and there doesn't seem to be any meaningful way to project a completion time. Nothing we can do about that now, of course.
But, I am wondering whether it was necessary to run these updates on *both* replicas of s1 (s1-sql-rr and s1-sql-user) at the same time? This is really a naive question, as I don't know enough about mysql administration to even guess at the answer. Perhaps someone more knowledgeable can enlighten us. (I do observe, however, that WMF seems to have managed to update all of its database slaves in some kind of sequential fashion that didn't impact access to enwiki.)
I could have added the field before for enwiki too as we did for the other databases. Then exchanging infos with the WMF guys would have done the job I think much better than this time.
Cheers nosy
toolserver-l@lists.wikimedia.org