First of all, thank you Ryan, Faidon, Brandon and others for the related thread. Indeed it feels like a rough environment sometimes, especially on lists, sometimes also on IRC and I really appreciated seeing colleagues step in for me in that way.
On Fri, Aug 17, 2012 at 1:59 PM, MZMcBride z@mzmcbride.com wrote:
I've always found you to be incredibly helpful on IRC, on the mailing lists, and elsewhere and I've always appreciated having you around. I apologize if my initial message suggested otherwise.
That said, also thank you for that, MZ. apology gladly accepted.
I read your reply to Guillom's post as "shit happens." And it most certainly does. But you said that the archives were last rebuilt two weeks ago, which is where the timeline kind of fell apart in my head.
That's understandable. To me it sounded like "he just walked away" when most of the messages had been sent during the night in PST and i had just arrived at the office. Until Guillaume brought it up on the list I thought of it as a drawback of removing mails, which i had mentioned before and which makes us do as little removals as possible but we would have to live with as it had happened before.
There was no communication to the list and its members and the archive being rebuilt two weeks ago and the consequences of doing so.
One of the reasons for not sending any announcement for this to the list was that it was about removing private data, so i did not want to go "Look, here is this private data i am now going to delete". Of course i could have still pointed out that archives are being rebuilt without giving the details.
Ok, back to the technical issue:
On Sat, Aug 18, 2012 at 1:19 AM, Guillaume Paumier gpaumier@wikimedia.org wrote:
After April 2012: The link http://lists.wikimedia.org/pipermail/wikitech-l/2012-July/061691.html was posted on meta to reference a message of mine from July 2012. That ID (061691) had to be changed to 061614 after the rebuild from 2 weeks ago (i.e. a translation of -77). After yesterday's rebuild, it's now at ID 061621 (a translation of +7 consistent with the 7 empty messages you've reinserted).
:( This is really unfortunate, but sorry, i don't have an explanation for the difference of -77 unless there has been another rebuild that i am not aware of or it actually was broken before my latest change...or it's a mailman bug..:/
So, it appears that the archives have been corrupted inconsistently besides the simple translations of -77 or -7. Someone can probably verify that with other links (e.g. from the Signpost pages).
Is it really random or at least consistent -77 since after April 2012? Well, but as Ryan pointed out this whole issue has happened several times in the past, so i expect you could always find broken links somewhere depending on the time they have been created.
the August 2012 archive page contains several "No subject" messages
I don't know where these come from. Are they really new since this incident? I would really prefer to not delete anything at this point and break links once again.
On Sat, Aug 18, 2012 at 11:35 AM, Platonides Platonides@gmail.com wrote:
If I remember right, the issue of deleting old mails was not just that the ids moved the number of deleted mails, but that when rebuilding the archive, new versions rebuilt it differently. Thus the changed numeration.
That would indeed explain inconsistencies in old links before April. I think though that we have rebuilt archives more than once with this current mailman version. Still would explain corruption from the past though.
Can we restore the old files from backups?
Again, really unfortunate, but we can't at this point. Backups are going back one week.
I'm afraid the best i can do now is to help fixing external links. If somebody wants to point me to WP pages with broken links to mailman archives, i would gladly help to fix them in an edit sprint.
Daniel Zahn dzahn@wikimedia.org Operations Engineer