Historical Wikipedia edits on excel files - Wiki-research-l

Jodi Schneider

16 May 16 May

5:52 p.m.

Great work, Lior! I don't have any immediate need for these, but I do want them to be stewarded by the community. Please do find a place to deposit them (an institutional repository, Google Docs, the Internet Archive, ...). We will definitely want to make use of them in the future! -Jodi On Mon, May 16, 2011 at 4:46 PM, lior gimel <lior.gimel(a)gmail.com> wrote:

...

Hi, Following the discovery of the Wikipedia backups from 2001, and with much appreciated help from Jospeh Reagle, I created excel files from the diff_log and rcl_log files that together include all the edits from January 15th to August 17th (the last in the backups). the files include the time of edit and the editor's username and IP. While they are a bit buggy, and could use some more editing (or preferably, conversion to wiki format), they are simple, easy to work with (using searches and filters), and full of interesting stuff. If anyone is interested, I'd be happy to share them. Lior _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Reply

Piotr Konieczny

17 May 17 May

8:45 p.m.

lior gimel wrote:

...

Hi, Following the discovery of the Wikipedia backups from 2001, and with much appreciated help from Jospeh Reagle, I created excel files from the diff_log and rcl_log files that together include all the edits from January 15th to August 17th (the last in the backups). the files include the time of edit and the editor's username and IP. While they are a bit buggy, and could use some more editing (or preferably, conversion to wiki format), they are simple, easy to work with (using searches and filters), and full of interesting stuff. If anyone is interested, I'd be happy to share them.

Perhaps a public distribution on Google Docs would allow interested researches to do so, wiki-style? -- Piotr Konieczny PhD Candidate Dept of Sociology Uni of Pittsburgh "To be defeated and not submit, is victory; to be victorious and rest on one's laurels, is defeat." --Józef Pilsudski

Reply

Dario Taraborelli

18 May 18 May

12:37 a.m.

...this reminds me that I was planning to invite people on this list to comment on a review of open data sharing options but I forgot to! As part of the overhaul of Meta:Research mentioned in my previous post [1] we are trying to identify the best solution to set up an open repository for publishing and sharing research data among wiki researchers. There is a lot of resistance to data sharing in many research communities, but we believe the Wikimedia Foundation could play a key role in spearheading this initiative within the wiki research community. If you have experience using open data repositories and would like to share with us your thoughts, please send us your feedback at http://bit.ly/OpenDataPlatforms Dario [1] http://lists.wikimedia.org/pipermail/wiki-research-l/2011-May/001506.html On May 17, 2011, at 1:00 PM, lior gimel wrote:

...

Hi, I doing a little makeover on the files before releasing them, I expect they will be ready tomorrow. I'll try uploading to Google Docs, if they're not to large to handle. Lior On Tue, May 17, 2011 at 9:45 PM, Piotr Konieczny <piokon(a)post.pl> wrote: lior gimel wrote:

Hi, Following the discovery of the Wikipedia backups from 2001, and with much appreciated help from Jospeh Reagle, I created excel files from the diff_log and rcl_log files that together include all the edits from January 15th to August 17th (the last in the backups). the files include the time of edit and the editor's username and IP. While they are a bit buggy, and could use some more editing (or preferably, conversion to wiki format), they are simple, easy to work with (using searches and filters), and full of interesting stuff. If anyone is interested, I'd be happy to share them.

Perhaps a public distribution on Google Docs would allow interested researches to do so, wiki-style? -- Piotr Konieczny PhD Candidate Dept of Sociology Uni of Pittsburgh "To be defeated and not submit, is victory; to be victorious and rest on one's laurels, is defeat." --Józef Pilsudski _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Reply

Tim Starling

23 May 23 May

5:41 p.m.

On 17/05/11 01:46, lior gimel wrote:

...

Hi, Following the discovery of the Wikipedia backups from 2001, and with much appreciated help from Jospeh Reagle, I created excel files from the diff_log and rcl_log files that together include all the edits from January 15th to August 17th (the last in the backups). the files include the time of edit and the editor's username and IP. While they are a bit buggy, and could use some more editing (or preferably, conversion to wiki format), they are simple, easy to work with (using searches and filters), and full of interesting stuff. If anyone is interested, I'd be happy to share them.

The problem with using diff_log and rclog alone is that they don't contain a lot of changes. UseMod allowed administrators to do global search and replace operations, page renames and deletions, and these weren't represented in the logs. Back in December, I reconstructed a lot of these admin operations by hand, using other data from the UseMod backup to help. I committed my findings to MediaWiki Subversion in the form of a script, and uploaded the output as a MediaWiki-format XML dump: http://noc.wikimedia.org/~tstarling/wikipedia-2001-08-xml.7z See my mailing list post: http://article.gmane.org/gmane.science.linguistics.wikipedia.english/107675 Here is the UseMod backup, so you can repeat or extend this work if you feel like it: http://noc.wikimedia.org/~tstarling/wikipedia-2001-08-backup.7z I edited out the admin password, since despite it being a very silly and obvious password, it's not generally known and somebody might still be using it for something. There were some tar files inside the main archive, I expanded them to make it easier to edit the passwords out of them. Then I recompressed the whole thing with 7zip. -- Tim Starling

Reply