<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">2014-05-13 15:06 GMT+02:00 Aaron Halfaker <span dir="ltr"><<a href="mailto:aaron.halfaker@gmail.com" target="_blank">aaron.halfaker@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Emilio,<div><br></div><div>I'm very interested in making your XML dump processing work easier. If you file any bugs against the old[1] or new[2] libraries, I'll be quick to turn around on them. </div>
<div><br></div><div>1. <a href="https://bitbucket.org/halfak/wikimedia-utilities" target="_blank">https://bitbucket.org/halfak/wikimedia-utilities</a></div><div>2. <a href="https://github.com/halfak/mediawiki-utilities" target="_blank">https://github.com/halfak/mediawiki-utilities</a></div>
</div></blockquote><div><br></div><div>Thanks Aaron, I'm going to use the new version. I hope I can help to your project, reporting bugs, sending some patch, scripts for the example directory, or anyway. I like processing XML dumps and your library is very useful. Fav'ed on Github.<br>
<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span class="HOEnZb"><font color="#888888">
<div><br></div><div>-Aaron </div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, May 12, 2014 at 10:30 AM, Morten Wang <span dir="ltr"><<a href="mailto:nettrom@gmail.com" target="_blank">nettrom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Emilio,<div><br></div><div>You're probably aware of it, but one way to handle your own installs is to use virtual environments: <a href="https://virtualenv.pypa.io/en/latest/" target="_blank">https://virtualenv.pypa.io/en/latest/</a></div>
<div><br></div><div>BTW, the Python utilities you pointed to is now deprecated in favour of a newer version, but the newer version is Python 3.x only: <a href="http://pythonhosted.org/mediawiki-utilities/" target="_blank">http://pythonhosted.org/mediawiki-utilities/</a></div>
<div><br></div><div>I have the older version of his utilities installed in my virtual environment. When I processed the English dump about a month ago I used tools-dev for testing and then submitted jobs to the job servers when it was ready, running over the smaller split files of the dump for parallelisation and less memory usage.</div>
<div><br></div><div>From what I've heard the newer library is considerably faster than the 2.x version, but I haven't yet had a project where I could test that.</div><div><br></div></div></blockquote></div></div>
</div></div></blockquote><div><br></div><div>Thanks Morten for the virtualenv tip. I'm using it now.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Regards,</div>
<div>Morten</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On 11 May 2014 13:10, Emilio J. Rodríguez-Posada <span dir="ltr"><<a href="mailto:emijrp@gmail.com" target="_blank">emijrp@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div>Hi;<br><br></div>I would like to process some Wikipedia dumps. The right place for this is tools-dev? I don't see Wikimedia Utilities[1] available there.<br>
<br>Do I have to install it or this is a task for an admin?<br>
<br>Regards<br><br>[1] <a href="https://bitbucket.org/halfak/wikimedia-utilities/wiki/Home" target="_blank">https://bitbucket.org/halfak/wikimedia-utilities/wiki/Home</a><br></div>
<br></div></div>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org" target="_blank">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org" target="_blank">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div></div>