On Sun, Dec 16, 2012 at 2:42 AM, Yuri Astrakhan <span dir="ltr">&lt;<a href="mailto:yuriastrakhan@gmail.com" target="_blank">yuriastrakhan@gmail.com</a>&gt;</span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

After a very productive talk with legoktm @IRC, it seems some of these changes have already been made as part of the rewrite at <a href="http://svn.wikimedia.org/viewvc/pywikipedia/branches/rewrite/" target="_blank">http://svn.wikimedia.org/viewvc/pywikipedia/branches/rewrite/</a> documented at <a href="http://botwiki.sno.cc/wiki/Rewrite" target="_blank">http://botwiki.sno.cc/wiki/Rewrite</a> , and only a few things remain:<div>



<br></div><div>* GIT migration</div><div>    Mediawiki will not support SVN forever. I wouldn&#39;t want to copy and break away completely - history is very important for maintenance (Yes, i saw the previous GIT posts here)</div>



<div><br></div></blockquote><div>I don&#39;t like gerrit, however I think we should move sooner and when we&#39;re ready rather than being forced to move. I&#39;m not a big fan of having to submit patches/issues on sourceforge either...<br>

 </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div></div><div>* Python3 migration<br></div><div>    Guido started it for a very good reason. Much better Unicode, cleaner libraries.</div>

</blockquote><div><br>Yes, yes, and yes.  This would fix nearly every single unicode problem possible.<br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>

<br></div><div>* Directory structure</div><div>   I propose two separate root dirs:</div>

<div>       pywikibot\ - current code<br></div><div>       pywikibot3\ - Python3 rewrite</div><div>   The directories should be at the same level, and not as branch, because we will never be able to merge them back together - anyone who wants to continue using Python26+ and older features will keep using pywikibot, and all the new code plus migrated scripts will move to pywikibot3</div>



<div><br></div></blockquote><div><br>Well I think we should keep the current wikipedia.py trunk version alive, and somewhat supported (as long as someone is willing to write patches?)<br>The current &quot;rewrite&quot; branch should be used as the basis of the pywikibot3 version.<br>

 </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div></div><div>* API only</div><div>    No need to maintain complex and cumbersome Special:Export and page  scraping methods. If mediawiki installation doesn&#39;t support API - they don&#39;t want bot access. If their installation is over 5 years old, they can use older pywiki framework version.</div>



<div>    There is still xml dump support in pywiki, but many functions it seems will not work properly, so maybe we should remove that as well.<br></div></blockquote><div><br>I think its important for pywiki to still be able to read from a XML dump. There are certain tasks that are much better done from a dump than fetching live wikitext.<br>

There are also a few things that still don&#39;t have API modules and we should still make it possible to screenscrape those, however anything with an API module should be using the API.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

<div><div><div></div></div></div><div><div>* Higher-level HTTP library</div>

</div><div>    Not as sure about this yet, but it seems python Requests is a better library for the API calls than the httplib2 - for complexity and cleanliness reasons. See <a href="http://docs.python-requests.org/" target="_blank">http://docs.python-requests.org/</a></div>

</blockquote><div> </div><div>Requests is the library I recommended to Yurik on IRC. It supports just about everything  you can think of, so I don&#39;t know why we&#39;re trying to recreate the wheel by using our own library. That said, I can see an argument for reducing the number of dependencies pywiki uses, but I feel the benefits that would come by using a library like requests outweigh the negatives.<br>

<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

<div> </div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div></div><div>When can we start? :)</div></blockquote><div> </div></div>As soon as possible! I&#39;m going to busy for the next week or so due to IRL commitments, but I&#39;m all for this as soon as I&#39;m done.<br>

Thanks for getting this started Yurik!<br><br clear="all"><div>-- Legoktm<br><a href="http://enwp.org/User:Legoktm" target="_blank">http://enwp.org/User:Legoktm</a><br></div><br>