Recently, I had the pleasure of using the Twisted python framework for a project... so it got me thinking whether it might be beneficial to re-develop Pywikipedia on the Twisted framework. One of the advantages I see with this is that the Twisted framework is asynchronous type of framework and by using the Twisted framework, you could technically queue and defer your requests where it would fire off your bot requests a lot more efficiently. Rather than waiting and processing one page at a time for a task, Twisted could technically handle multiple ones in a limiting fashion using their system of callbacks and defers. Anyway, just a thought, I wanted to see what people thought about it.
- Jason Lee AKA [[en:w:AllyUnion]]
Jason Y. Lee ha scritto:
task, Twisted could technically handle multiple ones in a limiting fashion using their system of callbacks and defers. Anyway, just a thought, I wanted to see what people thought about it.
PyWikipediaBot has a pretty primitive network code, any replacement is welcome.
On Sat, June 13, 2009 11:01 pm, Francesco Cosoleto wrote:
PyWikipediaBot has a pretty primitive network code, any replacement is welcome.
Yes. And no. I don't think changing the network code in the main branch is useful. My network code in the rewrite branch is largely untested though, so if a Twisted wrapper could be made, that might be useful. At least, I expect Twisted code to be less buggy than mine ;)
-valhallasw
Merlijn van Deen wrote:
On Sat, June 13, 2009 11:01 pm, Francesco Cosoleto wrote:
PyWikipediaBot has a pretty primitive network code, any
replacement is
welcome.
Yes. And no. I don't think changing the network code in the main branch is useful. My network code in the rewrite branch is largely untested though, so if a Twisted wrapper could be made, that might be useful. At least, I expect Twisted code to be less buggy than mine ;)
The issue here is not whether using Twisted might be a good idea, it is: who is going to actually do the work? We have a very small group of developers in this project, all of whom are volunteers, and none of whom seem to have a lot of spare time on their hands. If the OP, Jason, wants to undertake the effort of developing a new networking layer, more power to him, but he shouldn't expect much if any support from the rest of us.
Also, as Merlijn wrote, we ought to be focusing our development efforts on the rewrite branch. The rewrite uses the MediaWiki API consistently. It is very frustrating to me personally to see other developers introducing API code into the main branch on a one-off basis without any back-end consistency, instead of contributing these efforts to finishing the rewrite branch -- which is pretty close to being ready for release.
Russ
2009/6/14 Russell Blau russblau@imapmail.org:
Also, as Merlijn wrote, we ought to be focusing our development efforts on the rewrite branch. The rewrite uses the MediaWiki API consistently. It is very frustrating to me personally to see other developers introducing API code into the main branch on a one-off basis without any back-end consistency, instead of contributing these efforts to finishing the rewrite branch -- which is pretty close to being ready for release.
Just showing my support here. Let's focus on the rewrite, folks :)
pywikipedia-l@lists.wikimedia.org