https://bugzilla.wikimedia.org/show_bug.cgi?id=55889 --- Comment #2 from Strainu <crangasi2001(a)yahoo.com> --- As I understand it, I/O happens outside of GIL. As the API requests are the most time-consuming part of many of my robots (and more precisely the connection to the servers), being able to do requests from several threads should somewhat improve performance (as long as the throttling is not too aggressive). I've noticed that the preloading limit is not only 50 pages, making this problem even more stringent for many small pages. It's probably also a good idea for things like image upload/download. If it helps, we can do some tests to see if performance is increased for a simple file downloader? -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.wikimedia.org/show_bug.cgi?id=55889 --- Comment #4 from Strainu <crangasi2001(a)yahoo.com> --- (In reply to comment #3) I've noticed that while writing the answer to Gerard's questions today :) I've experimented with put_async with mixed results. When the upload works, it's mostly OK, however when one request hits an error (like a 504 from the server) it just keeps trying again and again, keeping the thread blocked. Instead, the request should probably be de-queued, processed and, if a callback has been registered, the callback should be called in order to allow the bot to re-queue the request. This, however, could cause trouble if the order of the requests is important. The bot can receive a callback, but AFAIK it cannot remove already queued requests. Also, what happens if no callback has been registered? Should we simply re-queue the request? I don't have a perfect solution at this time, but this is a point that should be considered. Another possible issue, that PWB can't really do much about, is that one can get a 504 even if the save is successful, making the re-queueing useless. I don't have a good solution for that either, but we could consult with the Wikimedia developers. This should be especially useful if it can be controlled by the user. Do you have any ideas on how to do this? I think there were some good ideas brought up on this bug. Should we start a thread on the mailing list so we can gather more input on this? -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.wikimedia.org/show_bug.cgi?id=55889 Ricordisamoa <ricordisamoa(a)live.it> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |55220 -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.wikimedia.org/show_bug.cgi?id=55889 Gerrit Notification Bot <gerritadmin(a)wikimedia.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |PATCH_TO_REVIEW -- You are receiving this mail because: You are the assignee for the bug.