Thanks.Now, there is no sleep time between writes. It will reduce our deployment time to a large extent. I tried several hacks to remove sleep time without success. I didnt imagine such a simple fix would settle this. I believed i was completely familiar with pywikipedia framework but i guess there is still a lot to know. I noticed several threads to improve pywikipedia framework documentation.I strongly agree now. :)
Thanks, Chinmay
On Wed, Aug 21, 2013 at 12:27 AM, Merlijn van Deen valhallasw@arctus.nlwrote:
The sleep time as implemented in pywikipedia is influenced by three factors:
- the minimum time between requests, minthrottle=1 (by default) in
user-config.py
- the minimum time between page saves, put_throttle=10
- the database lag on the DB servers - pages will not be saved unless the
lag is less than maxlag = 5
(all values in seconds).
In the case of wikidata, the maximum number of saves/edits per minute is 60 (iirc), so you could reduce put_throttle to 1 by adding
put_throttle=1
in your user-config.py
Best, Merlijn
On 20 August 2013 20:16, Chinmay Naik chin.naik26@gmail.com wrote:
Hi, I am Chinmay Naik, a Gsoc intern and operator of ProteinBoxBot(runs on pywikipedia - formerly rewrite branch). When i run the bot, there exists a sleep time of around 10 s (7s average) between two successive writes. The bot will handle around 40,000 wikidata items corresponding to gene wiki articles from http://en.wikipedia.org/wiki/Category:Human_proteins. Also , i will be uploading around 30 (claims + sources) for each wikidata item.
If u take a look at the recent edits ( https://www.wikidata.org/w/index.php?title=Special:Contributions/ProteinBoxB...), it takes around 5 mins to completely write all claims+sources to each wikidata item. This is a very large time lag and will hugely affect deployment. Is there any way to reduce this sleep time? Any pointers on this would be helpful.
Thanks, Chinmay
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l