Hello everyone,
I set my bot (Mys_bot) to run weblinkchecker.py on toolserver. It needs to check all pages on zh_wikipedia for the first time through api.
I made it run for a hour earlier. However, one hour is not enough. Currently I submit a 3 hour job to the queue per day manually.
Is it okay to let it run until it completes the dat file and schedule repeat checks on weekly basis? Also, how long should I set per run for weblinkchecker.py?
Regards, Yishen Miao (mys_721tx)
About how long do you think it would take to complete the dat file? I don't think its good to have a process like that running too long, so it would depend.
________________________________ From: Yishen Miao mys721tx@gmail.com To: toolserver-l@lists.wikimedia.org Sent: Friday, June 22, 2012 4:38 AM Subject: [Toolserver-l] Running weblinkchecker.py on Toolserver
Hello everyone,
I set my bot (Mys_bot) to run weblinkchecker.py on toolserver. It needs to check all pages on zh_wikipedia for the first time through api.
I made it run for a hour earlier. However, one hour is not enough. Currently I submit a 3 hour job to the queue per day manually.
Is it okay to let it run until it completes the dat file and schedule repeat checks on weekly basis? Also, how long should I set per run for weblinkchecker.py?
Regards, Yishen Miao (mys_721tx) _______________________________________________ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
On 22/06/12 11:38, Yishen Miao wrote:
Hello everyone,
I set my bot (Mys_bot) to run weblinkchecker.py on toolserver. It needs to check all pages on zh_wikipedia for the first time through api.
This task would be done much more efficiently if instead of grabbing all the wikipedia links from the api, it used the toolserver copy (externallinks table). Nonetheless, checking lots of foreign servers will still be very slow.
toolserver-l@lists.wikimedia.org