Oh, I thought bot approval was just for tasks that have an indefinite running time like cleaning vandalism, etc. I added a user-agent string to identify my IP address, but I hadn't run into problems on Wikipedia without one before. Am I supposed to re-apply each time I change a script? From: legoktm.wikipedia(a)gmail.com Date: Tue, 2 Apr 2013 00:21:44 -0500 To: wikidata-l(a)lists.wikimedia.org Subject: Re: [Wikidata-l] Running "Infobox film" import script Hi Michael, I had blocked your IP before I saw this email. We have a bot policy (https://www.wikidata.org/wiki/Wikidata:Bot) that requires coders to get approval before they can run their scripts (see https://www.wikidata.org/wiki/Wikidata:Requests_for_permissions/Bot). Your script did look pretty cool though :) -- Legoktm http://enwp.org/d:User:Legoktm On Tue, Apr 2, 2013 at 12:01 AM, Michael Hale <hale.michael.jr(a)live.com> wrote: Here's the contributions from my IP address if you want to monitor it: http://www.wikidata.org/wiki/Special:Contributions/108.235.225.145 From: hale.michael.jr(a)live.com To: wikidata-l(a)lists.wikimedia.org Date: Tue, 2 Apr 2013 00:58:05 -0400 Subject: [Wikidata-l] Running "Infobox film" import script I've been practicing using the Wikidata and MediaWiki APIs today. Even though the Wikidata API is still being developed, using it was more pleasant than parsing the templates. The good news is I'll probably be able to reuse a lot of that code for other infoboxes that still need to be imported. It's looking through 76000 articles for new statements it can add using Wikidata's currently supported film properties, excluding the IMDb ID because that isn't included in the infobox. So far it has added 691 new statements to 68 movies. I forgot to add a counter for when it finds one that already had all of the information entered. It will definitely have some errors, but I scanned the results for the first 100 movies before I started importing them, and I think the value-add will be much greater than the number of errors. _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

Bot approval is needed for any task that's going to be editing at high speeds or is (semi-)automated, which yours was. You can just request approval for "adding claims based off of Wikipedia infoboxes" and say you're going to start with films, and that will be good enough, so you won't need to request approval again. -- Legoktm http://enwp.org/d:User:Legoktm On Tue, Apr 2, 2013 at 12:31 AM, Michael Hale <hale.michael.jr(a)live.com>wrote;wrote:

Thanks for the guidance. The bot is now working properly at the preferred rate of no more than 5 edits per minute. From: hale.michael.jr(a)live.com To: wikidata-l(a)lists.wikimedia.org Date: Tue, 2 Apr 2013 02:01:47 -0400 Subject: Re: [Wikidata-l] Running "Infobox film" import script Ok, I created a request. From: legoktm.wikipedia(a)gmail.com Date: Tue, 2 Apr 2013 00:57:36 -0500 To: wikidata-l(a)lists.wikimedia.org Subject: Re: [Wikidata-l] Running "Infobox film" import script Bot approval is needed for any task that's going to be editing at high speeds or is (semi-)automated, which yours was. You can just request approval for "adding claims based off of Wikipedia infoboxes" and say you're going to start with films, and that will be good enough, so you won't need to request approval again. -- Legoktm http://enwp.org/d:User:Legoktm On Tue, Apr 2, 2013 at 12:31 AM, Michael Hale <hale.michael.jr(a)live.com> wrote: Oh, I thought bot approval was just for tasks that have an indefinite running time like cleaning vandalism, etc. I added a user-agent string to identify my IP address, but I hadn't run into problems on Wikipedia without one before. Am I supposed to re-apply each time I change a script? From: legoktm.wikipedia(a)gmail.com Date: Tue, 2 Apr 2013 00:21:44 -0500 To: wikidata-l(a)lists.wikimedia.org Subject: Re: [Wikidata-l] Running "Infobox film" import script Hi Michael, I had blocked your IP before I saw this email. We have a bot policy (https://www.wikidata.org/wiki/Wikidata:Bot) that requires coders to get approval before they can run their scripts (see https://www.wikidata.org/wiki/Wikidata:Requests_for_permissions/Bot). Your script did look pretty cool though :) -- Legoktm http://enwp.org/d:User:Legoktm On Tue, Apr 2, 2013 at 12:01 AM, Michael Hale <hale.michael.jr(a)live.com> wrote: Here's the contributions from my IP address if you want to monitor it: http://www.wikidata.org/wiki/Special:Contributions/108.235.225.145 From: hale.michael.jr(a)live.com To: wikidata-l(a)lists.wikimedia.org Date: Tue, 2 Apr 2013 00:58:05 -0400 Subject: [Wikidata-l] Running "Infobox film" import script I've been practicing using the Wikidata and MediaWiki APIs today. Even though the Wikidata API is still being developed, using it was more pleasant than parsing the templates. The good news is I'll probably be able to reuse a lot of that code for other infoboxes that still need to be imported. It's looking through 76000 articles for new statements it can add using Wikidata's currently supported film properties, excluding the IMDb ID because that isn't included in the infobox. So far it has added 691 new statements to 68 movies. I forgot to add a counter for when it finds one that already had all of the information entered. It will definitely have some errors, but I scanned the results for the first 100 movies before I started importing them, and I think the value-add will be much greater than the number of errors. _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

http://www.wikidata.org/wiki/User:Wakebrdkid%27s_bot/code

On Tue, Apr 2, 2013 at 5:25 AM, Michael Hale <hale.michael.jr(a)live.com> wrote: I got a 400 when fetching this (in Chrome). //Ed

The original URL is a 400 (looks like it's double URL encoded), but the version you quoted works fine. If you end up with %2527 in your browser bar, correct it to just %27 (apostrophes in the URL aren't such a great idea to start with though). Tom On Tue, Apr 2, 2013 at 2:44 PM, Ed Summers <ehs(a)pobox.com> wrote:

That sounds good in principle, but people might get upset ("why did we put this into Wikipedia then?") A compromise could be to import from (in this example) both Wikipedia and IMDb, add both as a reference to the same claim if they agree, and separately if not. We can then deal with the contradictions manually. On Tue, Apr 2, 2013 at 5:21 PM, Michael Hale <hale.michael.jr(a)live.com>wrote;wrote: