Glamtools

glamtools@lists.wikimedia.org

171 discussions

Is GWtoolset down? MWException error.
by Jesse de Vos 14 Mar '16

14 Mar '16

Hi everyone, Just trying to use the GWtoolset for an upload and I get the following error, also when trying an xml that previously worked: [70ed1b6b] 2016-03-09 15:15:49: Fatal exception of type "MWException” Any ideas of what this could mean? Thanks! Jesse -- Met vriendelijke groet, *Jesse de Vos* Researcher Interactive and New Media *T* 035 - 677 39 37 *Aanwezig:* ma t/m do <http://www.beeldengeluid.nl/> *Nederlands Instituut voor Beeld en Geluid* *Media Parkboulevard 1, 1217 WE Hilversum | Postbus 1060, 1200 BB Hilversum | * *beeldengeluid.nl* <http://www.beeldengeluid.nl/>

3 4

[Image donation] 760 historical bookbindings (National Library of the Netherlands)
by Olaf Janssen 11 Mar '16

11 Mar '16

Hi everybody, If you like nicely decorated books , this one is for you: Last month the KB used the GLAMwiki toolset to donate 760 historical bookbindings from the period 1100-1875 (public domain) to Commons : https://commons.wikimedia.org/wiki/Category:Bookbindings_from_Koninklijke_B… The bindings range from sober ones in leather and parchment to richly decorated in textile, silver or tortoise shell. They give a small insight into how historical binding techniques were used and how successive decorative styles developed over time. In the details one can recognize the requests of patrons, the craftsmanship of the binders and the preferences of collectors. The current donation is a small selection from the larger collection of some 12.000 bindings the KB takes care of: https://www.kb.nl/en/resources-research-guides/kb-collections/bookbindings-…. With kind regards Olaf Janssen Wikipedia & open data coordinator Koninklijke Bibliotheek - National Library of the Netherlands olaf.janssen(a)kb.nl<mailto:olaf.janssen@kb.nl> +31 (0)70 3140 388 @ookgezellig www.slideshare.net/OlafJanssenNL<http://www.slideshare.net/OlafJanssenNL> [Koninklijke Bibliotheek, National Library of the Netherlands] Prins Willem-Alexanderhof 5 | 2595 BE Den Haag Postbus 90407 | 2509 LK Den Haag | (070) 314 09 11 | www.kb.nl<http://www.kb.nl/> [http://www.kb.nl/sites/default/files/dots.jpg] English version<http://www.kb.nl/en/email> | Disclaimer<http://www.kb.nl/disclaimer>

1 0

[GWToolset] Uploading 350 books - via command line or API?
by Olaf Janssen 02 Mar '16

02 Mar '16

Hi, I'm preparing an image donation of some 350 picture books from 1810 to 1880 (taken from the collection http://www.geheugenvannederland.nl/?/en/collecties/prentenboeken_van_1810_t…) For every book I've constructed an XML file describing the pages (metadata). So eg. for a book of 20 pages I've an XML with 20 records. I can upload these in the normal way via the GWToolset webinterface, also assigning a Commons category to the book. For 1 book that's doable, but for 350 books I would need to upload 350 XML files, 1 by 1, using the GWT-webinterface (using the same json mapping file for all uploads). But this would take me a lot of time (and it's rather boring)... So I'm wondering if / how I could automate this. Is there a more direct/efficient way? I can image that I could do some command line interfacing (Pywiki??), with the XML, the json-mapping and the target Commonscat-name as input parameters. Would that be an option? Any tricks, tips & directions are very welcome Met vriendelijke groet / With kind regards Olaf Janssen Wikipedia & open data coordinator Koninklijke Bibliotheek - National Library of the Netherlands olaf.janssen(a)kb.nl<mailto:olaf.janssen@kb.nl> +31 (0)70 3140 388 @ookgezellig www.slideshare.net/OlafJanssenNL<http://www.slideshare.net/OlafJanssenNL> [Koninklijke Bibliotheek, National Library of the Netherlands] Prins Willem-Alexanderhof 5 | 2595 BE Den Haag Postbus 90407 | 2509 LK Den Haag | (070) 314 09 11 | www.kb.nl<http://www.kb.nl/> [http://www.kb.nl/sites/default/files/dots.jpg] English version<http://www.kb.nl/en/email> | Disclaimer<http://www.kb.nl/disclaimer>

5 6

Re: [Glamtools] Glamtools Digest, Vol 35, Issue 5
by Olaf Janssen 16 Feb '16

16 Feb '16

Hi Fae, Thanks for this insight, this is very helpful! I was not yet aware enough of that option in the regular GTW-interface . I'll modify my code to create 1 large XML-dump.... Best, Olaf -----Original Message----- From: Glamtools [mailto:glamtools-bounces@lists.wikimedia.org] On Behalf Of glamtools-request(a)lists.wikimedia.org Sent: dinsdag 16 februari 2016 13:00 To: glamtools(a)lists.wikimedia.org Subject: Glamtools Digest, Vol 35, Issue 5 Send Glamtools mailing list submissions to glamtools(a)lists.wikimedia.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/glamtools or, via email, send a message with subject or body 'help' to glamtools-request(a)lists.wikimedia.org You can reach the person managing the list at glamtools-owner(a)lists.wikimedia.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Glamtools digest..." Today's Topics: 1. Re: [GWToolset] Uploading 350 books - via command line or API? (Fæ) ---------------------------------------------------------------------- Message: 1 Date: Mon, 15 Feb 2016 12:53:18 +0000 From: Fæ <faewik(a)gmail.com> To: Conversations revolving around the development of GLAM Digital Tools <glamtools(a)lists.wikimedia.org> Subject: Re: [Glamtools] [GWToolset] Uploading 350 books - via command line or API? Message-ID: <CAH7nnD2MrU0GRA93j5oJCGfSfmn3288zJNeDQiYeqDFO1vrZNQ(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 The easiest way is to merge your XML files into one large XML, then put this through the GWT. Though I can imagine ways of automating the front end of GWT, it would be a clumsy way of going about it. If your concern is that you want to create separate book categories, then add a category field in the XML that can vary by book. You can add several variable categories as an option on the mappings page. For example <https://commons.wikimedia.org/wiki/File:DAILY_MENU_%28held_by%29_REVERE_HOU…> was uploaded with the category "NYPL Rare Book Division" automatically generated from the NYPL metadata. (To be fair, I'm not using the GWT for most of the NYPL material for reasons mentioned on the related project page.) Responding to J Hayes comment in this thread, you can mass upload to IA with off-the shelf Python modules. However just as much care should be taken to map out the metadata using IA's metadata options, as they are incredibly open/flexible, the archives tend to be confusingly inconsistent. This would still leave a challenge of finding a good mapping for Commons templates if you then wanted to upload from IA to Commons rather than from somewhere else. Fae On 15 February 2016 at 12:18, Olaf Janssen <Olaf.Janssen(a)kb.nl> wrote: > > Hi, > > I’m preparing an image donation of some 350 picture books from 1810 to > 1880 (taken from the collection > http://www.geheugenvannederland.nl/?/en/collecties/prentenboeken_van_1 > 810_tot_1950) > > For every book I’ve constructed an XML file describing the pages (metadata). So eg. for a book of 20 pages I’ve an XML with 20 records. I can upload these in the normal way via the GWToolset webinterface, also assigning a Commons category to the book. > > > > For 1 book that’s doable, but for 350 books I would need to upload 350 > XML files, 1 by 1, using the GWT-webinterface (using the same json > mapping file for all uploads). But this would take me a lot of time > (and it’s rather boring)… > > > > So I’m wondering if / how I could automate this. Is there a more direct/efficient way? > > > > I can image that I could do some command line interfacing (Pywiki??), with the XML, the json-mapping and the target Commonscat-name as input parameters. Would that be an option? > > > > Any tricks, tips & directions are very welcome > > > > > > Met vriendelijke groet / With kind regards > > > > Olaf Janssen > > > > Wikipedia & open data coordinator > > > > Koninklijke Bibliotheek - National Library of the Netherlands > olaf.janssen(a)kb.nl > > +31 (0)70 3140 388 > @ookgezellig > > www.slideshare.net/OlafJanssenNL -- faewik(a)gmail.com https://commons.wikimedia.org/wiki/User:Fae ------------------------------ Subject: Digest Footer _______________________________________________ Glamtools mailing list Glamtools(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/glamtools ------------------------------ End of Glamtools Digest, Vol 35, Issue 5 ****************************************

1 0

Beta GWToolset user rights
by Arne Wossink 09 Feb '16

09 Feb '16

Hi all, I would like to ask for Beta GWToolset user rights to test a bulk upload of images from the Catharijneconvent in the Netherlands. My username on Beta is AWossink Regards, Arne Wossink Projectleider / Project Lead Wikimedia Nederland *(Werkdagen: maandag, dinsdag, donderdag / Office hours: Monday, Tuesday, Thursday)* Tel. +31 (0)6 11000505 E-mail: wossink(a)wikimedia.nl *Postadres*: * Bezoekadres:* Postbus 167 Mariaplaats 3 3500 AD Utrecht Utrecht

2 1

GW Toolset problems
by Martin Poulter 05 Feb '16

05 Feb '16

I can supply a control condition. The Bodleian Library URLs, which work without a problem, have the similar form: http://iiif.bodleian.ox.ac.uk/iiif/image/e998e3d6-17e2-40ca-bf23-bc4278feb1… Note that there is a comma but no dot before the filename. This suggests that commas are not the problem. -----Original Message----- ------------------------------ Message: 2 Date: Thu, 4 Feb 2016 18:14:26 +0100 From: Sébastien Santoro <dereckson(a)espace-win.org> To: Conversations revolving around the development of GLAM Digital Tools <glamtools(a)lists.wikimedia.org> Subject: Re: [Glamtools] GW Toolset problems Message-ID: <CAKg6iAEi1dFqSZDXAhRAHFeybTjCKnDb5fqsVX34dLQ=pb4GEA(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Not only the site is whitelisted, but I've manually tested upload by URL using [[Special:Upload]] ant it works. The URL contains two non alphanumeric elements in path : "2.0" and "1000," so I'd guess it's where to find an explanation Hypothesis 1: GWT considers the extension is ".0/image/1293358/full/1000,/0/default.jpg", which is not ".jpg" Hypothesis 2: GWT stop to parse the URL before the comma and want to handle http://dams.llgc.org.uk/iiif/test/2.0/image/1293358/full/1000, where there isn't any extension and is 404. The dot in URL is rather frequent, the comma less, so the second hypothesis is more plausible. On Thu, Feb 4, 2016 at 6:08 PM, Jason J. Evans <jason.evans(a)llgc.org.uk> wrote: > Hello everyone, I am trying to do a batch upload using the GW Toolset but the tool will not recognize the file extension from the Image URL's provided. Here is an example of a url from the XML file: > > http://dams.llgc.org.uk/iiif/test/2.0/image/1293358/full/1000,/0/default.jpg > > Any ideas as to why this doesn't work? (site is white-listed ect) > > Thanks > > Jason > -- > Jason Evans > Wicipediwr Preswyl / Wikipedian in Residence > Llyfrgell Genedlaethol Cymru / National Library of Wales > jason.evans(a)llgc.org.uk > Ffon/Tel: +44 (0)1970 632405 > > _______________________________________________ > Glamtools mailing list > Glamtools(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/glamtools -- Sébastien Santoro aka Dereckson http://www.dereckson.be/

2 1

GW Toolset problems
by Jason J. Evans 05 Feb '16

05 Feb '16

Hello everyone, I am trying to do a batch upload using the GW Toolset but the tool will not recognize the file extension from the Image URL's provided. Here is an example of a url from the XML file: http://dams.llgc.org.uk/iiif/test/2.0/image/1293358/full/1000,/0/default.jpg Any ideas as to why this doesn't work? (site is white-listed ect) Thanks Jason -- Jason Evans Wicipediwr Preswyl / Wikipedian in Residence Llyfrgell Genedlaethol Cymru / National Library of Wales jason.evans(a)llgc.org.uk Ffon/Tel: +44 (0)1970 632405

5 6

Newbie feedback on GWToolset interface
by Martin Poulter 29 Jan '16

29 Jan '16

Having used the interface for the first time today, I hope it's okay if I give some feedback about the tool and its documentation. Firstly, I'm extremely glad this exists and that it is set out so clearly. I find it incredibly helpful and my only suggestions are relatively minor. 1) Accession number: My metadata has a couple of identifier fields: a shelfmark which identifies the object in the GLAM's catalogue, and a file identifier which identifies the digital image in the GLAM's database. The shelfmark is, from the GLAM's perspective, the most important single piece of information. It seemed sensible to use the Artwork template and put these identifiers into the Accession number field. However, when I tried, the field just didn't display (in the GWToolset's preview of the first three records). Maybe there was a problem with combining two source fields into that one field? Or maybe the Accession number field is expecting a certain format? When I instead assigned the identifiers to the Notes field, they displayed fine, and having them visible is the important thing, so it's not a huge problem. 2) Global categories: I chose a category which includes an apostrophe character: Files uploaded by the Bodleian Libraries' Wikimedian In Residence The system renders this as: Files uploaded by the Bodleian Libraries& This is easy to fix post-upload with cat-a-lot or a similar tool, but it would be nice to fix it in code. 3) Entering a template in a field: There are different ways a template can be specified: by its name (e.g. cc-by-sa-4.0 ), by wiki markup ( {{cc-by-sa-4.0}} ), by its name with namespace ( Template:Cc-by-sa-4.0 ) or by a web URL ( https://commons.wikimedia.org/wiki/Template:Cc-by-sa-4.0 ). When I put a template in the form, it's not always clear which of these formats to use, and in fact the answer is different for different fields. With a bit of experimentation it becomes clear, so it's not a severe problem, but it would be easier if the form was more clear about which format to enter. For the License template field, the form gives an example of {{Template:Cc-by-sa-3.0<http://commons.wikimedia.beta.wmflabs.org/w/index.php?title=Template:Templa…>}}, but the Template: namespace is unnecessary. 4) Partner templates: For the upload I'm doing now, I'd like to have *two* source templates. It would be welcome if this were a text box taking multiple lines of markup, rather than a text field. Again, this is no great problem to correct post-upload. Maybe more than one source template can be put into that box? If so, the form should say so. 5) Item specific categories: This is merely something that would be *nice* to have in the long run: having the phrase *after* the value of the metadata field. My metadata has lots of keywords, such as "Crowds", "Fish" or "Singing". It would be nice to auto-categorise these into Category:Crowds in art, Fish in art and so on. Commons has many hundreds of such categories. So those are my suggestions. To emphasise, my impression of the upload tool is very positive.

3 2

Re: [Glamtools] Internal error on Commons Beta (Sébastien Santoro)
by Martin Poulter 28 Jan '16

28 Jan '16

The error is gone and a test upload is in progress. Thanks, Sébastien. ---------------------------------------------------------------------- Message: 1 Date: Wed, 27 Jan 2016 17:12:29 +0100 From: Sébastien Santoro <dereckson(a)espace-win.org> To: Conversations revolving around the development of GLAM Digital Tools <glamtools(a)lists.wikimedia.org> Subject: Re: [Glamtools] Internal error on Commons Beta Message-ID: <CAKg6iAFo+j18TGM9ur2cTvaS4cKuT2JWw_OtN2-LSLh7CGBuRA(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Martin, could you test it again and confirm us it now works? On Tue, Jan 26, 2016 at 3:56 PM, Sébastien Santoro <dereckson(a)espace-win.org> wrote: > Good afternoon, > > We've currently an issue with the Redis server on our Beta cluster, > from which Commons Beta is a part of. > > You can track the issue at https://phabricator.wikimedia.org/T124677. > > Uploads to Wikimedia Commons in production are not affected by this > issue and should work normally. > > On Tue, Jan 26, 2016 at 12:20 PM, Martin Poulter > <martin.poulter(a)bodleian.ox.ac.uk> wrote: >> Hi, I'm going through a test upload on Beta, but having previewed the three test records and pressed "process batch", I get: >> >> Internal error: >> [f79958ce] /wiki/Special:GWToolset JobQueueConnectionError from line >> 753 of /srv/mediawiki/php- >> master/includes/jobqueue/JobQueueRedis.php: Unable to connect to redis server deployment-redis01.eqiad.wmflabs. >> >> Help! >> _______________________________________________ >> Glamtools mailing list >> Glamtools(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/glamtools > > > > -- > Sébastien Santoro aka Dereckson > http://www.dereckson.be/ -- Sébastien Santoro aka Dereckson http://www.dereckson.be/ ------------------------------ Subject: Digest Footer _______________________________________________ Glamtools mailing list Glamtools(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/glamtools ------------------------------ End of Glamtools Digest, Vol 34, Issue 12 *****************************************

1 0

Internal error on Commons Beta
by Martin Poulter 27 Jan '16

27 Jan '16

Hi, I'm going through a test upload on Beta, but having previewed the three test records and pressed "process batch", I get: Internal error: [f79958ce] /wiki/Special:GWToolset JobQueueConnectionError from line 753 of /srv/mediawiki/php- master/includes/jobqueue/JobQueueRedis.php: Unable to connect to redis server deployment-redis01.eqiad.wmflabs. Help!

2 2

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Glamtools