Thank you Jean-Fred and Maarten \o/

2015-08-29 1:29 GMT+02:00 Lodewijk lodewijk@effeietsanders.org:

+100 :)

On Sat, Aug 29, 2015 at 1:27 AM, Beppo.S@gmx.at wrote:

...

Hi, Jean-Fred, I hope everybody appreciates your valuable work as much as we do in the WLM team of Austria. When we talked at the Wikimania on the international level about connecting the different countries participating in WLM, somebody said, that it was much more important to have the monuments database and related software running again, than to have international prizes. Not everybody is aware, that we need the database to maintain and improve the lists of monuments all over the world, to show everybody where pictures are lacking and what the progress behind this photo competition is.

We like to thank you and Marten Dammers, who was the founder of that database, and all the others maintaining it and contributing to it. Kind regards from Beppo member of the Austrian WLM team

*Gesendet:* Freitag, 28. August 2015 um 12:23 Uhr *Von:* "Romaine Wiki" romaine.wiki@gmail.com *An:* "Wiki Loves Monuments Photograph Competition" < wikilovesmonuments@lists.wikimedia.org> *Betreff:* Re: [Wiki Loves Monuments] ErfgoedBot status Hello Jean-Fred,

Thank you for taking up this task!

Romaine

2015-08-28 12:14 GMT+02:00 Jean-Frédéric jeanfrederic.wiki@gmail.com:

...

Hi all,

ErfgoedBot is running again and the Monuments database was updated.

There are still issues going on that are being sorted out − in particular, several countries have not been harvested properly (see [T110420] for list).

Please report (best by filing a task on Phabricator, or by emailing here or me directly) any problem you notice with the Monuments database.

Cheers,

[T110420] https://phabricator.wikimedia.org/T110420

Jean-Fred

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

2015-08-30 13:08 GMT+02:00 Maarten Dammers maarten@mdammers.nl:

Hi,

Pierre-Selim schreef op 29-8-2015 om 13:27:

...

Thank you Jean-Fred and Maarten \o/

Thanks Pierre-Selim and other. Seems to be becoming a bit of a tradition to have a sprint last week of August to get stuff fixed again. I'm quite happy Jean-Fred picked up the role of maintainer.

The current status is:

• Update database runs, but some sources (countries) need to be updated

and fixed ( https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/S... )

Can anyone check why es-es is missing in that table?

• Categorization bot seems to work again. I see a lot of files being

categorized a couple of days ago ( https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/C... )

• Unused images bot is still broken, but I found the source of the problem

and looks easy to fix (https://phabricator.wikimedia.org/T110829)

I was fixing stuff but toollabs just died. I hope it will be back online later this afternoon so I can get some of the bugs fixed.

Every year the categorization bot gets overloaded. Right now it has about 30.000 images it can't categorize. That means that every day it runs it goes through the same 30.000 images and can't to anything. That's quite a waste of resources and time. If you look at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization... you'll see the top "offenders":

• About 17.000 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_I...

• About 3.700 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_J...

• About 2.000 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_C...

I know quite a few people from Italy are on this list. Italian folks, could you please spend some time on emptying out this category? With the current backlog me or Jean-Fred are probably just going to disable the bot for Italy because it slows down everything. Italy is also high on https://commons.wikimedia.org/wiki/Commons:Monuments_database/Unused_images/... so I wonder if everything is set up correctly for Italy. Looks like a bit of a mess to me. Why do we even have https://it.wikipedia.org/wiki/Template:WLM2013-riga ?

Maarten

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

Both approaches are posible: to subcategorize them manually or to help the bot to do it botically. The last one is more efficient en the long term.

How-to: 1. Add to the file the monument template with the identifier. If it does not correspond to any monument listed, remove the file from the main categoy of monuments. 2. Add the template {{Commonscat}} to the list where the monument is listed. Usually it is the category for monuments of a city or region. 3. If the monument has its own category, add the parameter commonscat=, or similar, in the row template for the monument. If your row template does not have such parameter, consider to add it. It can be usuful for a "see more pics" link or, at least, it is useful for a precise categorization by ErfgoedBot.

More info at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization

BTW, I am not sure how it works if the template commonscat has its parameter migrated to Wikidata.

Vicenç

From: lodewijk@effeietsanders.org Date: Sun, 30 Aug 2015 13:29:50 +0200 To: wikilovesmonuments@lists.wikimedia.org Subject: Re: [Wiki Loves Monuments] ErfgoedBot status

Hi Maarten, Thanks for the update. Just to make sure people understand what exactly needs to happen to these categories: as I understand it, they should be bumped down at least one level to a region? Or could people help in a more structured way, by connecting the identifiers with the right category, somehow? What is the most intelligent approach in the current setup? Best,Lodewijk On Sun, Aug 30, 2015 at 1:08 PM, Maarten Dammers maarten@mdammers.nl wrote: Hi,

Pierre-Selim schreef op 29-8-2015 om 13:27:

Thank you Jean-Fred and Maarten \o/

Thanks Pierre-Selim and other. Seems to be becoming a bit of a tradition to have a sprint last week of August to get stuff fixed again. I'm quite happy Jean-Fred picked up the role of maintainer.

The current status is:

* Update database runs, but some sources (countries) need to be updated and fixed (https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/S...)

* Categorization bot seems to work again. I see a lot of files being categorized a couple of days ago (https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/C...)

* Unused images bot is still broken, but I found the source of the problem and looks easy to fix (https://phabricator.wikimedia.org/T110829)

I was fixing stuff but toollabs just died. I hope it will be back online later this afternoon so I can get some of the bugs fixed.

Every year the categorization bot gets overloaded. Right now it has about 30.000 images it can't categorize. That means that every day it runs it goes through the same 30.000 images and can't to anything. That's quite a waste of resources and time. If you look at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization... you'll see the top "offenders":

* About 17.000 images in https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_I...

* About 3.700 images in https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_J...

* About 2.000 images in https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_C...

I know quite a few people from Italy are on this list. Italian folks, could you please spend some time on emptying out this category? With the current backlog me or Jean-Fred are probably just going to disable the bot for Italy because it slows down everything. Italy is also high on https://commons.wikimedia.org/wiki/Commons:Monuments_database/Unused_images/... so I wonder if everything is set up correctly for Italy. Looks like a bit of a mess to me. Why do we even have https://it.wikipedia.org/wiki/Template:WLM2013-riga ?

Maarten

_______________________________________________

Wiki Loves Monuments mailing list

WikiLovesMonuments@lists.wikimedia.org

https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments

http://www.wikilovesmonuments.org

_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

Might be part of the reason that the italian template uses "| comcat=" instead of "|commonscat=" in their template? For example, I came across this file https://commons.wikimedia.org/wiki/File:Galata_in_atto_di_cadere,_copia_romano_imperiale_di_un_originale_di_Pergamo,_datato_tra_la_fine_del_III_e_la_met%C3%A0_del_II_sec._a.C.,_restauri_cinquecenteschi_di_Tiziano_Aspetti,_da_Roma,_(1).JPG in the category, which is on this list https://it.wikipedia.org/wiki/Progetto:Wiki_Loves_Monuments_2015/Monumenti/Veneto and has already a commonscat. Is my understanding correct that normally, it should not be in the category https://commons.wikimedia.org/w/index.php?title=Category:Cultural_heritage_monuments_in_Italy&filefrom=gany more?

Lodewijk

On Mon, Aug 31, 2015 at 5:42 PM, Beppo.S@gmx.at wrote:

Hello! Some experienced volunteers in Austria would like to help with the categories for Italy. Maybe they should have a partner in Italy, to not provide needless categorizations. Regards Beppo

*Gesendet:* Montag, 31. August 2015 um 16:59 Uhr *Von:* Vicenç vriullop@hotmail.com *An:* "wikilovesmonuments@lists.wikimedia.org" < wikilovesmonuments@lists.wikimedia.org> *Betreff:* Re: [Wiki Loves Monuments] ErfgoedBot status Both approaches are posible: to subcategorize them manually or to help the bot to do it botically. The last one is more efficient en the long term.

How-to:

1. Add to the file the monument template with the identifier. If it does

not correspond to any monument listed, remove the file from the main categoy of monuments. 2. Add the template {{Commonscat}} to the list where the monument is listed. Usually it is the category for monuments of a city or region. 3. If the monument has its own category, add the parameter commonscat=, or similar, in the row template for the monument. If your row template does not have such parameter, consider to add it. It can be usuful for a "see more pics" link or, at least, it is useful for a precise categorization by ErfgoedBot.

More info at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization

BTW, I am not sure how it works if the template commonscat has its parameter migrated to Wikidata.

Vicenç

---

From: lodewijk@effeietsanders.org Date: Sun, 30 Aug 2015 13:29:50 +0200 To: wikilovesmonuments@lists.wikimedia.org Subject: Re: [Wiki Loves Monuments] ErfgoedBot status

Hi Maarten,

Thanks for the update. Just to make sure people understand what exactly needs to happen to these categories: as I understand it, they should be bumped down at least one level to a region? Or could people help in a more structured way, by connecting the identifiers with the right category, somehow? What is the most intelligent approach in the current setup?

Best, Lodewijk

On Sun, Aug 30, 2015 at 1:08 PM, Maarten Dammers maarten@mdammers.nl wrote:

Hi,

Pierre-Selim schreef op 29-8-2015 om 13:27:

Thank you Jean-Fred and Maarten \o/

Thanks Pierre-Selim and other. Seems to be becoming a bit of a tradition to have a sprint last week of August to get stuff fixed again. I'm quite happy Jean-Fred picked up the role of maintainer.

The current status is:

• Update database runs, but some sources (countries) need to be updated

and fixed ( https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/S... )

• Categorization bot seems to work again. I see a lot of files being

categorized a couple of days ago ( https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/C... )

• Unused images bot is still broken, but I found the source of the problem

and looks easy to fix (https://phabricator.wikimedia.org/T110829)

I was fixing stuff but toollabs just died. I hope it will be back online later this afternoon so I can get some of the bugs fixed.

Every year the categorization bot gets overloaded. Right now it has about 30.000 images it can't categorize. That means that every day it runs it goes through the same 30.000 images and can't to anything. That's quite a waste of resources and time. If you look at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization... you'll see the top "offenders":

• About 17.000 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_I...

• About 3.700 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_J...

• About 2.000 images in

https://commons.wikimedia.org/wiki/Category:Cultural_heritage_monuments_in_C...

I know quite a few people from Italy are on this list. Italian folks, could you please spend some time on emptying out this category? With the current backlog me or Jean-Fred are probably just going to disable the bot for Italy because it slows down everything. Italy is also high on https://commons.wikimedia.org/wiki/Commons:Monuments_database/Unused_images/... so I wonder if everything is set up correctly for Italy. Looks like a bit of a mess to me. Why do we even have https://it.wikipedia.org/wiki/Template:WLM2013-riga ?

Maarten

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org _______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

---

Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org

Hi Vicenç,

BTW, I am not sure how it works if the template commonscat has its

parameter migrated to Wikidata.

This functionality was added a week agoo − see [T110003]. If the bot finds a {{Commonscat}} (or equivalent template) without parameter, it will try to find it in Wikidata (by going to the Wikidata item linked to the list and checking P373).

[T110003] https://phabricator.wikimedia.org/T110003

Hi Miguel-Ángel ,

The situation of the monument lists in the Spanish Wikipedia is as follows:

Thank you for this summary! This is really helpful :)

• Lists are made of {{tl|Fila BIC}} templates. There is one 'Fila LIC'

per monument.

• If available, the monument code is included by means of the 'bic'

parameter. The value for the key 'bic' is the monument code. This code is usually the argument of the {{tl|BIC}} template in commons.

All good.

• If there is a commons category for the monument, it's in the

'categoría-Commons' parameter.

Ah ah! The bot expected a 'commonscat' parameter. I changed that in <

https://phabricator.wikimedia.org/T111288%3E

• If such a category does not exist yet (no or very few pictures yet,

nobody is aware of the availability of pictures...) auxiliary categories have been defined, with the following names: categoría-Commons-auxiliar, categoría-Commons-auxiliar1, categoría-Commons-auxiliar2 and categoría-Commons-defecto.

Unfortunately the bot does not support such logic. Sorry :-/

However, at the moment, we're in the process of filling the category parameters in the monument lists. Most of them may be empty yet.

That’s good!

Hope this helps. Feel free to come back with any question or comment.

This report page is quite useful for you: < https://commons.wikimedia.org/wiki/Commons:Monuments_database/Unknown_fields...)

It shows the fields found in the lists not recognised by the bot. Some of them (the ones with small counts) are typos that you can fix (I also have more complete logs that include *where* the typo was found, which helps ;) ; others (rather the ones with big counts) may rather indicate a misconfiguration of the bot. That’s the case with 'categoría-Commons': the bot does not recognise it although it should. That one should be fixed at the next bot run.

Cheers,

Hi all,

Lodewijk:

as I understand it, they should be bumped down at least one level to a region? Or could people help in a more structured way, by connecting the identifiers with the right category, somehow? What is the most intelligent approach in the current setup?

It depends why the bot fails to categorize the picture:

* if the file does not have a Monument template (for it, {{Monumento italiano}}), then it needs manual attention − indeed being bumped down ; * it the file does have a monument template, then the bot should have been able to do something about it, but could not because of a mistake at some point in the process. More about answering Nemo below

Nemo wrote:

I know the Italian team is a bit confused, I don't know what we know. Thanks for the patience. For sure I don't understand the instructions, see https://commons.wikimedia.org/wiki/Commons_talk:Monuments_database/Categoriz...

Thanks for your questions! I answered on the talk page and tried clarifying the pages. Let me know if that helps.

AFAICS, User:Yiyi has specified a category for most monuments in the latest

Wikipedia lists, e.g. https://it.wikipedia.org/wiki/Progetto:Wiki_Loves_Monuments_2015/Monumenti/B... .

That’s good. We’ll have to check whether they are being slurped correctly in the database.

Why do we even have

...

https://it.wikipedia.org/wiki/Template:WLM2013-riga ?

Why is it strange? AFAICS all the others have a template to generate rows of monument lists tables. The current one is https://it.wikipedia.org/wiki/Template:WLM2015-riga . We use new templates and new lists every year because the authorised monuments vary every year.

Ok, here is the problem. The current design implies that one configuration (country, project) has one row template and only one. As a result, all monuments using WLM2014-riga and WLM2015-riga are *not* parsed. This also explains the problems with categorisation: the bot does find the monument in the database and thus cannot infer categories.

Looking at the WLM201X-riga templates, they appear to be perfect supersets of one another − all the fields of 2013 are in 2014, which are themselves in 2015. In this case, could we just unify the template?

(One may ask, « can’t the bot handle more than one row template per country/project ». Maybe so, but someone would have to code it ; and frankly I do not see a strong use case for it :)

2015-09-03 7:05 GMT+01:00 Federico Leva (Nemo) nemowiki@gmail.com:

Jean-Frédéric, 03/09/2015 01:52:

...

Ok, here is the problem. The current design implies that one configuration (country, project) has one row template and only one. As a result, all monuments using WLM2014-riga and WLM2015-riga are *not* parsed. This also explains the problems with categorisation: the bot does find the monument in the database and thus cannot infer categories.

Looking at the WLM201X-riga templates, they appear to be perfect supersets of one another − all the fields of 2013 are in 2014, which are themselves in 2015. In this case, could we just unify the template?

I did not set up the templates, but at worst a wrapper template can be made. However, can the bot fetch multiple sources (multiple roots for the lists) and merge data from multiple rows for a single ID?

No. I believe the latest crawl would replace previous values for the same ID.

In the meantime, I have changed the config to harvest the WLM2015-riga template. This has added some 3000 more monuments to the database [1] for Italy.

The bot also categorized hundreds of pictures from Italy, but I stopped it because of concerns raised on my talk page. See column 'Bugs' in [2]

[1] https://commons.wikimedia.org/w/index.php?title=Commons:Monuments_database/S... [2] https://phabricator.wikimedia.org/tag/wiki-loves-monuments-database/

Hi all,

Over the week-end, on top of fixing some outstanding bugs, I implemented for ErfgoedBot categorisation based on Wikidata.

I documented the updated process on: < https://commons.wikimedia.org/wiki/Commons:Monuments_database/Categorization...

I hope this will help getting more images categorised!

For more information, the Phabricator tasks: T110003 − “ErfgoedBot should support CommonsCat templates powered by Wikidata” https://phabricator.wikimedia.org/T110003

T111622 − “ErfgoedBot should try to retrieve CommonsCat from Wikidata sitelink, not only P373” https://phabricator.wikimedia.org/T111622

T111621 − “ErfgoedBot should always try to retrieve CommonsCats from Wikidata when inspecting monuments articles” https://phabricator.wikimedia.org/T111621

Cheers,