Hi Alex,
CCing the Wiki Loves Monuments list. Maybe someone feels like helping out.
Op 28-3-2012 0:10, Alexander Wagner schreef:
I only found these kgs_nr. Haven't found an easy way to harvest them.
I found a way, but not an easy one:
- Goto http://kgs-gis.admin.ch/kgs.php
- Zoom out to max
- check "Kulturgüter" on the left
- Use the (i)-tool on top and draw a rectangle over all switzerland
- all objects are shown in a list below the map. You also can
download CSVs, unfortunately seperated in some hundred objecttypelists...
Would this lists help you creating some script? I hope I don't need to add them manually...
We would somehow need to download all the csv files and put them in a big table at the toolserver. It's probably not too hard to have a program take the html page and look for all url's containing csv file. For each file download it and insert it in a database at the toolserver. Once we have this database we can compare it to the information we already have about Switzerland and match items. This is how I did it with the National Register of Historic Places. This information is in the p_erfgoed_nrhp_p database at the toolserver (server: sql). I wrote https://fisheye.toolserver.org/browse/multichill/bot/erfgoed/NRHP_refnums.py... for that. Could probably be adapted to work for Switzerland too.
Maarten
Hi!
Database with all data form http://kgs-gis.admin.ch/kgs.php is ready on toolserver (u_alexxw_p -> Swiss_KGS_Nr). I hope someone can help me with the second part :)
Greets -- Alex User:AleXXw http://de.wikipedia.org/wiki/Benutzer:AleXXw
On Wed, Mar 28, 2012 at 20:57, Maarten Dammers maarten@mdammers.nl wrote:
Hi Alex,
CCing the Wiki Loves Monuments list. Maybe someone feels like helping out.
Op 28-3-2012 0:10, Alexander Wagner schreef:
I only found these kgs_nr. Haven't found an easy way to harvest them.
I found a way, but not an easy one:
- Goto http://kgs-gis.admin.ch/kgs.php
- Zoom out to max
- check "Kulturgüter" on the left
- Use the (i)-tool on top and draw a rectangle over all switzerland
- all objects are shown in a list below the map. You also can download
CSVs, unfortunately seperated in some hundred objecttypelists...
Would this lists help you creating some script? I hope I don't need to add them manually...
We would somehow need to download all the csv files and put them in a big table at the toolserver. It's probably not too hard to have a program take the html page and look for all url's containing csv file. For each file download it and insert it in a database at the toolserver. Once we have this database we can compare it to the information we already have about Switzerland and match items. This is how I did it with the National Register of Historic Places. This information is in the p_erfgoed_nrhp_p database at the toolserver (server: sql). I wrote https://fisheye.toolserver.org/browse/multichill/bot/erfgoed/NRHP_refnums.py... that. Could probably be adapted to work for Switzerland too.
Maarten
Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.eu
The second part is fun to do. It's a nice puzzle to solve :-) You need a bit of python knowledge, a toolserver account and some time.
Maarten
Op 28 mrt 2012 om 23:42 heeft Alexander Wagner <alexander.wagner@wikipedia.at
het volgende geschreven:\
Hi!
Database with all data form http://kgs-gis.admin.ch/kgs.php is ready on toolserver (u_alexxw_p -> Swiss_KGS_Nr). I hope someone can help me with the second part :)
Greets
Alex User:AleXXw
On Wed, Mar 28, 2012 at 20:57, Maarten Dammers maarten@mdammers.nl wrote: Hi Alex,
CCing the Wiki Loves Monuments list. Maybe someone feels like helping out.
Op 28-3-2012 0:10, Alexander Wagner schreef:
I only found these kgs_nr. Haven't found an easy way to harvest them.
I found a way, but not an easy one:
- Goto http://kgs-gis.admin.ch/kgs.php
- Zoom out to max
- check "Kulturgüter" on the left
- Use the (i)-tool on top and draw a rectangle over all switzerland
- all objects are shown in a list below the map. You also can
download CSVs, unfortunately seperated in some hundred objecttypelists...
Would this lists help you creating some script? I hope I don't need to add them manually...
We would somehow need to download all the csv files and put them in a big table at the toolserver. It's probably not too hard to have a program take the html page and look for all url's containing csv file. For each file download it and insert it in a database at the toolserver. Once we have this database we can compare it to the information we already have about Switzerland and match items. This is how I did it with the National Register of Historic Places. This information is in the p_erfgoed_nrhp_p database at the toolserver (server: sql). I wrote https://fisheye.toolserver.org/browse/multichill/bot/erfgoed/NRHP_refnums.py... for that. Could probably be adapted to work for Switzerland too.
Maarten
Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.eu
Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.eu
Hi Alex,
Op 28-3-2012 23:42, Alexander Wagner schreef:
Database with all data form http://kgs-gis.admin.ch/kgs.php is ready on toolserver (u_alexxw_p -> Swiss_KGS_Nr). I hope someone can help me with the second part :)
I removed the truncating from the three Switzerland tables. This was causing strange results. Would be nice if someone could help out getting these numbers added. I already created https://commons.wikimedia.org/wiki/Template:Cultural_property_of_national_si... and https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Historic_sites/Unused_im... (+de and fr) and these will only work when the real KGS numbers are added.
Maarten
Greets
Alex User:AleXXw http://de.wikipedia.org/wiki/Benutzer:AleXXw
On Wed, Mar 28, 2012 at 20:57, Maarten Dammers <maarten@mdammers.nl mailto:maarten@mdammers.nl> wrote:
Hi Alex, CCing the Wiki Loves Monuments list. Maybe someone feels like helping out. Op 28-3-2012 0:10, Alexander Wagner schreef:
I only found these kgs_nr. Haven't found an easy way to harvest them. I found a way, but not an easy one: 1) Goto http://kgs-gis.admin.ch/kgs.php 2) Zoom out to max 3) check "Kulturgüter" on the left 4) Use the (i)-tool on top and draw a rectangle over all switzerland 5) all objects are shown in a list below the map. You also can download CSVs, unfortunately seperated in some hundred objecttypelists... Would this lists help you creating some script? I hope I don't need to add them manually...
We would somehow need to download all the csv files and put them in a big table at the toolserver. It's probably not too hard to have a program take the html page and look for all url's containing csv file. For each file download it and insert it in a database at the toolserver. Once we have this database we can compare it to the information we already have about Switzerland and match items. This is how I did it with the National Register of Historic Places. This information is in the p_erfgoed_nrhp_p database at the toolserver (server: sql). I wrote https://fisheye.toolserver.org/browse/multichill/bot/erfgoed/NRHP_refnums.py?hb=true for that. Could probably be adapted to work for Switzerland too. Maarten _______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org <mailto:WikiLovesMonuments@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.eu
Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.eu
On 15.04.2012 00:24, Maarten Dammers wrote:
Hi Alex,
Op 28-3-2012 23:42, Alexander Wagner schreef:
Database with all data form http://kgs-gis.admin.ch/kgs.php is ready on toolserver (u_alexxw_p -> Swiss_KGS_Nr). I hope someone can help me with the second part :)
I removed the truncating from the three Switzerland tables. This was causing strange results. Would be nice if someone could help out getting these numbers added. I already created https://commons.wikimedia.org/wiki/Template:Cultural_property_of_national_si... and https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Historic_sites/Unused_im... (+de and fr) and these will only work when the real KGS numbers are added.
Hi Alex and Maarten, if you address me correctly I may help you also to translate in the Swiss national languages (de+fr+it).
Hi,
Op 15-4-2012 10:00, Ilario Valdelli schreef:
Hi Alex and Maarten, if you address me correctly I may help you also to translate in the Swiss national languages (de+fr+it).
I'm not sure what you mean with addressing your correctly, but it would sure be nice to have it in Italian too to make it truly multilingual :-)
I don't think I'm able to make a bot to automatically add the numbers any time soon and we end up doing some manual work anyway so I created these three lists: * http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_de.txt * http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_en.txt * http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_fr.txt
These lists contain the items from Alex his database (thanks for that!) that are not yet in the monuments database. Would be nice if people could help with adding these numbers. I just did Zug in English and that goes pretty fast with a list like this. Not all items are in these lists because it's based on data from 2009.
Maarten
Ps. Yes, we seem to have some encoding problems......
I don't think I'm able to make a bot to automatically add the numbers any time soon and we end up doing some manual work anyway so I created these three lists:
http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_de.txt
http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_en.txt
http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_fr.txt
These lists contain the items from Alex his database (thanks for that!) that are not yet in the monuments database. Would be nice if people could help with adding these numbers. I just did Zug in English and that goes pretty fast with a list like this. Not all items are in these lists because it's based on data from 2009.
Maarten
English lists are currently filled with IDs.
There are two problems though. To explain them, let me call "lists" the Wikipedia lists like http://en.wikipedia.org/wiki/List_of_cultural_property_of_national_significa... this one, and "database" the pages Maarten compiled (like http://toolserver.org/~multichill/temp/queries/erfgoed/KGS_nr_to_add_en.txt this one). Until yesterday, the database was daily updated, and the items which had their numbers in the lists were removed. Consequently, only problematic objects should have stayed in the database in the end. (Right now, the English database is blank, but I hope this is just a glitch, since we still need it).
1. Some items in the lists have duplicate or multiple numbers (IDs) in the database. They are either really duplicate, or for instance the building contains a museum, both are protected and have the different ID, but physically this is of course still the same building. The record is six IDs for the same object, this is more like an exception, but two and three are common. Note that all duplicate items are still on the database: Maarten's bot only recognizes one ID per object (presumably, the first one). I put all the ID in the lists. However, this might be not such a good idea generally, since the uploaders will take the IDs from the lists, and if they code pictures of the same building with two different IDs, Maarten's bot could have trouble identifying them. If this is indeed a problem, we need to only keep one ID per object AND make sure that this one ID per object is the same on English, German, French, and potentially all other Wikipedia's which have lists. All ideas on how to best organize this are welcome.
2. There were several items in the database which were not on the lists, and I could not reasonably identify them with any objects in the lists (I tried really hard). There are not so many of them, may be couple of dozens for the whole country. On couple of occasions, I was sure they are just plainly forgotten, and add them (they are easy to spot since they have no coordinates). However, the remaining items, which are still in the database, require an expert attention. They may be just lost for whatever reason.
Cheers Yaroslav
On 15/04/12 00:24, Maarten Dammers wrote:
Hi Alex,
I removed the truncating from the three Switzerland tables. This was causing strange results. Would be nice if someone could help out getting these numbers added. I already created https://commons.wikimedia.org/wiki/Template:Cultural_property_of_national_si... and https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Historic_sites/Unused_im... (+de and fr) and these will only work when the real KGS numbers are added.
Maarten
What's the reason for *not* truncating everywhere? Avoid the state where not all rows are available? We could do BEGIN; DELETE; INSERT ...; COMMIT
The REPLACEs don't remove wrong data which was once present in the wiki.
On 28/03/12 20:57, Maarten Dammers wrote:
Hi Alex,
CCing the Wiki Loves Monuments list. Maybe someone feels like helping out.
Op 28-3-2012 0:10, Alexander Wagner schreef:
I only found these kgs_nr. Haven't found an easy way to harvest them.
I found a way, but not an easy one:
- Goto http://kgs-gis.admin.ch/kgs.php
- Zoom out to max
- check "Kulturgüter" on the left
- Use the (i)-tool on top and draw a rectangle over all switzerland
- all objects are shown in a list below the map. You also can
download CSVs, unfortunately seperated in some hundred objecttypelists...
Would this lists help you creating some script? I hope I don't need to add them manually...
We would somehow need to download all the csv files and put them in a big table at the toolserver. It's probably not too hard to have a program take the html page and look for all url's containing csv file. For each file download it and insert it in a database at the toolserver. Once we have this database we can compare it to the information we already have about Switzerland and match items.
This trick works:
var v = document.getElementsByTagName('a'); for (var i = 0; v < v.length; i++) { print(v[i].href); }
I have downloaded the 145 files at ~erfgoed/kgs-gis.admin.ch
wikilovesmonuments@lists.wikimedia.org