Maps-l

maps-l@lists.wikimedia.org

495 discussions

Redacted database content
by Marlen Caemmerer 06 Dec '13

06 Dec '13

Hello, I was asked by WMF to make the database views on Toolserver fit to their privacy policy they applied at Toollabs. This means some data in the databases should not be displayed any longer. It regards fields in the tables revision, logging, recentchanges, oldimage and filearchive. Table entries that should be hidden will not be displayed by the Toolserver database servers. These entries can be unhidden by the Steward level users of the Wikipedias and are then accessible again. Some fields are not even shown in the default view because queries on these tables then no longer can use indexes which makes the databases and queries run very slow. I put alternative views in place in this case in case you need more fields you'll find slower views with most of the fields (but redacted). Please note that this may change the number of rows and other results of your tools. If you have any issues with your tools because of this change please let me know. Cheers Marlen/nosy

1 0

Fwd: [Labs-l] Labs datacenter migration
by aude 03 Dec '13

03 Dec '13

FYI ---------- Forwarded message ---------- From: Andrew Bogott <abogott(a)wikimedia.org> Date: Fri, Nov 15, 2013 at 8:29 PM Subject: [Labs-l] Labs datacenter migration To: "A list for announcements and discussion related to the Wikimedia Labs project." <labs-l(a)lists.wikimedia.org>, Wikimedia developers < wikitech-l(a)lists.wikimedia.org> Almost a year ago, the Wikimedia Foundation migrated most of our services from our old data center in Tampa to the new one in Ashburn [1]. In the next couple of months Labs and Tool Labs will be following suit -- we expect to have everything moved to Ashburn by mid-January at the latest. This move will provide some immediate benefits (lower latency with production, quicker database replication) and many long-term benefits (better stability, happier Operations staff). We don't yet have a specific timeline for stages of the migration, but there are a few things you can do now to help us prepare for the change and to bolster your projects against possible disruption. 1) Subscribe to Labs-l, and read it. [2] Labs-l is low-volume, and future migration announcements may not be sent to other lists. 2) Tool Labs users: As long as your tools are properly managed by the grid engine and can survive stops and restarts, the migration will be quite painless. If your tools aren't, or can't... fix them :) 3) Labs project admins: Clean up old projects and instances. If you have instances that are no longer of interest, delete them. If you know of entire projects that are no longer in use, please contact me directly and I'll mop up. 4) Labs instance owners: Make sure that puppet is running properly on your instances. If '$sudo puppetd -tv' produces any red lines, then fix them or contact me for help with fixing. When instances move to the new data center we'll be relying on puppet to update location-specific settings, so instances without puppet may not survive the move. If your instance uses self-hosting puppet (via puppetmaster::self or role::puppet::self) then you will also need to update your local puppet repo. [3] 5) All Labs users: if you have valuable data residing on local instance storage, start backing it up to shared storage in /data/project. You should be doing this anyway -- no instance is safe from catastrophe, ever. 6) If your project or tool generates log files, have a look at purging old log data. The last time we did a data migration there was at least one terabyte-sized logfile that really gummed up the works. Updates about this change will be posted to this list as soon as we know about them. Any potential downtime will be announced well in advance. In the meantime, don't hesitate to ask questions about the above steps on IRC or the mailing list. -Andrew [1] https://blog.wikimedia.org/2013/01/19/wikimedia-sites- move-to-primary-data-center-in-ashburn-virginia/ [2] https://lists.wikimedia.org/mailman/listinfo/labs-l [3] https://wikitech.wikimedia.org/wiki/Help:Self-hosted_puppetmaster#FAQ _______________________________________________ Labs-l mailing list Labs-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/labs-l -- @wikimediadc / @wikidata

1 0

Please join the Wikimaps design & developer meeting!
by Susanna Ånäs 01 Dec '13

01 Dec '13

Hi all! There will be 2 overlapping creative communities for planning Wikimaps 1. Wikimaps Designers & Developers (GLAM & map Wikipedians, OpenHistoricalMappers etc) 2. Wikimaps Nordic (Regional collaboration: Wikimedia chapters, wikimedians, mappers and GLAMs from the Nordic countries) This is an invitation to join the international designer & developer community. Wikimaps Nordic will form a project community in the beginning of January. We’ll arrange a video conference with *http://videoconf.wikimedia.ch/ <http://videoconf.wikimedia.ch/>* Tuesday December 3 at 8:00 PDT - 11:00 EST - 16:00 GMT - 17:00 CET - 18:00 EET - 21:00 IST Sign up in the collaborative Etherpad *http://wmfi.muistio.tieke.fi/wikimaps-online-1 <http://wmfi.muistio.tieke.fi/wikimaps-online-1>* Let's talk freely about what Wikimaps could be. Here are some ideas to get you started. See, comment and discuss them here: *https://docs.google.com/presentation/d/1EvXZtoV9vihtXZxVhA5ZKTEfmKFVI2btNQUREF-6so0/edit#slide=id.g11cd82f09_034 <https://docs.google.com/presentation/d/1EvXZtoV9vihtXZxVhA5ZKTEfmKFVI2btNQU…>* *Meeting agenda* Introductions 30 min 1. Round for presentations ~1 min per person 2. Brief presentations of Wikimaps projects ~2 min / ~8 projects Discussion 70 min 3. Discuss expectations. Create working groups. Focus on a project or two. 4. What kind of technologies and communication tools we will use Practicalities 20 min 5. Future schedule for the process 6. To-Do before the next meeting See you soon! Susanna Ånäs / Wikimedia Suomi ps. Info about Wikimaps Nordic with links to Wikimaps info: https://docs.google.com/document/d/1EhKgIUshgUHq3c4NeCDvMTPoRPCXxF-IInwtHHE… I've cross-posted this on maps-l, commons-l, cultural partners in wikimedia and historic at osm -- *Susanna Ånäs *Käyttäjä:Susannaanas Wikimedia Suomi <http://fi.wikimedia.org/wiki/Etusivu> – GLAM <http://fi.wikipedia.org/wiki/Wikipedia:GLAM> – tiedotus – yhteistyö blog.wikimedia.fi / @ <https://twitter.com/WMFinland> WMFinland <https://twitter.com/WMFinland> / Facebook <https://www.facebook.com/WikimediaSuomi> / Liity jäseneksi! <http://fi.wikimedia.org/wiki/Liity_j%C3%A4seneksi>

1 1

Re: [Maps-l] OpenStreetMap plans
by Andreas Trawoeger 12 Nov '13

12 Nov '13

Hi Quim, Thanks for keeping me updated. OpenGeoServer.org is currently running on a small rented 100 Euro/Month server and with funds from Wikimedia Germany running out it would be a pity to shut the service down, because no CPU cycles & disk storage can be found within the vast Wikimedia Foundation infrastructure. cu andreas 2013/11/7 Quim Gil <qgil(a)wikimedia.org>: > On 11/06/2013 03:03 AM, Andreas Trawoeger wrote: >> Hi Sumana! >> >> I'm the guy working at opengeoserver.org mentioned in [0]. > > Hi Andreas, I'm the guy tryng to cover Sumana's responsibilities while > she is on a Sabbatical. > > I don't have anything to add to the comments made here and at the > related bug report, but I didn't want to leave the impression that > nobody in Sumana's end was listening. :) > > Like others have said, I believe the best approach now (and always) is > regular and transparent communication and documentation of ongoing / > pending tasks. This way contributors like you know what is going on and > perhaps are able to help. > > -- > Quim Gil > Technical Contributor Coordinator @ Wikimedia Foundation > http://www.mediawiki.org/wiki/User:Qgil > > _______________________________________________ > Maps-l mailing list > Maps-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/maps-l

2 3

OpenStreetMap plans
by Sumana Harihareswara 11 Nov '13

11 Nov '13

Hi folks! I am the Engineering Community Manager at the Wikimedia Foundation, which means I'm a liaison for people who want to improve the technical capabilities of our projects. I spoke with a few of you offlist (including at Wikimania) about maps and Wikimedia. While I'm still getting up to speed on the proposals and tools people have made, I wanted to give you an update regarding what's happening with Wikimedia's planned OpenStreetMap tileserver. We know that we will set up a production tileserver. That is, Wikimedia plans to have an OpenStreetMap tileserver, in production (that is, fully supported by our Operations department and with uptime expectations similar to those for our main sites). We need this not only for the tools community but also for use in our official mobile apps. Brandon Black is the contact for this. He is a member of the Operations team at WMF. In order to set up a production-level tileserver, we will also need a production OSM database, so he is working on figuring out how best to grab that data from the official OSM database regularly (I am oversimplifying here). Brandon has already started on the project of setting up the production tileserver. I know that he will be posting regular updates at https://wikitech.wikimedia.org/wiki/OSM_Tileserver but he is still in the investigation and architecture phase so he does not have a timeline yet. I'm sorry, I know that is frustrating. Very soon after we have the production tileserver running, we plan to replicate that OSM database to Wikimedia Labs for use on Tool Labs, or do similar SQL import magic. People using Labs will be able to draw data directly from that replica, and will be able to grab tiles from the production tileserver. I'm the contact on this but since Coren (Marc-Andre Pelletier) is the domain expert on this particular bit of database replication, please go ahead and include both of us on any questions. Thanks, all. -- Sumana Harihareswara Engineering Community Manager Wikimedia Foundation

10 12

Fw: [Wikimediaindia-l] Request to review the Wikimaps Atlas Grant
by Pradeep Mohandas 18 Oct '13

18 Oct '13

hi, Thought this might have been of interest to this list. warm regards, Pradeep Pradeep Mohandas How Pradeep uses email? - http://goo.gl/6v1I9 On Friday, 18 October 2013 7:45 PM, Arun Ganesh <arun.planemad(a)gmail.com> wrote: Dear community, User:Yug[1] from the Graphic Labs and I, User:Planemad[2] from the Indian community, have put together a comprehensive IEG proposal to improve all the base maps used on WIkimedia projects with updated cartographic conventions and accuracy based on research. The output will be a large set of editable vector maps, research documentation and a comprehensive map creation workflow that anyone can reuse. The total grant request is for an amount of USD 10500, which will allow both of us to work on this project full time for a period of 3 months. The grant also includes a budget to hire external consultants where required to accomplish the stated goals. You can view the proposal and provide your valuable feedback/endorsement here: https://meta.wikimedia.org/wiki/Grants:IEG/Wikimaps_Atlas PS: This is the first step of the much larger Wikimaps project that aims to be the open source map engine that brings together Wikidata, Commons and Openstreetmap. [1] https://en.wikipedia.org/wiki/User:Yug [2] https://en.wikipedia.org/wiki/User:Planemad [3] https://meta.wikimedia.org/wiki/Wikimaps -- Arun Ganesh (planemad) _______________________________________________ Wikimediaindia-l mailing list Wikimediaindia-l(a)lists.wikimedia.org To unsubscribe from the list / change mailing preferences visit https://lists.wikimedia.org/mailman/listinfo/wikimediaindia-l

1 0

Making Toolserver work - rate limited OSM
by Marlen Caemmerer 08 Oct '13

08 Oct '13

Hello, in the last days Toolserver experienced outages of web pages which were caused by too many queries from only a few hosts. They are using OSM images and - please dont ask me why - single IPs tend to query about 40-50 pictures per second - for minutes or hours, peaks can be worse. At some points our web server give up then. Yes sorry ;). I can proudly say that only today about 11.7 millions web queries were answered somehow. I tried to mitigate the problem of "too many requests per IP" via blocking but it is not an option. One problem is that users of at least one portal then complain and another is that the IP addresses seem random - coming even from dial up ranges. There might be something badly wrong with cache-control headers for the images (or probably we can tweak at that point) or - I dont know what it could be. To make the long story short - I rate limited the OSM tile delivery to 40 images per second per IP - allowed burst is 55. Users will then get a 503 error if the rate exceeds until it decreases - but delivery isnt stopped completely. It seems to work since I have some notices which IPs were throtteled and these are IPs that have heavy usage. I used this here to throttle: http://nginx.org/en/docs/http/ngx_http_limit_req_module.html I dont want to have this option configured forever - I rather hope we can do something about caching or give the pictures they need to the projects themselves (I doubt we have to deliver hill shading pictures for everyone - this is Toolserver) If anyone has an idea what to do / questions - please let me know. Cheers Marlen/nosy

4 5

Maintenance: PHP update tomorrow
by Marlen Caemmerer 17 Sep '13

17 Sep '13

Hello, I will do a PHP update tomorrow Thursday 7pm - 9pm If interruptions occur they will be short. Cheers Marlen/nosy

1 2

State of the Map 2013 - OSM conference in Birmingham, England
by Cristian Consonni 04 Sep '13

04 Sep '13

Hi all, sorry for arriving to this thread so late, I subscribed to this list a couple of weeks ago but then forgot to confirm it. Should be fixed now. I am giving the talk "Data coherence between OSM and Wikipedia"[1]: this is about what I have been doing as part of my work at Fondazione Bruno Kessler in Trento (Italy). In a nutshell we want to compare the data in OSM and the data in Wikipedia and seek inconsistencies. The long term goal would be to produce useful tools for OSMers and Wikipedians to find (and therefore fix) everything that looks like an error. Incidentally I am a Wikipedian (I'm active mostly on it.wiki) since 2007 and I am the vicepresident of Wikimedia Italia, the Italian WMF chapter. I will upload my slide as soon as I can. Cristian [1] http://2013.stateofthemap.org/info/programme/

1 0

State of the Map 2013 - OSM conference in Birmingham, England
by Andy Mabbett 27 Aug '13

27 Aug '13

Hi folks - I'm new to the list. My en.Wikipedia user page has a potted biography. The OSM equivalent of Wikimania, State of the Map <http://2013.stateofthemap.org>, is in Birmingham, England (my home town), from 6-8 September, just under two weeks away (places are still available!). I will be there, facilitating some of the sessions and generally helping out. Are any of you coming? Should we have Wikipedia meetup? -- Andy Mabbett @pigsonthewing User:Pigsonthewing http://pigsonthewing.org.uk

4 5

← Newer
1
...
16
17
18
19
20
21
22
...
50
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Maps-l