Hey all, migrating the remainder of the conversation here to mobile-l, with Ulf's and Jon's okay.
From: Ulf Buermeyer ulf@ijure.org To: Jon Katz jkatz@wikimedia.org, Internal communication for WMF Reading team reading-wmf@lists.wikimedia.org Cc: Yuri Astrakhan yastrakhan@wikimedia.org, Brian Gerstle < bgerstle@wikimedia.org>, Bernd Sitzmann bernd@wikimedia.org, Max Semenik < msemenik@wikimedia.org>, Corey Floyd cfloyd@wikimedia.org Date: Sat, 13 Jun 2015 16:12:52 -0700 Subject: Re: [reading-wmf] [WikimediaMobile] maps w/ nearby
Jon,
thanks for following up!
Just in order to share what we discussed yesterday let me briefly summarize:
Last year I submitted a pull request that introduces a simple map view to display places referenced by coordinates mentioned in an article (replacing the somewhat clumsy "look, there are 34 or so map providers on the internet" page). The code was technically fine but not merged in so far for policy reasons as MapViews on iOS display non-free content & hit Apple's servers.
So here are the options to move forward:
* WMF may revisit the issue and approve use of Apple maps + could be rolled out almost immediately - non-free content - privacy issue? - community may be fond of that map selection page?
* the MapView could be quickly tweaked to show nearby places (accompanying the current nearby list) + quick to implement - non-free content - privacy issue?
I have also coded an OpenStreetMap layer that can be displayed in an ordinary iOS MapView but replacing Apple map content so it could be used in both scenarios mentioned above to avoid hitting Apple servers. + free content - some time to implement (currently running in one of my apps but not yet transformed into a separate component)
As for the OSM tiles to display in said layer, there are once again three options:
* set up WMF tile rendering infrastructure + control over tile design + download from WMF servers -> no privacy issue - HUGE impact on engineering & operations resources
* cache official OSM tiles + download from WMF servers -> no privacy issue + small impact on operations, little engineering - some tile expiration logic / tweaking needed - need to consult with admins: http://wiki.openstreetmap.org/wiki/Tile_usage_policy
* proxy official OSM tiles + download from WMF servers -> no privacy issue + small impact on operations, practically no engineering - OSM guys from the UK may have concerns regarding load on their infrastructure - need to consult with admins: http://wiki.openstreetmap.org/wiki/Tile_usage_policy
Hope that helps, happy to help.
Best, Ulf
On 12/06/15 18:08, Jon Katz wrote:
Hi All, Tilman brought Ulf (copied) to the office today and it was great and energizing to sit down with him and hear his thoughts on maps and nearby options. As Tilman said earlier, Ulf has already done some work in this area and is happy to help out. He also has some ideas about how we could make things scale. I think the only thing holding him back right now is not knowing where he can add the most value.
I encourage Corey, Dmitry, Yuri and Max, in particular, to include Ulf in any conversations about this moving forward.
-J
On Fri, Jun 5, 2015 at 4:55 PM, Tilman Bayer <tbayer@wikimedia.org mailto:tbayer@wikimedia.org> wrote:
Hi all, as it happens, Ulf (CCed, the author of last year's GitHub pull request <https://github.com/wikimedia/apps-ios-wikipedia/pull/3> that Brian mentioned; he is the developer of a fairly popular geolocation iOS app <http://mapalarm.net/wp/> and runs his own OSM tileserver) is in SF again currently; we'll have lunch early next week. He would be delighted to come by the office and chat with people working in this area. (IIRC Max, Monte and Brion already met him last year, which led to that pull request.) Would some of the apps/maps people be interested in talking to him around 2pm on Monday, say? I showed Wikiminiatlas <https://wma.wmflabs.org/> to Ulf yesterday and he would be excited to use it to update his iOS app patch for a proof of concept that's in line with the privacy policy. I guess he could also use the new Karta server instead for something more production-like.
On Wed, May 27, 2015 at 3:32 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Disclaimer: I do not know much about the specifics of the iOS implementation, so can only speak about overall approach and possible concerns.
- Per WMF privacy policy, we cannot use outside servers if that exposes
our user's browsing behaviour. Thus any outside servers must be proxied to be consumable by our users.
- OSM data is not the same as the tiles people see - there is data (clone
of OSM db), there is data tiles (vector tiles) that only contain the features we try to show on a specific zoom level for that area, and there is rendering of that data as an image. From what I have learnt so far, most of the OSM-based stacks implement "raster tiles" - all images are pre-rendered as PNGs for speed.
- We are trying to switch to vector based approach, where tiles are stored
as data, and converted to image on request, either by the server or by the user's browser. Which means that if you have retina screen, the image will be rendered twice the regular size, and will be much crispier. This also means that you can rotate it on your device, or dynamically change the language.
- I suspect, and please update me on this if you know the details, that
the built-in iOS implementation can only handle images (raster) from the 3rd party (WMF). Mapbox's vector format is gaining in acceptance, and IIRC, has been selected by OSM community in general for the future development, but I am not sure iOS supports it natively.
On Wed, May 27, 2015 at 10:24 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
Agree with Corey, I've mentioned this elsewhere https://www.mail-archive.com/mobile-l@lists.wikimedia.org/msg03395.html, but it's important to note that as of iOS 6 https://blog.openstreetmap.org/2012/10/02/apple-maps/, Apple has been (and still appears to be http://applemapsmarketing.com/2014/11/open-street-map-apple-maps/) using OSM for map data. I don't know the rationale behind why we need our own OSM servers, but does that rationale also prevent our iOS app from using Apple-provided OSM data?
On Wed, May 27, 2015 at 4:05 PM, Corey Floyd cfloyd@wikimedia.org wrote:
Any reason to import a 3rd party library for functionality built in to the iOS SDK? Seems like work to explore/integrate a dependency where none is needed. Without this extra work, this is something that can be built in < 1 day.
On Wed, May 27, 2015 at 10:01 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
Cool, looks like mapbox has an iOS SDK https://www.mapbox.com/mapbox-ios-sdk/. Is there somewhere that the progress on funding is being tracked? Put another way, where should I direct PM, design, etc. to get this prioritized? Also, I think it'd be worth it to ship this to a percentage of users to (further) validate the feature.
On Wed, May 27, 2015 at 3:56 PM, Yuri Astrakhan < yastrakhan@wikimedia.org> wrote:
Current state of affairs: https://karta.wmflabs.org/static/ is up and running, and should have all the data soon. It uses mapbox stack, which means that it generates vector data tiles, and creates a PNG tiles on the fly. This also means that in a few days, you will be able to view WebGL based maps there too - rendered on the browser, with multiple styles, and rotatable.
The community needs this service, and has already built a large number of amazing projects even without the production-level vector service. Examples include
- atlas-style drill-down map
http://umap.openstreetmap.fr/de/map/wikipedia-clustermap_36725 (umap, see more info http://umap.openstreetmap.fr/)
- Wikidata-based map of pages by class
https://tools.wmflabs.org/wp-world/wikidata/superclasses.php?lang=en, e.g. all rollercoasters https://tools.wmflabs.org/wiwosm/osm-on-ol/kml-on-ol.php?lang=en&uselang=en&zoom=3&lat=0&lon=0&classes=204832 .
- There is an amazing presentation
https://tools.wmflabs.org/wp-world/docs/ICC2013-WP-OSM-white.pdf (pdf) by Kolossos (Tim Adler), that gives many more examples of the community-built map services and projects (OpenOffice format https://tools.wmflabs.org/wp-world/docs/ICC2013-WP-OSM-white.odp)
P.S. Tomasz, it's Yuri, not Yuvi, and don't blame IRC auto-complete :)
On Wed, May 27, 2015 at 7:57 PM, Max Semenik maxsem.wiki@gmail.com wrote:
Hey - yes, we're workig on serving map tiles. Both raster and MapBox vectors. Our current demo implementation is at https://karta.wmflabs.org/static/ - only raster tiles ATM, vectors coming in 1-2 days. The main problem, however, is budgeting - yet if we get very little hardware, concentrating on maps for apps might even be a reasonable option as apps traffic might be lower than if we exposed maps to all web users. Getting no bugdet at all is also a possible outcome, though.
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- Corey Floyd Software Engineer Mobile Apps / iOS Wikimedia Foundation
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l