We now have built a vector-based Tile rendering service, called Kartotherian (work in progress). It is based on the components from Mapbox & Mapnik, and data from OSM->osm2pgsql->Postgress w/Postgis.Hi everyone, sorry for not replying earlier. Ulf, thank you for your offer to help, and everyone’s support of the map project!
Here's our accomplishments so far and the approximate path forward:
At this point, we are implementing a server-side Vector->PNG conversion, similar to mod_tile approach. But, internally, all data is stored as vector tiles, and converted to images on the fly. We are capable of also serving those tiles raw, without rasterisation, but we will still need to do some extra work to optimize their size at low zoom levels, wait for the complete font implementation (e.g. Arabic is not showing correctly), and a number of other minor issues. So client-side vector rendering is pushed for the second stage.On Fri, Jun 19, 2015 at 3:24 PM, Corey Floyd <cfloyd@wikimedia.org> wrote:I've been waiting to follow up on this until I got official clarification on WMF's position for using the Apple's framework…Good news, WMF legal has no privacy/legal concerns with using the MapKit framework - so we will be able to proceed with implementing map features using the official iOS SDK!This should make it much easier to develop this feature.On Sat, Jun 13, 2015 at 7:12 PM, Ulf Buermeyer <ulf@ijure.org> wrote:
Jon,
thanks for following up!
Just in order to share what we discussed yesterday let me briefly summarize:
Last year I submitted a pull request that introduces a simple map view
to display places referenced by coordinates mentioned in an article
(replacing the somewhat clumsy "look, there are 34 or so map providers
on the internet" page). The code was technically fine but not merged in
so far for policy reasons as MapViews on iOS display non-free content &
hit Apple's servers.
So here are the options to move forward:
* WMF may revisit the issue and approve use of Apple maps
+ could be rolled out almost immediately
- non-free content
- privacy issue?
- community may be fond of that map selection page?
* the MapView could be quickly tweaked to show nearby places
(accompanying the current nearby list)
+ quick to implement
- non-free content
- privacy issue?
I have also coded an OpenStreetMap layer that can be displayed in an
ordinary iOS MapView but replacing Apple map content so it could be used
in both scenarios mentioned above to avoid hitting Apple servers.
+ free content
- some time to implement (currently running in one of my apps but not
yet transformed into a separate component)
As for the OSM tiles to display in said layer, there are once again
three options:
* set up WMF tile rendering infrastructure
+ control over tile design
+ download from WMF servers -> no privacy issue
- HUGE impact on engineering & operations resources
* cache official OSM tiles
+ download from WMF servers -> no privacy issue
+ small impact on operations, little engineering
- some tile expiration logic / tweaking needed
- need to consult with admins:
http://wiki.openstreetmap.org/wiki/Tile_usage_policy
* proxy official OSM tiles
+ download from WMF servers -> no privacy issue
+ small impact on operations, practically no engineering
- OSM guys from the UK may have concerns regarding load on their
infrastructure
- need to consult with admins:
http://wiki.openstreetmap.org/wiki/Tile_usage_policy
Hope that helps, happy to help.
Best, Ulf
On 12/06/15 18:08, Jon Katz wrote:
> Hi All,
> Tilman brought Ulf (copied) to the office today and it was great and
> energizing to sit down with him and hear his thoughts on maps and nearby
> options. As Tilman said earlier, Ulf has already done some work in this
> area and is happy to help out. He also has some ideas about how we
> could make things scale. I think the only thing holding him back right
> now is not knowing where he can add the most value.
>
> I encourage Corey, Dmitry, Yuri and Max, in particular, to include Ulf
> in any conversations about this moving forward.
>
> -J
>
> On Fri, Jun 5, 2015 at 4:55 PM, Tilman Bayer <tbayer@wikimedia.org
> <mailto:tbayer@wikimedia.org>> wrote:
>
> Hi all,
>
> as it happens, Ulf (CCed, the author of last year's GitHub pull
> request
> <https://github.com/wikimedia/apps-ios-wikipedia/pull/3> that Brian
> mentioned; he is the developer of a fairly popular geolocation iOS
> app <http://mapalarm.net/wp/> and runs his own OSM tileserver) is in
> SF again currently; we'll have lunch early next week. He would be
> delighted to come by the office and chat with people working in this
> area. (IIRC Max, Monte and Brion already met him last year, which
> led to that pull request.) Would some of the apps/maps people be
> interested in talking to him around 2pm on Monday, say?
>
> I showed Wikiminiatlas <https://wma.wmflabs.org/> to Ulf yesterday
> and he would be excited to use it to update his iOS app patch for a
> proof of concept that's in line with the privacy policy. I guess he
> could also use the new Karta server instead for something more
> production-like.
>
>
> On Wed, May 27, 2015 at 3:32 PM, Yuri Astrakhan
> <bgerstle@wikimedia.org <mailto:bgerstle@wikimedia.org>> wrote:> <yastrakhan@wikimedia.org <mailto:yastrakhan@wikimedia.org>> wrote:
>
> Disclaimer: I do not know much about the specifics of the iOS
> implementation, so can only speak about overall approach and
> possible concerns.
>
> * Per WMF privacy policy, we cannot use outside servers if that
> exposes our user's browsing behaviour. Thus any outside servers
> must be proxied to be consumable by our users.
> * OSM data is not the same as the tiles people see - there is
> data (clone of OSM db), there is data tiles (vector tiles) that
> only contain the features we try to show on a specific zoom
> level for that area, and there is rendering of that data as an
> image. From what I have learnt so far, most of the OSM-based
> stacks implement "raster tiles" - all images are pre-rendered as
> PNGs for speed.
> * We are trying to switch to vector based approach, where tiles
> are stored as data, and converted to image on request, either by
> the server or by the user's browser. Which means that if you
> have retina screen, the image will be rendered twice the regular
> size, and will be much crispier. This also means that you can
> rotate it on your device, or dynamically change the language.
> * I suspect, and please update me on this if you know the
> details, that the built-in iOS implementation can only handle
> images (raster) from the 3rd party (WMF). Mapbox's vector format
> is gaining in acceptance, and IIRC, has been selected by OSM
> community in general for the future development, but I am not
> sure iOS supports it natively.
>
> On Wed, May 27, 2015 at 10:24 PM, Brian Gerstle
>
> Agree with Corey, I've mentioned this elsewhere
> <https://www.mail-archive.com/mobile-l@lists.wikimedia.org/msg03395.html>,
> but it's important to note that as of iOS 6
> <https://blog.openstreetmap.org/2012/10/02/apple-maps/>,
> Apple has been (and still appears to be
> <http://applemapsmarketing.com/2014/11/open-street-map-apple-maps/>)
> using OSM for map data. I don't know the rationale behind
> why we need our own OSM servers, but does that rationale
> also prevent our iOS app from using Apple-provided OSM data?
>
>
> On Wed, May 27, 2015 at 4:05 PM, Corey Floyd
> <cfloyd@wikimedia.org <mailto:cfloyd@wikimedia.org>> wrote:
>
> Any reason to import a 3rd party library for
> functionality built in to the iOS SDK? Seems like work
> to explore/integrate a dependency where none is needed.
> Without this extra work, this is something that can be
> built in < 1 day.
>
>
>
> On Wed, May 27, 2015 at 10:01 PM, Brian Gerstle
> <bgerstle@wikimedia.org <mailto:bgerstle@wikimedia.org>>
> wrote:
>
> Cool, looks like mapbox has an iOS SDK
> <https://www.mapbox.com/mapbox-ios-sdk/>. Is there
> somewhere that the progress on funding is being
> tracked? Put another way, where should I direct PM,
> design, etc. to get this prioritized? Also, I think
> it'd be worth it to ship this to a percentage of
> users to (further) validate the feature.
>
> On Wed, May 27, 2015 at 3:56 PM, Yuri Astrakhan
> <yastrakhan@wikimedia.org
> <mailto:yastrakhan@wikimedia.org>> wrote:
>
> Current state of affairs:
> https://karta.wmflabs.org/static/ is up and
> running, and should have all the data soon. It
> uses mapbox stack, which means that it generates
> vector data tiles, and creates a PNG tiles on
> the fly. This also means that in a few days, you
> will be able to view WebGL based maps there too
> - rendered on the browser, with multiple styles,
> and rotatable.
>
> The community needs this service, and has
> already built a large number of amazing projects
> even without the production-level vector
> service. Examples include
> * atlas-style drill-down map
> <http://umap.openstreetmap.fr/de/map/wikipedia-clustermap_36725>
> (umap, see more info
> <http://umap.openstreetmap.fr/>)
> * Wikidata-based map of pages by class
> <https://tools.wmflabs.org/wp-world/wikidata/superclasses.php?lang=en>,
> e.g. all rollercoasters
> <https://tools.wmflabs.org/wiwosm/osm-on-ol/kml-on-ol.php?lang=en&uselang=en&zoom=3&lat=0&lon=0&classes=204832>.
> * There is an amazing presentation
> <https://tools.wmflabs.org/wp-world/docs/ICC2013-WP-OSM-white.pdf>
> (pdf) by Kolossos (Tim Adler), that gives many
> more examples of the community-built map
> services and projects (OpenOffice format
> <https://tools.wmflabs.org/wp-world/docs/ICC2013-WP-OSM-white.odp>)
>
>
> P.S. Tomasz, it's Yuri, not Yuvi, and don't
> blame IRC auto-complete :)
>
> On Wed, May 27, 2015 at 7:57 PM, Max Semenik
> <maxsem.wiki@gmail.com
> <mailto:maxsem.wiki@gmail.com>> wrote:
>
> Hey - yes, we're workig on serving map
> tiles. Both raster and MapBox vectors. Our
> current demo implementation is
> at https://karta.wmflabs.org/static/ - only
> raster tiles ATM, vectors coming in 1-2
> days. The main problem, however, is
> budgeting - yet if we get very little
> hardware, concentrating on maps for apps
> might even be a reasonable option as apps
> traffic might be lower than if we exposed
> maps to all web users. Getting no bugdet at
> all is also a possible outcome, though.
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> <mailto:Mobile-l@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> <mailto:Mobile-l@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
>
>
> --
> EN Wikipedia user
> page: https://en.wikipedia.org/wiki/User:Brian.gerstle
> IRC: bgerstle
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> <mailto:Mobile-l@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
>
>
> --
> Corey Floyd
> Software Engineer
> Mobile Apps / iOS
> Wikimedia Foundation
>
>
>
>
> --
> EN Wikipedia user
> page: https://en.wikipedia.org/wiki/User:Brian.gerstle
> IRC: bgerstle
>
>
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org <mailto:Mobile-l@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
>
>
> --
> Tilman Bayer
> Senior Analyst
> Wikimedia Foundation
> IRC (Freenode): HaeB
>
> _______________________________________________
> reading-wmf mailing list
> reading-wmf@lists.wikimedia.org <mailto:reading-wmf@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/reading-wmf
>
>
--Corey FloydSoftware EngineerMobile Apps / iOSWikimedia Foundation
_______________________________________________
reading-wmf mailing list
reading-wmf@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/reading-wmf