I noticed the other day that HappyCow https://www.happycow.net/ uses OpenStreetMap tiles loaded from maps.wikimedia.org.
I guess it's a good thing (they're very visible in their niche and at the same time it must not be a lot of traffic), although not a primary purpose of the service. I'd argue it's a way to promote the importance of privacy, because now you don't need to query Google to find a restaurant with their internal search.
What cases of third-party usage of maps.wikimedia.org come to your mind which could be considered positive examples?
Federico
Hi Frederico,
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Maarten
On 26-01-20 11:31, Federico Leva (Nemo) wrote:
I noticed the other day that HappyCow https://www.happycow.net/ uses OpenStreetMap tiles loaded from maps.wikimedia.org.
I guess it's a good thing (they're very visible in their niche and at the same time it must not be a lot of traffic), although not a primary purpose of the service. I'd argue it's a way to promote the importance of privacy, because now you don't need to query Google to find a restaurant with their internal search.
What cases of third-party usage of maps.wikimedia.org come to your mind which could be considered positive examples?
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://phabricator.wikimedia.org/T245145#5880898 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) <nemowiki@gmail.com
:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Albin Larsson, 01/03/20 11:52:
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
Ah yes, and HappyCow has switched to their own tileserver, map.happycow.net, at some point in the last month. So it was indeed a (too) significant usage after all.
Federico
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://phabricator.wikimedia.org/T245145#5880898
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) < nemowiki@gmail.com>:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps. The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://phabricator.wikimedia.org/T245145#5880898 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) nemowiki@gmail.com:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hi Folks, On behalf of the WMF, I wanted to share more information about what has happened and what our plan is moving forward. We were actually discussing how to communicate this when the thread popped up, and I apologize for not sending something out sooner.
Basically, the load on our maps servers increased to the point where the service was falling down. We had an incident in February where this happened and while the root cause is not well understood yet, it was clear that the issue is systemic and this was not a 1-time event. Details are here https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps. A big part of our map traffic (~35% is from 3rd parties) and by limiting support for third parties, we were able to mitigate the issue.
The immediate change we made was to drop cache misses from 3rd parties, but to keep serving the tiles that are in cache. This lessens our loads without turning 3rd party support off entirely, but is understandably confusing to the few 3rd parties who are using our service. In terms of long term support, we do not have any plans to increase it and are actively discussing how we might further limit 3rd party support in the future. We're doing this proactively so that we can provide appropriate warning, details and dates for any further decreases in service that might be coming down the pike.
As always your feedback is very welcome. What do you think? What are your primary questions, etc. We may not be able to respond right away as finding answers can sometimes require communication across many time zones, but that doesn't mean we're not listening and taking it in.
Thanks,
Jon Director of Product Management, Wikimedia Foundation
On Sun, Mar 1, 2020 at 7:23 AM AntiCompositeNumber < anticompositenumber@gmail.com> wrote:
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at <https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps
.
The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still
use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy is
changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha
scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny
all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) <
nemowiki@gmail.com>:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
So, are you planning to definitively drop support for 3rd parties ? I would suggest you to use something like a key system plus a ratelimit.
You could whitelist only requests coming with an apikey set somewhere in the url (or based on a referrer whitelist) and under a certain threshold, like 1 request every 2 seconds for each tile. In this way , you are forting all 3rd parties to use a caching system and, for tiles not already cached, they won't have any service interruption thanks to the whitelist.
Il giorno lun 2 mar 2020 alle ore 20:49 Jon Katz jkatz@wikimedia.org ha scritto:
Hi Folks, On behalf of the WMF, I wanted to share more information about what has happened and what our plan is moving forward. We were actually discussing how to communicate this when the thread popped up, and I apologize for not sending something out sooner.
Basically, the load on our maps servers increased to the point where the service was falling down. We had an incident in February where this happened and while the root cause is not well understood yet, it was clear that the issue is systemic and this was not a 1-time event. Details are here. A big part of our map traffic (~35% is from 3rd parties) and by limiting support for third parties, we were able to mitigate the issue.
The immediate change we made was to drop cache misses from 3rd parties, but to keep serving the tiles that are in cache. This lessens our loads without turning 3rd party support off entirely, but is understandably confusing to the few 3rd parties who are using our service. In terms of long term support, we do not have any plans to increase it and are actively discussing how we might further limit 3rd party support in the future. We're doing this proactively so that we can provide appropriate warning, details and dates for any further decreases in service that might be coming down the pike.
As always your feedback is very welcome. What do you think? What are your primary questions, etc. We may not be able to respond right away as finding answers can sometimes require communication across many time zones, but that doesn't mean we're not listening and taking it in.
Thanks,
Jon Director of Product Management, Wikimedia Foundation
On Sun, Mar 1, 2020 at 7:23 AM AntiCompositeNumber anticompositenumber@gmail.com wrote:
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps. The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://phabricator.wikimedia.org/T245145#5880898 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) nemowiki@gmail.com:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hi Gandalf, I don't have an answer for you today, but am hopeful that we can have one for you by the end of March. Thank you for your suggestion though! Best,
Jon
On Thu, Mar 5, 2020 at 3:54 AM Gandalf Corvotempesta < gandalf.corvotempesta@gmail.com> wrote:
So, are you planning to definitively drop support for 3rd parties ? I would suggest you to use something like a key system plus a ratelimit.
You could whitelist only requests coming with an apikey set somewhere in the url (or based on a referrer whitelist) and under a certain threshold, like 1 request every 2 seconds for each tile. In this way , you are forting all 3rd parties to use a caching system and, for tiles not already cached, they won't have any service interruption thanks to the whitelist.
Il giorno lun 2 mar 2020 alle ore 20:49 Jon Katz jkatz@wikimedia.org ha scritto:
Hi Folks, On behalf of the WMF, I wanted to share more information about what has
happened and what our plan is moving forward. We were actually discussing how to communicate this when the thread popped up, and I apologize for not sending something out sooner.
Basically, the load on our maps servers increased to the point where the
service was falling down. We had an incident in February where this happened and while the root cause is not well understood yet, it was clear that the issue is systemic and this was not a 1-time event. Details are here. A big part of our map traffic (~35% is from 3rd parties) and by limiting support for third parties, we were able to mitigate the issue.
The immediate change we made was to drop cache misses from 3rd parties,
but to keep serving the tiles that are in cache. This lessens our loads without turning 3rd party support off entirely, but is understandably confusing to the few 3rd parties who are using our service. In terms of long term support, we do not have any plans to increase it and are actively discussing how we might further limit 3rd party support in the future. We're doing this proactively so that we can provide appropriate warning, details and dates for any further decreases in service that might be coming down the pike.
As always your feedback is very welcome. What do you think? What are
your primary questions, etc. We may not be able to respond right away as finding answers can sometimes require communication across many time zones, but that doesn't mean we're not listening and taking it in.
Thanks,
Jon Director of Product Management, Wikimedia Foundation
On Sun, Mar 1, 2020 at 7:23 AM AntiCompositeNumber <
anticompositenumber@gmail.com> wrote:
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at <
https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps%3E.
The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application
still use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy
is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha
scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should
deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) <
nemowiki@gmail.com>:
Maarten Dammers, 29/02/20 22:12: > Late reply. Prado has been using the Wikimedia maps for years, see > https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Hello maps folks,
I wanted to follow up on Jon's email earlier this month. I'm a product manager on Jon's team and temporarily helping with maps while Jon is out on personal leave.
We had hoped to provide a more definite statement on the future of maps by now. However, I don't have much concrete to share as of today. Thats partly due to changing world circumstances, but also because finding a long term sustainable path for maps continues to be difficult.
That said, we're still working towards a longer term statement and plan around maps. To reiterate Jon's suggestion, turning back on full support for open use across commercial and 3rd parties is unlikely. We've got so many vital uses of maps on wiki and we need to focus on those. No final decision has been made, and there is no imminent plan to close the service. But it if you are a non-Wikimedia user of our maps service, I would begin making contingency plans for another tile provider and expect that the current "cached only" tiles limitation will remain in place going forward.
Thanks, Josh
On Mon, Mar 2, 2020 at 11:49 AM Jon Katz jkatz@wikimedia.org wrote:
Hi Folks, On behalf of the WMF, I wanted to share more information about what has happened and what our plan is moving forward. We were actually discussing how to communicate this when the thread popped up, and I apologize for not sending something out sooner.
Basically, the load on our maps servers increased to the point where the service was falling down. We had an incident in February where this happened and while the root cause is not well understood yet, it was clear that the issue is systemic and this was not a 1-time event. Details are here https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps. A big part of our map traffic (~35% is from 3rd parties) and by limiting support for third parties, we were able to mitigate the issue.
The immediate change we made was to drop cache misses from 3rd parties, but to keep serving the tiles that are in cache. This lessens our loads without turning 3rd party support off entirely, but is understandably confusing to the few 3rd parties who are using our service. In terms of long term support, we do not have any plans to increase it and are actively discussing how we might further limit 3rd party support in the future. We're doing this proactively so that we can provide appropriate warning, details and dates for any further decreases in service that might be coming down the pike.
As always your feedback is very welcome. What do you think? What are your primary questions, etc. We may not be able to respond right away as finding answers can sometimes require communication across many time zones, but that doesn't mean we're not listening and taking it in.
Thanks,
Jon Director of Product Management, Wikimedia Foundation
On Sun, Mar 1, 2020 at 7:23 AM AntiCompositeNumber < anticompositenumber@gmail.com> wrote:
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at <https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps
.
The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still
use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy
is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha
scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny
all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) <
nemowiki@gmail.com>:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l