So, are you planning to definitively drop support for 3rd parties ? I would suggest you to use something like a key system plus a ratelimit.
You could whitelist only requests coming with an apikey set somewhere in the url (or based on a referrer whitelist) and under a certain threshold, like 1 request every 2 seconds for each tile. In this way , you are forting all 3rd parties to use a caching system and, for tiles not already cached, they won't have any service interruption thanks to the whitelist.
Il giorno lun 2 mar 2020 alle ore 20:49 Jon Katz jkatz@wikimedia.org ha scritto:
Hi Folks, On behalf of the WMF, I wanted to share more information about what has happened and what our plan is moving forward. We were actually discussing how to communicate this when the thread popped up, and I apologize for not sending something out sooner.
Basically, the load on our maps servers increased to the point where the service was falling down. We had an incident in February where this happened and while the root cause is not well understood yet, it was clear that the issue is systemic and this was not a 1-time event. Details are here. A big part of our map traffic (~35% is from 3rd parties) and by limiting support for third parties, we were able to mitigate the issue.
The immediate change we made was to drop cache misses from 3rd parties, but to keep serving the tiles that are in cache. This lessens our loads without turning 3rd party support off entirely, but is understandably confusing to the few 3rd parties who are using our service. In terms of long term support, we do not have any plans to increase it and are actively discussing how we might further limit 3rd party support in the future. We're doing this proactively so that we can provide appropriate warning, details and dates for any further decreases in service that might be coming down the pike.
As always your feedback is very welcome. What do you think? What are your primary questions, etc. We may not be able to respond right away as finding answers can sometimes require communication across many time zones, but that doesn't mean we're not listening and taking it in.
Thanks,
Jon Director of Product Management, Wikimedia Foundation
On Sun, Mar 1, 2020 at 7:23 AM AntiCompositeNumber anticompositenumber@gmail.com wrote:
The current restriction is still a temporary mitigation for an outage. Draft information about the outage is available at https://wikitech.wikimedia.org/wiki/Incident_documentation/20200204-maps. The Wikimedia Foundation has not yet made a policy decision about external access to the maps service. You should expect a public announcement when that decision is made; if it's not sent to this list I'll forward it across.
On Sun, Mar 1, 2020 at 7:47 AM Gandalf Corvotempesta gandalf.corvotempesta@gmail.com wrote:
A whitelist is possibile? We use cloudflare with a forced 1 year cache but our application still use Wikimedia maps as tile server
We talked about this 1 year ago and It was ok for you, if your policy is changed with no warning, it's a huge issue for us as we have to refactor a lot of things.....
Il dom 1 mar 2020, 12:57 Albin Larsson albin.post@gmail.com ha scritto:
Hi,
It's worth mentioning that WMFs Kartotherian instance now should deny all non-cached tile requests from third-party domains. Tiles cached in Varnish should however return as normal.
https://phabricator.wikimedia.org/T245145#5880898 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/de8777646...
// Albin
Den lör 29 feb. 2020 kl 23:24 skrev Federico Leva (Nemo) nemowiki@gmail.com:
Maarten Dammers, 29/02/20 22:12:
Late reply. Prado has been using the Wikimedia maps for years, see https://www.museodelprado.es/ .
Thanks! I had missed that (I rarely scroll that far). Funny.
Their website seems to have less traffic but the map is on the front page. I'll definitely mention it next time I need an example. :-)
Federico
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l