Hello,
in the last days Toolserver experienced outages of web pages which were caused by too many queries from only a few hosts.
They are using OSM images and - please dont ask me why - single IPs tend to query about 40-50 pictures per second - for minutes or hours, peaks can be worse.
At some points our web server give up then.
Yes sorry ;). I can proudly say that only today about 11.7 millions web queries were answered somehow.
I tried to mitigate the problem of "too many requests per IP" via blocking but it is not an option.
One problem is that users of at least one portal then complain and another is that the IP addresses seem random - coming even from dial up ranges.
There might be something badly wrong with cache-control headers for the images (or probably we can tweak at that point) or - I dont know what it could be.
To make the long story short - I rate limited the OSM tile delivery to 40 images per second per IP - allowed burst is 55.
Users will then get a 503 error if the rate exceeds until it decreases - but delivery isnt stopped completely.
It seems to work since I have some notices which IPs were throtteled and these are IPs that have heavy usage.
I used this here to throttle: http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
I dont want to have this option configured forever - I rather hope we can do something about caching or give the pictures they need to the projects themselves (I doubt we have to deliver hill shading pictures for everyone - this is Toolserver)
If anyone has an idea what to do / questions - please let me know.
Cheers
Marlen/nosy
Hello all,
for adding a SSL-certificate I need to restart the web-loadbalancer
and/or the webserver (not sure at the moment). I plan to do this at
SUNDAY, 13:00 UTC.
If all works right everything will be done in <1 minute, but if there
are problems there could be some downtime (max 1/2h) for webpages.
Sincerely,
DaB.
https://jira.toolserver.org/browse/MNT-1347
Hello all,
how the WMF announced [1], the password-hashes and email addresses of
many users were public accessible in WikiLabs (and so ToolLabs) for 6
months.
So please make sure that you and your bots get a new password as soon as
possible! A well known bot in the wrong hands is dangerous, so change
the password now – don’t wait if you get a mail by the WMF (I got none,
but be affected AFAIS).
Sincerely,
DaB.
[1] https://meta.wikimedia.org/wiki/October_2013_private_data_security_issue
Hi all!
When it comes to the question of migrating tools to Tool Labs, some of you
might notice that you don't have the amount of time needed for maintenance
that you had earlier. WMDE offered to help you with the actual migration
but we can't maintain abandoned tools in the long run. For migration
support, contact me!
Note that in Tool Labs *every* tool is a multi-maintainer project! This
shall make it easier for you to bring others in. On the toolserver you can
add a ticket in Jira to request for your project to become an MMP, so
others can join you before migration and help with it.
Best, Silke
--
Silke Meyer
Internes IT-Management und Projektmanagement Toolserver
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. (030) 219 158 260
http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
I'm about to head off for three months for a sabbatical, so I wanted to
let you know of a few more underappreciated resources.
https://lists.wikimedia.org/mailman/listinfo/wikitech-announce is a very
low-traffic list with announcements of office hours, meetups, bug days,
and so on. And https://meta.wikimedia.org/wiki/Tech/News is a weekly
summary of changes in the world of Wikimedia tech.
http://vmartinezdelacruz.com/in-a-nutshell-how-openstack-works/ is an
introduction to how OpenStack works. Wikimedia Labs is built in
OpenStack; this will help you if you've ever wondered "WTF is Nova?"
And there's a sample Python application
http://tools.wmflabs.org/flask-stub/ "for those who want some yeast to
start their own project." More information:
https://wikitech.wikimedia.org/wiki/Flask-stub Thanks, User:Little green
rosetta.
I'm unsubscribing from the toolserver-l list and labs-l for my
sabbatical now -- more info at
http://www.harihareswara.net/sumana/2013/08/28/1 . If you need to talk
about Wikimedia technical community stuff before January, please consult
Quim Gil, qgil at wikimedia dot org. Silke and Coren will be able to
answer your questions and keep momentum going on Tool Labs, as they
already are already. :-) Thanks! Looking forward to coming back in January.
--
Sumana Harihareswara
Engineering Community Manager
Wikimedia Foundation
Erwin85 is looking for people to maintain his very popular tools [1] (as he
wrote on [2]). They are currently running on the toolserver. If you are
looking for a project to help with, please have a look at the tools.
If you are interested in helping to maintain one or a few of them, here are
your next steps:
* Contact Erwin85. ;)
* Ask in Jira [3] to have the tool you want to work on transformed into a
multi-maintainer project and have yourself added to it.
* Migrate them to Tool Labs. (WMDE can help here, if you want. Contact me!)
* Take over responsibility for long term maintenance of one or several of
the tools.
Cheers, Silke
[1] https://toolserver.org/~erwin85/
[2] https://meta.wikimedia.org/wiki/User_talk:Erwin#Tool_Labs
[3] https://jira.toolserver.org
--
Silke Meyer
Internes IT-Management und Projektmanagement Toolserver
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. (030) 219 158 260
http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help
If you know anyone who's been confused by Labs documentation in the
past, the revamped Tool Labs guide is pretty helpful. Thanks to Ryan
Kaldari for recently adding instructions for configuring MySQLWorkbench
(an alternative to phpMyAdmin).
--
Sumana Harihareswara
Engineering Community Manager
Wikimedia Foundation
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello DaB!
Could you please give me a short update - the end of 2013 is
approaching, what is the current state for TS resp. Labs?
Thanks and Greetings
DrTrigon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlIGCboACgkQAXWvBxzBrDAKPwCgjsEcGfRtbKaw/eDwTxNhUFfo
XgwAoLzV6J+TCdJZ3lvwvkqsMWihQmr+
=1dN+
-----END PGP SIGNATURE-----