Hello folks,
service-template-node v0.4.0 has just been released~[1]. The new version
represents an important security and feature upgrade from v0.3.2 and you
are urged to update as soon as possible~[2].
On the feature side, this release brings out-of-the-box support for sending
metrics for all requests made against a service, which means that after
upgrading you will be able to set up your own grafana dashboard with
relevant metrics~[3] very easily.
Security-wise, there were some possible RegEx exploits in one of the node
module dependencies. This has been mitigated by updating the relevant
modules to a version that does not have the deficiency. Additionally, from
now on a node-module-security scan is being run every time the service is
tested to ensure our infrastructure is kept safe.
Please update as soon as possible if you have a service based on the
service template running in WMF production. And, as always, should you have
any questions or concerns, feel free to reach out to me.
Cheers,
Marko
[1] https://github.com/wikimedia/service-template-node/tree/v0.4.0
[2] you can follow the guide on
https://www.mediawiki.org/wiki/ServiceTemplateNode/Updating
[3] dashboards a la
https://grafana.wikimedia.org/dashboard/db/restbase?panelId=16&fullscreen
--
Marko Obrovac, PhD
Senior Services Engineer
Wikimedia Foundation
If you have a client that connects to RCStream
<https://wikitech.wikimedia.org/wiki/RCStream>, please take a moment to
make sure that you are using a secure connection. Are you connecting to '
http://stream.wikimedia.org/rc' or '//stream.wikimedia.org/rc'? If so, the
only thing you need to change is the URL scheme, replacing any http: and
protocol-relative URLs with HTTPS (that is, '
https://stream.wikimedia.org/rc').
Some of the mobile apps teams have been wanting some way to store data "in
the cloud", so they can sync things like app preferences and reading lists
between devices. They asked us (Reading Infrastructure) to help draft an
RFC, which is now posted in https://phabricator.wikimedia.org/T128602.
Please comment there, there are several open questions. Thanks!
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
Kibana, the software running logstash.wikimedia.org, as well as
elasticsearch, the server behind the scenes storing all the logs, have both
been upgraded today to their latest versions. Several historical indices
could not be directly loaded into the new servers and instead have been
dumped and are being reloaded. This reloading process may take some time,
so please be patient. The data will be there sooner or later :)
Play around with them, and create tickets in the wikimedia-logstash project
for any issues you find.
Hey,
This is the (12 + 1)th [1] weekly update from revision scoring team that we
have sent to this mailing list.
New developments:
- ORES review tool is deployed as a beta feature in Turkish Wikipedia.
Now six Wikis have this tool. [2]
- CopyPatrol tool soon will show ORES scores if they pass a certain
threshold. [3]
- We are talking about integration with Detox, feel free to chime in. [4]
Maintenance and robustness:
- Currently we are dealing with increasing memory pressure on scb nodes.
Actions we did to reduce this pressure:
- We migrated most of RandomForrest models to GradientBoosting, which
will reduce memory pressure greatly without affecting accuracy of models
noticeably [5]
- It seems there is a memory leak issue with celery. We bypassed that by
setting a periodic restart of workers. [6]
- We reduced maximum number of precaching requests in order to prevent
spikes that might cause memory pressure on other services. [7]
- We reduced number of web processes to 2/3. It is still fine. [8]
- We finished up the refactor and it will soon goes to the production
cluster. [9]
- Damaging and goodfaith models had issue in Dutch Wikipedia. Got fixed.
[10]
- Our metrics collector now sends timed requests. We will have a
dashboard in grafana for that soon. [11]
- There will be a link to "ORES review tool" page [12] in legend of
RecentChanges and Watchlist. [13]
- When revscoring fails for any unknown reason, ORES return a proper
message now. [14]
- We fixed a puppet issue that caused trouble while creating new web
instances. Got fixed [15]
1. It's 13. We are not superstitious, just kidding ;)
2. https://phabricator.wikimedia.org/T139992
3. https://phabricator.wikimedia.org/T139009
4. https://phabricator.wikimedia.org/T139007
5. https://phabricator.wikimedia.org/T139963
6. https://phabricator.wikimedia.org/T140020
7. https://gerrit.wikimedia.org/r/299559
8. https://gerrit.wikimedia.org/r/298739
9. https://github.com/wiki-ai/ores/pull/155
10. https://phabricator.wikimedia.org/T140038
11. https://phabricator.wikimedia.org/T137442
12. https://www.mediawiki.org/wiki/ORES_review_tool
13. https://phabricator.wikimedia.org/T140361
14. https://phabricator.wikimedia.org/T140301
15. https://phabricator.wikimedia.org/T140265
Sincerely,
Amir from the Revision Scoring team
This update will definitely improve our experience with git. As others have said before, the inline editing will greatly speed up the review process and improve code quality. This is especially true for the extensions, as you now do not have to clone a repo or reject the commit just to fix a typo or some coding conventions. Thanks for making this happen!
I found that the behavior of my old search filters has changed. I used to look for bluespice status:open to get all open commits in all bluespice repos. This however does not work in the test instance, I only get a small subset of the commits I need to see. There are ways around this, e.g. ownerin:bluespice status:open, so I am not too concerned about it. Am not sure whether it affects other extension writers, though.
Cheers,
Markus
(mglaser)
