Hi!
We are happy to announce the new domain 'toolforge.org' is now ready to be
adopted by our Toolforge community.
There is a lot of information related to this change in a wikitech page we have
for this:
https://wikitech.wikimedia.org/wiki/News/Toolforge.org
The most important change you will see happening is a new domain/scheme for
Toolforge-hosted webservices:
* from https://tools.wmflabs.org/<toolname>/
* to https://<toolname>.toolforge.org/
A live example of this change can be found in our internal openstack-browser
webservice tool:
* legacy URL: https://tools.wmflabs.org/openstack-browser/
* new URL: https://openstack-browser.toolforge.org
This domain change is something we have been working on for months previous to
this announcement. Part of our work has been to ensure we have a smooth
transition from the old domain (and URL scheme) to the new canonical one.
However, we acknowledge the ride might be bumpy for some folks, due to technical
challenges or cases we didn't consider when planning this migration. Please
reach out intermediately if you find any limitation or failure anywhere related
to this change. The wikitech page also contains a section with information for
common problems.
You can check now if your webservice needs any specific change by creating a
temporal redirection to the new canonical URL:
$ webservice --canonical --backend=kubernetes start [..]
$ webservice --canonical --backend=gridengine start [..]
The --canonical switch will create a temporal redirect that you can turn on/off.
Please use this to check how your webservice behaves with the new domain/URL
scheme. If you start the webservice without --canonical, the temporal redirect
will be removed.
We aim to introduce permanent redirects for the legacy URLs on 2020-06-15. We
expect to keep serving legacy URLs forever, by means of redirections to the new
URLs. More information on the redirections can also be found in the wikitech page.
The toolforge.org domain is finally here! <3
--
Arturo Borrero Gonzalez
SRE / Wikimedia Cloud Services
Wikimedia Foundation
Hi,
Do you use, or want to use Rust on Toolforge? While it's not officially
supported, some of us are discussing on Phabricator what the current
pain points are, how to work around them and what we'd like to see in
the future.
If you have comments/suggestions/feedback, we'd appreciate it either the
task[1] or just as a reply on this mailing list. I've also started a
documentation page that may help people out[2] - let us know if you'd
like to see something added to or clarified on that page.
Additionally, if you're currently using Rust, please add your name &
tool to the list I've started on the wikitech page[2] so we can gauge
what the usage and interest is.
[1] https://phabricator.wikimedia.org/T194953
[2] https://wikitech.wikimedia.org/wiki/User:Legoktm/Rust_on_Toolforge
Thanks,
-- Legoktm
[Apologies for cross-posting]
Hi,
TL;DR: We will take our Gerrit instance at gerrit.wikimeda.org offline
on Saturday, 27th of June from 17:00--22:00 UTC (hopefully much
shorter) to upgrade to the latest Gerrit v3.2.2.
(10:00--15:00 San Francisco time, and 19:00--24:00 Central
European time)
---------------------------------
We're currently running Gerrit v2.15, which is no longer supported and
we will upgrade to Gerrit v3.2 on Saturday 27th of June. This new
version brings lots of improvements. The most noticeable change is
probably the UI overhaul. The (default) "old UI" gets removed and the
"new UI" has been completely revamped. It's much more modern looking
and easier to use. And it also became far more usable on mobile
phones. But the new version of course also comes with fixes and
additions around the ssh interface, and the HTTP API.
Since this is such a big upgrade, we want to give you a chance to look
at it beforehand. You can test at
https://gerrit-test.wikimedia.org/
It holds a copy of our Gerrit data as of 15th of June, and it runs the
new Gerrit v3.2.2. We'll keep tuning things there during this week, so
it might get an occasional restart. But look around and test the new
UI, the HTTP API, etc.
If you run into issues or discover bugs, please open tickets in
Phabricator and set the Gerrit 3.x upgrade task (T254158) as parent.
Note that this test Gerrit instance is already hooked up with our
Phabricator. If you upload / merge / abandon / etc changes with a
`Bug: Tnnnn` footer, you'll get comments on the corresponding tasks in
Gerrit. So please use our test task T775 for your experiments.
As everything, also this upgrade comes with a few downsides:
* The new UI no longer works on Internet Explorer (currently ~0.02% of
Gerrit's web requests).
Microsoft's Edge browser works though.
So if you are still using Internet Explorer, please consider switching
to a different browser.
* Users of git-review need to use at least version 1.27 (released
September 2018). Otherwise uploading changes will fail.
* Your git should be at least version 2.2.0 (released November 2014)
to take advantage of the new, much simpler commit-msg hook.
Keeping my fingers crossed for a painless update,
Christian with lots of helping hands from RelEng, and SRE
I'm running a web server with "webservice --backend=kubernetes python3.7". As I tail the uwsgi.log file, requests to my server get logged with very long delays. I just timed one at about a minute and a half between when the request was served (03:07:57 UTC 2020) and when it showed up in the log file (03:09:23 UTC 2020).
What is going on here? Is there some way to make it not do this?
I'm mostly a python guy, who only dabbles in javascript when I can't avoid it :-)
I've got a django-based tool (https://tools.wmflabs.org/spi-tools-dev/spi/ <https://tools.wmflabs.org/spi-tools-dev/spi/>) which uses selectize.js. For simple cases, everything works fine. But, if I click on the drop-down, delete the contents, type in a new value, and submit the form (Sock Info button), the submitted form data has case_name blank.
Is there anybody here who is familiar with javascript and/or selectize.js who would be willing to point me in the right direction?
I'm interested in running some long-ish scripts that loop through the dump
replicas on Toolforge. Eventually, this sort of thing might move to
crontab, but for now it would be nice to run a screen session as we test /
debug the scripts. The problem is that if I run the scripts from my tool
account (i.e. after "become <tool-name>"), I get the following error: Cannot
open your terminal '/dev/pts/28' - please check.
The internet indicates that this is because there is a user mismatch
because of the "become" command [1]. The suggested solutions weren't
clearly applicable:
* I can't just skip the "become" step and run the screen session as my
personal username as that's explicitly against the rules [2] (though it is
possible to start a screen session from my personal username without
getting the above error).
* It's not clear to me that running the screen session as /dev/null is
possible / smart / secure etc. Essentially, I don't have a great read on
whether that actually is the correct solution or just another bad hack.
So is it possible to run screen sessions from my tool account? If so, does
anyone know how I might get around this issue?
[1]
https://stackoverflow.com/questions/21328140/screen-cannot-open-your-termin…
[2] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Rules
Best,
Isaac
--
Isaac Johnson (he/him/his) -- Research Scientist -- Wikimedia Foundation
Hi,
we just enabled email ratelimiting in our MTA server [0] in Toolforge.
Please, report any problem or issue you may find related to this.
The current limit is 100 messages per hour per sender address. We may tune the
value as we observe the behavior of the system and the users.
regards.
[0] https://en.wikipedia.org/wiki/Message_transfer_agent
--
Arturo Borrero Gonzalez
SRE / Wikimedia Cloud Services
Wikimedia Foundation
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
This message is a notice about forthcoming removal of a response field in
the REST API’s /page/summary endpoint. [1] This message was posted to
wikitech-l and is being cross-posted on mediawiki-api-announce.
The endpoint is used in the Page Previews (“hovercards”) functionality on
the classic web (desktop) and Android & iOS experiences for Wikipedia, in
addition to numerous external experiences. We don't anticipate impacts on
the mainline Wikipedia experiences from this forthcoming field removal, as
the endpoint will still be operational.
The REST API's /page/summary endpoint provides an api_urls field in its
response with links to several other REST API endpoints supported by the
Page Content Service.
Three of the api_urls subfields refer to experimental endpoints that have
been removed from the REST API altogether in favor of newer API endpoints.
1. media (/page/media)
2. references (/page/references)
3. metadata (/page/metadata).
api_urls also contains subfields referring to stable endpoints that
continue to exist in the REST API:
1. summary (which is self referential)
2. edit_html (/page/html - the Parsoid HTML)
3. talk_html (/page/html/Talk:<title> - the Parsoid HTML for the
corresponding Talk page)
Wikimedia’s Product Infrastructure team intends to remove the api_urls
field of the /page/summary response.
A cursory review at https://codesearch.wmflabs.org/ and Wikimedia Git
mirrors suggests api_urls isn’t in use in consuming code.
Review of web logs suggests traffic for the following endpoints:
- The media endpoint is at about 5% of its original traffic before its
decommission and it appears to be from old Wikipedia for Android clients.
This is expected.
- The references endpoint’s Wikipedia for Android traffic does not seem
present and its other traffic appears to be from non-user application
software based on User-Agent header components.
-The metadata endpoint’s traffic seems to have all but stopped.
This change is being announced in advance of the change because the
endpoint is advertised as stable. [2]
Please update your clients if you rely on the presence of the api_urls
field. If this change poses a problem for your clients, please do let us
know as soon as possible at the tracking task:
https://phabricator.wikimedia.org/T247991
We plan to remove the api_urls field described here on or after 14 July
2020.
Thank you.
Adam Baso
Director of Engineering
Wikimedia Foundation
[1]
https://en.wikipedia.org/api/rest_v1/#/Page%20content/get_page_summary__tit…
[2] Refer to
https://www.mediawiki.org/wiki/API_versioning#End_point_stability and
https://www.mediawiki.org/wiki/Wikimedia_Product/Wikimedia_Product_Infrastr…
for more information on stability designations.
_______________________________________________
Mediawiki-api-announce mailing list
Mediawiki-api-announce(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce
Tomorrow (June 11th) at 1600 UTC, we will be failing over the primary
NFS server to do maintenance and upgrades on it. The secondary partner
in the cluster is already upgraded and ready, and recent changes
*should* make it a fairly straightforward failover with a brief period
of high load. If it doesn't proceed smoothly, it will be a slightly
longer period of high load and NFS lockup as failover completes (10-20
min or so). After maintenance it will be failed back, which will also,
hopefully, be quick and painless.
--
Brooke Storm
SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org
IRC: bstorm_
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Hello!
Next week we'll be rebuilding and upgrading the hardware that provides
DNS service to cloud-vps and toolforge. These rebuilds will start at
14:00 UTC and the whole process may take 2-3 hours. It's likely that DNS
lookups will be somewhat slower as clients fail over between the
in-progress and the working server. In theory there should be few other
user-facing effects from these upgrades.
In practice, though, this isn't something that we've done for quite a
while, and touching DNS is always risky since it underlies pretty much
everything. Here are some things to be ready for:
- As a precaution we'll be disabling Horizon during the window to
prevent new VMs or DNS changes landing in an inconsistent state.
- Some badly-behaved DNS clients won't fail over properly and will
report errors when their primary DNS server is down.
- Puppet will almost certainly experience transient failures, since
Puppet is known to be one of those badly-behaved clients.
- If things go very badly there may be periods of total DNS outage which
will result in many WMCS-hosted services failing. There's no particular
reason that this /should/ happen, but this is the worst-case scenario.
For additional context, the phabricator task for this work is
https://phabricator.wikimedia.org/T253780
- Andrew + the WMCS team
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce