May 2019 - Cloud - lists.wikimedia.org

[Cloud-announce] Dropping user text columns from replica views 2019-05-27
by Brooke Storm 04 Jun '19

04 Jun '19

Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 <https://phabricator.wikimedia.org/T223406>. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th. The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time. Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm(a)wikimedia.org <mailto:bstorm@wikimedia.org> IRC: bstorm_ _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

3 5

[Cloud-announce] cloudservices1003 rebuild on 2019-06-03
by Arturo Borrero Gonzalez 03 Jun '19

03 Jun '19

Hi! On 2019-06-03 UTC+2 14:00 (next monday) we will be rebuilding the cloudservices1003 server, that holds the designate service which serves DNS request for CloudVPS and Toolforge. We have a backup server -cloudservices1004-, so we don't expect a lot of downtime. But DNS queries are really fast, and there may be a lot of them that will fail while we stabilize the DNS service. Please reach out to the WMCS team if you need more details or have any doubts. regards. -- Arturo Borrero Gonzalez Operations Engineer / Wikimedia Cloud Services Wikimedia Foundation _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 1

Upgrade to MariaDB 10.2
by Huji Lee 02 Jun '19

02 Jun '19

Hi all, Does anyone know when we will be upgrading the Clouds replica DBs to MariaDB 10.2? I am asking mainly because we are on 10.1.33, 10.3 is now out, and since 10.2 support for CTEs (WITH statements) has been added which is very handy. Thanks, Huji

3 4

[Cloud-announce] NFS scratch mount changes 2019-05-28@1800 UTC
by Brooke Storm 28 May '19

28 May '19

As part of the efforts to retire labstore1003 and move to more modern hardware with some redundancy, we will begin the process of switching the mounts for /data/scratch to the new server starting 2019-05-28. It is advised to not use the /data/scratch NFS mount during the maintenance, starting next Tuesday at 1800 UTC until after it is announced over, which should be around an hour, since the mount will be changing location and will be generally somewhat unstable. NFS changes have occasionally caused more holistic problems within Toolforge in the past, but this should be fairly low-impact since it isn’t affecting /data/project or /home. Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm(a)wikimedia.org <mailto:bstorm@wikimedia.org> IRC: bstorm_ _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 2

Wednesday: Technical Advice IRC Meeting
by Michael Schönitzer 28 May '19

28 May '19

Reminder: Technical Advice IRC meeting this week **Wednesday 3-4 pm UTC** on #wikimedia-tech. The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

Where is webchat for wikimedia-labs?
by Thomas Stieve 24 May '19

24 May '19

Hello all, Could someone tell me where webchat for wikimedia-labs is now? https://webchat.freenode.net/?channels=wikimedia-labs Whenever I try to log in, I get: [15:43] -herbert.freenode.net- *** Notice -- TS for #wikimedia-labs changed from 1558651383 to 1320108738 [15:43] == tomthirteen was kicked from #wikimedia-labs by ChanServ [Invite only channel] How could I get invited? Where is the new chat? Thank you for your help! Tom -- Thomas Stieve Ph.D. Candidate School of Geography and Development University of Arizona

7 13

dologmsg command available on Toolforge for easier logging
by Lucas Werkmeister 19 May '19

19 May '19

Hi all! For a long time, tool developers have had the option to log their actions to a tool-specific server admin log (SAL), by sending messages like !log tools.*toolname* *text* to #wikimedia-cloud. This lets you and others keep track of which changes were made at what time, in case a problem is discovered later. However, to log messages you need to be on IRC, which can be inconvenient or even impossible on weak internet connections (I sometimes work on tools on the train, where the connection is just good enough for mosh <https://www.wikidata.org/wiki/Special:GoToLinkedPage/enwiki/Q15992114> but not really for IRC). Instead, you can now use the dologmsg command, which was implemented at the Wikimedia Hackathon in Prague (T222244 <https://phabricator.wikimedia.org/T222244>). Call it with your message as the argument(s), and it will relay it to #wikimedia-cloud through wm-bot, automatically including the tool name and the real user name of the caller. For example, when I ran the command dologmsg 'deployed 83d01bb93b (background UI improvements)' > the following entry was added to the QuickCategories SAL <https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL> : 09:37 wm-bot: <lucaswerkmeister> deployed 83d01bb93b (background UI > improvements) > You can also watch this in action in a demo screencast <https://commons.wikimedia.org/wiki/File:Toolforge_dologmsg_QuickCategories_…>. Of course, you can still log directly to #wikimedia-cloud, but I hope that doing it from the same shell where you also do your work will be more convenient :) Cheers, Lucas -- Lucas Werkmeister Full Stack Developer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

[Cloud-announce] [Toolforge] Node.js 10 and Python 3.5 types now available for use by Kubernetes webservices
by Bryan Davis 19 May '19

19 May '19

Good news from the Wikimedia Hackathon in Prague! We now have some newer language runtimes for Node.js and Python3 available for Kubernetes webservices. These newer versions match the versions that were added for grid engine webservices when we upgraded to Debian Stretch. These new versions are available in parallel with the older Node.js 6.11 and Python 3.4 versions. This will be the pattern used in the future when we add all newer language runtime versions so that migrations are a bit easier for all existing users. The new type names are: * node10 * python3.5 == Node.js 10 == $ webservice --backend=kubernetes node10 shell Defaulting container name to interactive. Use 'kubectl describe pod/interactive -n bd808-test' to see all of the containers in this pod. If you don't see a command prompt, try pressing enter. $ nodejs --version v10.4.0 $ npm --version 6.5.0 $ logout Session ended, resume using 'kubectl attach interactive -c interactive -i -t' command when the pod is running Pod stopped. Session cannot be resumed. == Python 3.5 == $ webservice --backend=kubernetes python3.5 shell Defaulting container name to interactive. Use 'kubectl describe pod/interactive -n bd808-test' to see all of the containers in this pod. If you don't see a command prompt, try pressing enter. $ python3 --version Python 3.5.3 $ logout Session ended, resume using 'kubectl attach interactive -c interactive -i -t' command when the pod is running Pod stopped. Session cannot be resumed. Bryan, on behalf of the Toolforge admin team -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Manager, Technical Engagement Boise, ID USA irc: bd808 v:415.839.6885 x6855 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 0

Electric maintenance on 2019-05-16
by Arturo Borrero Gonzalez 16 May '19

16 May '19

Hi! on 2019-05-16 13:00 UTC there will be a maintenance operation in one of the Wikimedia Foundation datacenter racks that affects 2 of our servers running virtual machines [0]. There is a risk that this maintenance operation can result in power loss of the servers, affecting the virtual machines running on it. However, there is no way to know for sure if there will be any outage at all. If you are an admin of any of the VMs in the list and you want the VM to be reallocated into other servers previous to the operation, please get in touch with us as soon as possible. Remember that, right now, reallocating the VM to other server means shutting down the VM briefly. Here is a list of affected virtual machines: cloudvirt1028.eqiad.wmnet: af-puppetdb01.automation-framework.eqiad.wmflabs bastion-eqiad1-02.bastion.eqiad.wmflabs fridolin.catgraph.eqiad.wmflabs cloud-puppetmaster-02.cloudinfra.eqiad.wmflabs cloudstore-dev-01.cloudstore.eqiad.wmflabs commtech-nsfw.commtech.eqiad.wmflabs clm-test-01.community-labs-monitoring.eqiad.wmflabs cyberbot-exec-iabot-01.cyberbot.eqiad.wmflabs deployment-db05.deployment-prep.eqiad.wmflabs deployment-memc05.deployment-prep.eqiad.wmflabs deployment-sca01.deployment-prep.eqiad.wmflabs deployment-pdfrender02.deployment-prep.eqiad.wmflabs ign.ign2commons.eqiad.wmflabs integration-slave-docker-1050.integration.eqiad.wmflabs integration-castor03.integration.eqiad.wmflabs api.openocr.eqiad.wmflabs osmit-umap.osmit.eqiad.wmflabs builder-envoy.packaging.eqiad.wmflabs jmm-buster.puppet.eqiad.wmflabs a11y.reading-web-staging.eqiad.wmflabs adhoc-utils01.security-tools.eqiad.wmflabs util-abogott-stretch.testlabs.eqiad.wmflabs canary1028-01.testlabs.eqiad.wmflabs stretch.thumbor.eqiad.wmflabs tools-worker-1023.tools.eqiad.wmflabs tools-proxy-04.tools.eqiad.wmflabs tools-docker-builder-06.tools.eqiad.wmflabs tools-sgewebgrid-generic-0904.tools.eqiad.wmflabs tools-sgeexec-0942.tools.eqiad.wmflabs tools-sgeexec-0941.tools.eqiad.wmflabs tools-sgeexec-0940.tools.eqiad.wmflabs tools-sgeexec-0939.tools.eqiad.wmflabs tools-sgeexec-0937.tools.eqiad.wmflabs tools-sgeexec-0929.tools.eqiad.wmflabs tools-sgeexec-0921.tools.eqiad.wmflabs tools-sgeexec-0920.tools.eqiad.wmflabs tools-sgeexec-0911.tools.eqiad.wmflabs tools-sgeexec-0909.tools.eqiad.wmflabs toolsbeta-proxy-01.toolsbeta.eqiad.wmflabs vconverter-instance.videowiki.eqiad.wmflabs perfbot.webperf.eqiad.wmflabs wdhqs-1.wikidata-history-query-service.eqiad.wmflabs cloudvirt1014.eqiad.wmnet: commonsarchive-prod.commonsarchive.eqiad.wmflabs deployment-imagescaler03.deployment-prep.eqiad.wmflabs dumps-5.dumps.eqiad.wmflabs dumps-4.dumps.eqiad.wmflabs incubator-mw.incubator.eqiad.wmflabs webperformance.integration.eqiad.wmflabs saucelabs-01.integration.eqiad.wmflabs integration-puppetmaster01.integration.eqiad.wmflabs maps-puppetmaster.maps.eqiad.wmflabs maps-wma.maps.eqiad.wmflabs mwoffliner3.mwoffliner.eqiad.wmflabs mwoffliner1.mwoffliner.eqiad.wmflabs phlogiston-5.phlogiston.eqiad.wmflabs discovery-testing-01.shiny-r.eqiad.wmflabs snuggle-enwiki-01.snuggle.eqiad.wmflabs canary-1014-01.testlabs.eqiad.wmflabs tools-sgeexec-0901.tools.eqiad.wmflabs wdqs-test.wikidata-query.eqiad.wmflabs Toolforge won't be affected by this operation. You can read more details about the datacenter operation itself in phabricator [1]. Sorry for the short notice, regards. [0] Cloud Services: reallocate workload from rack B5-eqiad https://phabricator.wikimedia.org/T223148 [1] Install new PDUs into b5-eqiad https://phabricator.wikimedia.org/T223126 -- Arturo Borrero Gonzalez Operations Engineer / Wikimedia Cloud Services Wikimedia Foundation

2 2

Today: Technical Advice IRC Meeting (special topic: wb_terms migration)
by Michael Schönitzer 15 May '19

15 May '19

Reminder: Technical Advice IRC meeting this week **Today 3-4 pm UTC** on #wikimedia-tech. Question can be asked in English, Arabic & Spanish! Today and next week we have the special Topic "Preparing Wikidata Tools for the wb_terms migration". If you have any tool that might be affected by this change feel free to join and ask! Other topics are welcome too. The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud May 2019