- Cloud-announce - lists.wikimedia.org

Neutron region migrations starting next Friday
by Andrew Bogott 19 Oct '18

19 Oct '18

Beginning next week, the cloud team will start migrating projects to Neutron[1] in earnest. I will attempt to reach out individually to affected project admins as well, but here is the upcoming migration schedule: Friday, 2018-10-19: analytics Monday, 2018-10-22: antiharassment, catgraph, codereview, cvn Tuesday, 2018-10-23: cyberbot, fastcci, general-k8s Wednesday, 2018-10-24: huggle, logging, mobile, mwstake Thursday, 2018-10-25: planet, pluggableauth, privpol-captcha, qna Friday, 2018-10-26: reading-web-staging, suggestbot, test-twemproxy, wikibase-registry, wikibrain, wikidata-primary-sources-tool That seemingly-arbitrary selection of projects is chosen to free up some hardware space and make room for additional migrations in the future. Even if you don't see your project name in that list, please keep an eye on your inbox :) - Andrew + the WMCS team [1] https://phabricator.wikimedia.org/phame/post/view/120/neutron_is_here/

1 1

Network maintenance Thursday, 2018-10-04 at 16:00 UTC
by Andrew Bogott 04 Oct '18

04 Oct '18

Routine network upgrades are scheduled for Thursday which may result in brief WMCS service interruptions. In particular, Wikitech and Horizon may stop working, and instance creation/deletion/updating may briefly fail. The network engineers have reserved a two-hour window beginning at 16:00 UTC (9 AM Pacific time). Their worst case scenario is 30 minutes of downtime but if all goes well the interruption should not be noticeable, just the time to unplug a cable from one switch and plug it into another.

1 1

Ubuntu deprecation plans
by Arturo Borrero Gonzalez 28 Sep '18

28 Sep '18

Hi! We would like to share some information regarding Wikimedia Cloud Services plans for deprecating Ubuntu, specially Trusty. Ubuntu Trusty's end-of-life is April 2019 and the WMF decided to consolidate in a single operating system, which is Debian. In Cloud VPS, projects containing Ubuntu virtual machine instances have been contacted by means of a Phabricator task. Toolforge users aren't affected by this right now, because Toolforge itself is currently running Trusty. But we are already working on the next, Debian-based, Toolforge version. All this information, more details (and timelines), can be found on this Wikitech page: https://wikitech.wikimedia.org/wiki/News/Trusty_deprecation Please, let us know any question or doubt you may have. Best regards.

1 0

NFS maintenance this Thursday 2018-09-27
by Brooke Storm 25 Sep '18

25 Sep '18

At 6pm UTC, Thursday 2018-09-27, we’ll be taking the primary tools/project NFS system offline for a brief moment to attempt some upgrades, which should cause load to climb a bit and then settle. This should be a relatively low-impact change and has been thoroughly tested in a separate environment, but if anything goes wrong with the process, corrective action may cause a few minutes of problems. Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm(a)wikimedia.org IRC: bstorm_

1 0

Very brief NFS maintenance today 2018-09-17
by Brooke Storm 17 Sep '18

17 Sep '18

At 6pm UTC, we will be briefly resetting the main NFS server for tools and projects to upgrade the tooling around NFS there. This should cause a brief climb in processes waiting for IO and then that will settle out. If things don’t operate as expected, rolling back the change could take a few minutes, but that seems unlikely. Please let us know if there are prolonged issues in the #wikipedia-cloud channel. Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm(a)wikimedia.org IRC: bstorm_

1 0

57 VPS projects are candidates for deletion
by Andrew Bogott 17 Sep '18

17 Sep '18

There are currently 57 unclaimed projects on https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge. I will start shutting down unclaimed projects at the beginning of next month, and those projects will be left behind in the future network migration[1] and, eventually, deleted. If you see anything in this list that you care about, please indicate that the project is in use on the Wikitech page. Here are the 57 projects that are currently unmarked: aicaptcha bots bstorm-test chicotestproject ci-staging collection-alt-renderer contributors deep-learning-services discourse-wam download etytree fastcci glampipe globaleducation hat-imagescalers hound iiifls1 ircd jupyter maps maps-team matrix mcr-dev mediahandler-tests mediawiki-docker multimedia mw-api-testing mw-extension-ids newsletter nonfreewiki openocr orig pagemigration piwik project-smtp rdfiodev reading-lists redirects reportcard sentry services services-testbed t136871 test-twemproxy tor traffic wcdo wdq-mm wikibrain wikidata-page-banner wikidata-primary-sources-tool wikidumpparse wikifactmine wikimetrics wikisource-tools wikistream wlmjudging [1] https://phabricator.wikimedia.org/phame/post/view/112/neutron_is_finally_co…

1 0

Datacenter switchover on September 11
by James Hare 10 Sep '18

10 Sep '18

All, As you may have read about in other venues, the Wikimedia Foundation will be conducting a switchover of datacenters on September 11 and 12, with the switch back to occur on October 10 and 11. During the switchover and switch-back events, the wikis will be in read-only mode. Cloud VPS and Toolforge are not affected by these switchovers. However, if you run bots or scripts that make edits to the wikis, they may be unable to make edits during this time. This should appear to your bots as the standard read-only mode which occurs unplanned on the production wikis from time to time. Please let me know if you have any questions. ---- James Hare Associate Product Manager Wikimedia Foundation https://wikimediafoundation.org

1 0

NFS maintenance next week
by Brooke Storm 29 Aug '18

29 Aug '18

We will be performing some updates (including reboots) on the tools NFS servers for Toolforge and CloudVPS instances that use project NFS (with the exception of dumps) at 1500 UTC on Wed Aug 29. The maintenance window will be two hours and more than one NFS server failover is expected during that time. This could cause some temporary impact to performance and load on the various connected servers during failovers. Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm(a)wikimedia.org <mailto:bstorm@wikimedia.org> IRC: bstorm_

1 3

VPS users, please claim your projects (second notice)
by Andrew Bogott 27 Aug '18

27 Aug '18

In an attempt to identify abandoned VPS projects, I've created a wiki page that lists all existing projects, here: https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge Currently 85 projects[2] on that list are unclaimed. If you are a VPS user, please visit that page and mark any projects that you use as {{Used}}. Note that it's not necessary for you to be a project admin to mark something -- if you know that you're currently using a resource and want to keep using it, go ahead and mark it accordingly. If you /are/ a project admin, please take a moment to mark which VMs are or aren't used in your projects. When October arrives, I will shut down and begin the process of reclaiming unused projects. If you think you use a VPS project but aren't sure which, I encourage you to poke around on https://tools.wmflabs.org/openstack-browser/ to see what looks familiar. You can also log in to http://horizon.wikimedia.org which will provide you with a handy menu of projects that you are currently a member of. Thank you! -Andrew and the WMCS team [1] https://phabricator.wikimedia.org/phame/post/view/112/neutron_is_finally_co… [2] Here, for good measure, is that list. Every one of these projects is currently a candidate for deletion: aicaptcha analytics bots chicotestproject ci-staging codereview collection-alt-renderer commonsarchive community-labs-monitoring contributors dashiki deep-learning-services discourse discourse-wam download etytree fastcci getstarted glampipe globaleducation hat-imagescalers hound iiab iiifls1 ircd jupyter kubernetes-testing maps maps-team math matrix mcr-dev mediahandler-tests mediawiki-docker multimedia mw-api-testing mw-extension-ids mwfuzz mwoffliner newsletter nonfreewiki openocr ores orig otrs pagemigration paws phlogiston piwik project-smtp rcm rdfiodev reading-lists reading-web-staging recommendation-api redirects reportcard sciencesource sentry services services-testbed t136871 test-twemproxy thumbor tor traffic twl utrs video wcdo wdq-mm wikibase-nearest-neighbors wikibrain wikidata-federation wikidata-page-banner wikidata-primary-sources-tool wikidumpparse wikifactmine wikimetrics wikisource-tools wikistream wikitolearn-dev wildcat wlmjudging wmf-research-tools

1 0

VPS users, please claim your projects
by Andrew Bogott 13 Aug '18

13 Aug '18

We're drawing close to a painful migration event[1], during which we will (probably) have to copy VMs between hosts one project at a time, largely by hand. For that reason, I'm feeling even stingier than usual about preserving unused and/or abandoned projects and instances. It's been a couple of years since we last did this, and I'm sure there are some newly forgotten projects in the cloud. So, I've created a wiki page the lists all existing projects, here: https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge If you are a VPS user, please visit that page and mark any projects that you use as {{Used}}. Note that it's not necessary for you to be a project admin to mark something -- if you know that you're currently using a resource and want to keep using it, go ahead and mark it accordingly. If you /are/ a project admin, please take a moment to mark which VMs are or aren't used in your projects. When October arrives, I will shut down and begin the process of reclaiming unused projects. If you think you use a VPS project but aren't sure which, I encourage you to poke around on https://tools.wmflabs.org/openstack-browser/ to see what looks familiar. Worst case, just email cloud(a)lists.wikimedia.org with a description of your use case and we'll sort it out there. Exclusive toolforge users are free to ignore this task. Thank you! -Andrew and WMCS team [1] The eqiad->eqiad1 migration, also known as the 'nova-network to neutron' migration which I hope to write more about soon. It won't happen by surprise, I promise.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud-announce