- Cloud - lists.wikimedia.org

Who is ${user_agent} ?
by Platonides 20 Dec '21

20 Dec '21

Hello everyone We have noticed that since Septembrer 2020, our WLM-related site www.wikilm.es has been receiving a couple of meta=siteinfo api queries per day from Cloud VPS NAT egress 185.15.56.1 with a user agent of... ${user_agent} (yes, literally that. Probably a case of single quotes when double were needed) It's probably some kind of Wiki Loves Monuments statistics tool. I have looked on some suspects with no luck. Anyone has an idea which tool may be doing that? Regards

2 1

Save the date! Coolest Tool Award 2021: Friday, January 14th, 17:00 UTC
by Andre Klapper 17 Dec '21

17 Dec '21

Hi everyone! The second edition of the Coolest Tool Award https://meta.wikimedia.org/wiki/Coolest_Tool_Award will happen online on Friday 14 January 2022 at 17:00 UTC! See https://zonestamp.toolforge.org/1642179615 for your timezone. The awarded tools will be showcased in a virtual event, with broadcasted video and chat channels for socializing. We will send more details and links soon. Save the date, and join us celebrating the great work volunteer developers do for the Wikimedia communities. We hope to see you there! andre, for the Coolest Tool Academy 2021 -- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

1 0

Eliminating Debian Stretch in Cloud VPS
by Tim Moody 16 Dec '21

16 Dec '21

What's the best way to do this? I noticed a rebuild instance option and tried it to upgrade to bullseye. It processed for awhile, rebooted, and says the instance based on the bullseye image, but I don't see that it upgraded anything. Tim

2 1

Looking for new maintainer for packagist-mirror.wmcloud.org
by Kunal Mehta 10 Dec '21

10 Dec '21

Hi, <https://packagist-mirror.wmcloud.org/> is exactly what it sounds like, a mirror of the metadata on packagist.org. It's very simple, just a systemd timer and some Apache config. Since setting it up 2 or 3 years ago I've barely had to touch it. And based on access logs, some people are actually using it (though we never switched CI over to it, as the idea/goal once was). But after being reminded about it by the current Cloud VPS cleanup, I'm not interested in running this anymore. packagist.org is actually open source (*looks at npmjs.com in sadness*) and hasn't had any reliability issues recently AFAIR. There's also a bit of work to do, the VM needs to be upgraded from stretch, and it should be switched over to the new composer/mirror script from upstream. So if someone would like to take it over, comment on <https://phabricator.wikimedia.org/T296968> and I'm happy to add you and explain how it works. Otherwise please consider this notice that I intend to shut it down at the end of the year. -- Legoktm

1 0

Source command not working with zsh
by Huji Lee 05 Dec '21

05 Dec '21

Hi all, Starting Nov 7, a number of the jobs I would run through Toolforge grid have stopped working. Each job consists of a .sh file like this <https://github.com/PersianWikipedia/fawikibot/blob/master/HujiBot/grid/jobs…> on the first line of which I use the source command to activate a python virtual environment. When I run source by hand, subsequent lines work. But when I call the .sh file and it tries to run the source command, I get a "source: not found" message, the virtual environment does not get activated and indeed running *which python* returns */usr/bin/python* which is bad. All my scripts depend on pip packages that are installed in the virtual env and not available with the system python. The main thing I did on Nov 7 was to add a line at the end of my too's account's .bash_profile as below: exec zsh This is because when I manually log into toolforge, I would like zsh to be my shell, and since tool accounts don't support chsh, I thought executing zsh directly from bash would be okay. But apparently, that now breaks the source command somehow. So I wonder: (a) Is there a way to properly change the default shell of tool accounts? (b) Is there a way to make *source* work under zsh? Importantly, I know the problem is with *exec zsh* because once I removed it and logged out and back in, all scripts worked correctly. Thanks, Huji

4 20

[Cloud-announce] Cloud VPS users, please claim your projects (and, introducing Komla)
by Andrew Bogott 02 Dec '21

02 Dec '21

Hello cloud-vps users! It's time for our annual cleanup of unused projects and resources. Our new developer advocate Komla Sapaty will be guiding this process; please respond promptly to his emails and do your best to make him feel welcome! Every year or so the Cloud Services team tries to identify and clean up unused projects and VMs. We do this via an opt-in process: anyone can mark a project as 'in use,' and that project will be preserved for another year. I've created a wiki page that lists all existing projects, here: https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2021_Purge If you are a VPS user, please visit that page and mark any projects that you use as {{Used}}. Note that it's not necessary for you to be a project admin to mark something -- if you know that you're currently using a resource and want to keep using it, go ahead and mark it accordingly. If you /are/ a project admin, please take a moment to mark which VMs are or aren't used in your projects. When February arrives, I will shut down and begin the process of reclaiming resources from unused projects. If you think you use a VPS project but aren't sure which, I encourage you to poke around on https://tools.wmflabs.org/openstack-browser/ to see what looks familiar. Worst case, just email cloud(a)lists.wikimedia.org with a description of your use case and we'll sort it out there. Exclusive toolforge users are free to ignore this email and future related things. Thank you! -Andrew and the WMCS team _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Search engine indexing of webapplication
by Finn Årup Nielsen 23 Nov '21

23 Nov '21

Dear Toolforge cloud people, I am running the Scholia web application on Toolforge and interested in have some of the pages indexed by search engines. We have '<meta name="robots" content="index, nofollow">' which should index but not crawl the Scholia website. We have 3 kinds of content on the webpages generated by Scholia: 1) "Static" content generated from Flask jinja2 templates. These gets indexed (but not that much). 2) Dynamic jQuery-based content based on the Wikidata API service. This does not seem to get indexed by some search engines. 3) Dynamic Wikidata Query Serviece-based content. This does not get indexed. I can understand 1) and 3), but not 2). https://query.wikidata.org/robots.txt is blocking bots request for 3), but as far as I can see https://www.wikidata.org/robots.txt does not block Wikidata API requests for 2). On a webpage on the public web, I have a link to https://scholia.toolforge.org/author/Q20980928, so I would think that that Scholia page would be indexed, and that the h1 tag that is set via the Wikidata API would be indexed. As far as I can determine the page is indexed at Bing and Quant, but not Duckduckgo and not Google. I am wondering whether there is anyone that can explain the discrepancy? As far as I understand Google does indeed index jQuery javascript-generated content. Should we refrain from having bots getting into Scholia and define a restrictive robots.txt to avoid burdening the Toolforge infrastructure too much? In Scholia, we at the moment have a pull request that implement serverside calls to the Wikidata Query Service to generate some metadata for the search engines that can be reached without hitting the WDQS robots.txt restriction. I have been reluctant to merge that pull request due to the extra load on the Toolforge as well as the extra time the request takes blocking the Scholia web application. We have around 100'000 requests per day according to Toolviews, - how much bot activity I do not know. I am wondering whether there is anyone who can give us advise here? best regards Finn Årup Nielsen

1 0

Shared Rust toolchain available on Toolforge
by Kunal Mehta 04 Nov '21

04 Nov '21

Hi, I have setup a "rustup" tool on Toolforge that contains the latest Rust toolchain for use by other tools so it doesn't need to be installed individually in each tool. Documentation on how to use it in your tool is available at <https://wikitech.wikimedia.org/wiki/Tool:Rustup>, all you have to do is add two lines to your tool's ~/.profile. If you need an older version or some other components, let me know and we can install it. Hopefully this makes developing Rust tools just a bit easier :) -- Legoktm

1 0

qsub waiting time on Toolforge
by Anis Elebiary 02 Nov '21

02 Nov '21

Hi all, I have a python script that creates a tsv for the redirect table. I submitted the job through qsub and it's been waiting for more than a day now. Is it normal for the waiting time to be this long ? And, if there's a better way to do the aforementioned task, please let me know. Thank you in advance. Best, Anis

4 8

[Cloud-announce] 2021-11-02: Cloud VPS network outage
by Arturo Borrero Gonzalez 02 Nov '21

02 Nov '21

Hi, Today 2021-11-02 we had a severe network outage on Cloud VPS and Toolforge. Several network connections were affected from 11:40 UTC to 13:20 UTC (1h40m duration). As of this writing the problem has been corrected. Detailed information can be seen in Phabricator: https://phabricator.wikimedia.org/T294853 Sorry for the inconvenience. regards. -- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud