Thanks Galder :-) - answers are below:
On Thu, Oct 14, 2021 at 9:38 PM Galder Gonzalez Larrañaga <
galder158(a)hotmail.com> wrote:
> Thanks, Brigit, for this hub, it is great to have it! I have tried and
> can't find any way to look for tools that are not nominated as "Coolest
> Tool Award" besides looking for name. Is there a way for searching by
> categories?
>
Today you can try searching for various keywords to find tools in the
catalog (i.e. "Wikidata", "image", "editor", "template" ...). We're
interested in adding support for the community to organize tools in
categories/based on use cases in the future. There are some notes from past
Design Research on that in the Data Model documentation. [0]
Please feel welcome to comment on the talk page if you have further
questions or ideas! [1]
Birgit
[0] https://meta.wikimedia.org/wiki/Toolhub/Data_model#Tool_use_cases
[1] https://meta.wikimedia.org/wiki/Talk:Toolhub
>
> Thanks
> Galder
> ------------------------------
> *From:* Birgit Müller <bmueller(a)wikimedia.org>
> *Sent:* Thursday, October 14, 2021 4:58 PM
> *To:* wikitech-l <wikitech-l(a)lists.wikimedia.org>;
> wikimedia-l(a)lists.wikimedia.org <wikimedia-l(a)lists.wikimedia.org>;
> Wikimedia Cloud Services general discussion and support <
> cloud(a)lists.wikimedia.org>; wikidata(a)lists.wikimedia.org <
> wikidata(a)lists.wikimedia.org>; wikitech-ambassadors(a)lists.wikimedia.org <
> wikitech-ambassadors(a)lists.wikimedia.org>
> *Subject:* [Wikimedia-l] Toolhub 1.0 is launched! Discover software tools
> used at Wikimedia
>
>
> Hi All,
>
> We are happy to announce the launch of Toolhub
> <https://toolhub.wikimedia.org/> – a community-authored catalogue that
> aims to make software tools
> <https://meta.wikimedia.org/wiki/Toolhub#What_is_a_%22tool%22?> used in
> the Wikimedia movement discoverable to everyone.
>
> Community developed tools – including web applications, bots, gadgets,
> user scripts, lua modules, and more – play a significant role in the
> Wikimedia projects. These software applications address a wide range of use
> cases including finding bad faith edits and other content curation, bulk
> editing, collecting statistical information, creating special citations,
> and much more. About ⅓ of all edits are made by bots and tools. In
> addition, semi-automated edits are helped by user scripts, gadgets, and
> other editing assistance tools that run from the user's local computer or
> directly inside the wikis. There are thousands of tools available, but how
> can you find them?
>
> With Toolhub, you can document and find tools
> <https://meta.wikimedia.org/wiki/Toolhub>, promote their use in your wiki
> community, and help improve them by contributing data. You can create and
> share lists of tools relevant to your work - for example, for GLAM tools,
> or for wiki projects such as Women in Red.
>
> This first release provides a core set of functionalities
> <https://meta.wikimedia.org/wiki/Toolhub/Roadmap>, and contains an
> initial data set of about 1500 tools. Most of the initial tools in the
> catalog are imported from the same data files developers have created for Hay's
> Directory <https://hay.toolforge.org/directory/> which has been a major
> inspiration for Toolhub.
>
> Toolhub serves developers and users of tools alike. It is part of our
> efforts to improve the infrastructure and services for technical
> contributors, captured under one of Technology’s top level objectives in
> the FY 2020-2021 and 2021-2022 annual plans: Tech Community Building
> <https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/ERF_OKR:_T…>.
> We hope to continue conversations with developers and users of tools, plan
> to improve Toolhub, and to further expand the functionality.
>
> A collaborative system and open developer platform
>
> Toolhub is built as an API driven platform that makes it possible to
> extend and remix the catalogue, and to make collecting and reusing
> information about tools as open and collaborative as we can. Everything
> that can be done interactively with the Toolhub website can also be done
> remotely through the API. We would love to hear from technical
> contributors interested in using the Toolhub API
> <https://meta.wikimedia.org/wiki/Talk:Toolhub#API-use> to build new tools
> that make new ways to add or consume information from Toolhub's catalog.
>
> Our decision record
> <https://meta.wikimedia.org/wiki/Toolhub/Decision_record> and weekly
> progress reports
> <https://meta.wikimedia.org/wiki/Toolhub/Progress_reports> on Meta
> provide more insights in technical implementation details and decisions
> made throughout the development process. The Toolhub/About page
> <https://meta.wikimedia.org/wiki/Toolhub/About> provides information on
> project origin, research, use cases, data model, and roadmap. This recording
> from a lightning talk at ‘21 Wikimania
> <https://www.youtube.com/watch?v=J2hNm7bKjDo> gives an overview of the
> main aspects in 10 minutes.
>
> Thank you <3
>
> This project wouldn’t have been possible without the support, knowledge,
> ideas and prior work of many. One of the nicest side-effects of a release
> is that it’s a great opportunity to thank folks for their time and
> contributions :-)
>
>
> -
>
> Husky <https://meta.wikimedia.org/wiki/User:Husky>, whose Hay's
> Directory <https://hay.toolforge.org/directory/> provided the
> foundation for the data model used by Toolhub and inspired some of its
> features.
> -
>
> Harej <https://meta.wikimedia.org/wiki/User:Harej>, for his invaluable
> contributions in the early stages of the Toolhub project.
> -
>
> Our 'advisory board' - Giuseppe
> <https://meta.wikimedia.org/wiki/User:GLavagetto_(WMF)> (SRE), Risker
> <https://meta.wikimedia.org/wiki/User:Risker> (editor, admin), Reedy
> <https://meta.wikimedia.org/wiki/User:Reedy_(WMF)> (Security), Keegan
> <https://meta.wikimedia.org/wiki/User:Keegan_(WMF)> (Community
> Relations), and Eran
> <https://meta.wikimedia.org/wiki/User:%D7%A2%D7%A8%D7%9F> (volunteer
> developer & RTL expert) for providing their perspectives on key questions
> throughout the development process.
> -
>
> Giuseppe, Kunal <https://meta.wikimedia.org/wiki/User:Legoktm>, Manuel
> <https://meta.wikimedia.org/wiki/User:MArostegui_(WMF)>, Effie
> <https://meta.wikimedia.org/wiki/User:EMouzeli_(WMF)>, Cole
> <https://meta.wikimedia.org/wiki/User:CWhite_(WMF)> and Emanuele
> <https://meta.wikimedia.org/wiki/User:ERocca_(WMF)> from SRE and
> Majavah <https://en.wikipedia.org/wiki/User:Majavah> for their help on
> finding and resolving deployment issues.
> -
>
> Dan <https://www.mediawiki.org/wiki/User:Dzduvall> and Jeena
> <https://www.mediawiki.org/wiki/User:JHuneidi_(WMF)> from Release
> Engineering for help with build tooling.
> -
>
> Guillaume <https://www.mediawiki.org/wiki/User:GLederrey_(WMF)> and
> the rest of the Search Platform team for supporting our search index needs.
> -
>
> Manuel for supporting our database needs.
> -
>
> Niklas <https://meta.wikimedia.org/wiki/User:Nlaxstrom-WMF> and the
> whole translatewiki.net community for help with localization and
> internationalization.
> -
>
> Rita <https://meta.wikimedia.org/wiki/User:RHo_(WMF)>, Olga
> <https://meta.wikimedia.org/wiki/User:OTichonova_(WMF)>, Alex
> <https://meta.wikimedia.org/wiki/User:AHollender_(WMF)>, and Matthew
> <https://meta.wikimedia.org/wiki/User:MWilliams_(WMF)> from the Product
> Design <https://www.mediawiki.org/wiki/Design> team for their feedback
> on the Toolhub user interface.
> -
>
> Scott <https://www.mediawiki.org/wiki/User:SBassett_(WMF)> from the
> Security team for our security readiness review.
> -
>
> Amire <https://en.wikipedia.org/wiki/User:Amire80>, Kunal, Eran,
> Reedy, and Dan for contributing code to the project.
> -
>
> Ricordisamoa <https://meta.wikimedia.org/wiki/User:Ricordisamoa>, Quim
> <https://meta.wikimedia.org/wiki/User:Qgil-WMF>, and the people
> participating in conversations on wikitech-l
> <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
> for T115650 <https://phabricator.wikimedia.org/T115650> which inspired
> this whole project.
> -
>
> Finally, a huge thanks to all the folks who gave input and feedback on
> the talk page, in Phabricator, and at sessions - this is really
> appreciated!
>
>
> We hope that this new resource will be fun to explore, inspire you with
> new ideas, and ultimately be useful for your work.
>
> Feedback, bug reports, ideas and questions are more than welcome on the talk
> page of the project <https://meta.wikimedia.org/wiki/Talk:Toolhub>, or in
> Phabricator <https://phabricator.wikimedia.org/project/board/3224/>. Bryan
> (tech lead) & Seve (our new Product Manager) will be there to chat with
> interested folks and help with any questions. We are looking forward to
> evolving this project step-by-step and jointly with everyone!
>
> Birgit – on behalf of Technical Engagement & our Toolhub project team
>
>
> --
> Birgit Müller (she/her)
> Director of Technical Engagement
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines
> at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> Public archives at
> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org…
> To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org
--
Birgit Müller (she/her)
Director of Technical Engagement
Wikimedia Foundation <https://wikimediafoundation.org/>
A long time coming! The interface looks amazing and seems very well thought
out. It's nice to have an official, single go-to for all things tools, user
scripts, gadgets, etc. Huge thanks and kudos to everyone involved!
~ MA
On Thu, Oct 14, 2021 at 3:37 PM Galder Gonzalez Larrañaga <
galder158(a)hotmail.com> wrote:
> Thanks, Brigit, for this hub, it is great to have it! I have tried and
> can't find any way to look for tools that are not nominated as "Coolest
> Tool Award" besides looking for name. Is there a way for searching by
> categories?
>
> Thanks
> Galder
> ------------------------------
> *From:* Birgit Müller <bmueller(a)wikimedia.org>
> *Sent:* Thursday, October 14, 2021 4:58 PM
> *To:* wikitech-l <wikitech-l(a)lists.wikimedia.org>;
> wikimedia-l(a)lists.wikimedia.org <wikimedia-l(a)lists.wikimedia.org>;
> Wikimedia Cloud Services general discussion and support <
> cloud(a)lists.wikimedia.org>; wikidata(a)lists.wikimedia.org <
> wikidata(a)lists.wikimedia.org>; wikitech-ambassadors(a)lists.wikimedia.org <
> wikitech-ambassadors(a)lists.wikimedia.org>
> *Subject:* [Wikimedia-l] Toolhub 1.0 is launched! Discover software tools
> used at Wikimedia
>
>
> Hi All,
>
> We are happy to announce the launch of Toolhub
> <https://toolhub.wikimedia.org/> – a community-authored catalogue that
> aims to make software tools
> <https://meta.wikimedia.org/wiki/Toolhub#What_is_a_%22tool%22?> used in
> the Wikimedia movement discoverable to everyone.
>
> Community developed tools – including web applications, bots, gadgets,
> user scripts, lua modules, and more – play a significant role in the
> Wikimedia projects. These software applications address a wide range of use
> cases including finding bad faith edits and other content curation, bulk
> editing, collecting statistical information, creating special citations,
> and much more. About ⅓ of all edits are made by bots and tools. In
> addition, semi-automated edits are helped by user scripts, gadgets, and
> other editing assistance tools that run from the user's local computer or
> directly inside the wikis. There are thousands of tools available, but how
> can you find them?
>
> With Toolhub, you can document and find tools
> <https://meta.wikimedia.org/wiki/Toolhub>, promote their use in your wiki
> community, and help improve them by contributing data. You can create and
> share lists of tools relevant to your work - for example, for GLAM tools,
> or for wiki projects such as Women in Red.
>
> This first release provides a core set of functionalities
> <https://meta.wikimedia.org/wiki/Toolhub/Roadmap>, and contains an
> initial data set of about 1500 tools. Most of the initial tools in the
> catalog are imported from the same data files developers have created for Hay's
> Directory <https://hay.toolforge.org/directory/> which has been a major
> inspiration for Toolhub.
>
> Toolhub serves developers and users of tools alike. It is part of our
> efforts to improve the infrastructure and services for technical
> contributors, captured under one of Technology’s top level objectives in
> the FY 2020-2021 and 2021-2022 annual plans: Tech Community Building
> <https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/ERF_OKR:_T…>.
> We hope to continue conversations with developers and users of tools, plan
> to improve Toolhub, and to further expand the functionality.
>
> A collaborative system and open developer platform
>
> Toolhub is built as an API driven platform that makes it possible to
> extend and remix the catalogue, and to make collecting and reusing
> information about tools as open and collaborative as we can. Everything
> that can be done interactively with the Toolhub website can also be done
> remotely through the API. We would love to hear from technical
> contributors interested in using the Toolhub API
> <https://meta.wikimedia.org/wiki/Talk:Toolhub#API-use> to build new tools
> that make new ways to add or consume information from Toolhub's catalog.
>
> Our decision record
> <https://meta.wikimedia.org/wiki/Toolhub/Decision_record> and weekly
> progress reports
> <https://meta.wikimedia.org/wiki/Toolhub/Progress_reports> on Meta
> provide more insights in technical implementation details and decisions
> made throughout the development process. The Toolhub/About page
> <https://meta.wikimedia.org/wiki/Toolhub/About> provides information on
> project origin, research, use cases, data model, and roadmap. This recording
> from a lightning talk at ‘21 Wikimania
> <https://www.youtube.com/watch?v=J2hNm7bKjDo> gives an overview of the
> main aspects in 10 minutes.
>
> Thank you <3
>
> This project wouldn’t have been possible without the support, knowledge,
> ideas and prior work of many. One of the nicest side-effects of a release
> is that it’s a great opportunity to thank folks for their time and
> contributions :-)
>
>
> -
>
> Husky <https://meta.wikimedia.org/wiki/User:Husky>, whose Hay's
> Directory <https://hay.toolforge.org/directory/> provided the
> foundation for the data model used by Toolhub and inspired some of its
> features.
> -
>
> Harej <https://meta.wikimedia.org/wiki/User:Harej>, for his invaluable
> contributions in the early stages of the Toolhub project.
> -
>
> Our 'advisory board' - Giuseppe
> <https://meta.wikimedia.org/wiki/User:GLavagetto_(WMF)> (SRE), Risker
> <https://meta.wikimedia.org/wiki/User:Risker> (editor, admin), Reedy
> <https://meta.wikimedia.org/wiki/User:Reedy_(WMF)> (Security), Keegan
> <https://meta.wikimedia.org/wiki/User:Keegan_(WMF)> (Community
> Relations), and Eran
> <https://meta.wikimedia.org/wiki/User:%D7%A2%D7%A8%D7%9F> (volunteer
> developer & RTL expert) for providing their perspectives on key questions
> throughout the development process.
> -
>
> Giuseppe, Kunal <https://meta.wikimedia.org/wiki/User:Legoktm>, Manuel
> <https://meta.wikimedia.org/wiki/User:MArostegui_(WMF)>, Effie
> <https://meta.wikimedia.org/wiki/User:EMouzeli_(WMF)>, Cole
> <https://meta.wikimedia.org/wiki/User:CWhite_(WMF)> and Emanuele
> <https://meta.wikimedia.org/wiki/User:ERocca_(WMF)> from SRE and
> Majavah <https://en.wikipedia.org/wiki/User:Majavah> for their help on
> finding and resolving deployment issues.
> -
>
> Dan <https://www.mediawiki.org/wiki/User:Dzduvall> and Jeena
> <https://www.mediawiki.org/wiki/User:JHuneidi_(WMF)> from Release
> Engineering for help with build tooling.
> -
>
> Guillaume <https://www.mediawiki.org/wiki/User:GLederrey_(WMF)> and
> the rest of the Search Platform team for supporting our search index needs.
> -
>
> Manuel for supporting our database needs.
> -
>
> Niklas <https://meta.wikimedia.org/wiki/User:Nlaxstrom-WMF> and the
> whole translatewiki.net community for help with localization and
> internationalization.
> -
>
> Rita <https://meta.wikimedia.org/wiki/User:RHo_(WMF)>, Olga
> <https://meta.wikimedia.org/wiki/User:OTichonova_(WMF)>, Alex
> <https://meta.wikimedia.org/wiki/User:AHollender_(WMF)>, and Matthew
> <https://meta.wikimedia.org/wiki/User:MWilliams_(WMF)> from the Product
> Design <https://www.mediawiki.org/wiki/Design> team for their feedback
> on the Toolhub user interface.
> -
>
> Scott <https://www.mediawiki.org/wiki/User:SBassett_(WMF)> from the
> Security team for our security readiness review.
> -
>
> Amire <https://en.wikipedia.org/wiki/User:Amire80>, Kunal, Eran,
> Reedy, and Dan for contributing code to the project.
> -
>
> Ricordisamoa <https://meta.wikimedia.org/wiki/User:Ricordisamoa>, Quim
> <https://meta.wikimedia.org/wiki/User:Qgil-WMF>, and the people
> participating in conversations on wikitech-l
> <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
> for T115650 <https://phabricator.wikimedia.org/T115650> which inspired
> this whole project.
> -
>
> Finally, a huge thanks to all the folks who gave input and feedback on
> the talk page, in Phabricator, and at sessions - this is really
> appreciated!
>
>
> We hope that this new resource will be fun to explore, inspire you with
> new ideas, and ultimately be useful for your work.
>
> Feedback, bug reports, ideas and questions are more than welcome on the talk
> page of the project <https://meta.wikimedia.org/wiki/Talk:Toolhub>, or in
> Phabricator <https://phabricator.wikimedia.org/project/board/3224/>. Bryan
> (tech lead) & Seve (our new Product Manager) will be there to chat with
> interested folks and help with any questions. We are looking forward to
> evolving this project step-by-step and jointly with everyone!
>
> Birgit – on behalf of Technical Engagement & our Toolhub project team
>
>
> --
> Birgit Müller (she/her)
> Director of Technical Engagement
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines
> at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> Public archives at
> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org…
> To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org
If you were running a Toolforge web tool in Kubernetes before the toollabs-webservice label changes were deployed on 2021-09-29 (https://sal.toolforge.org/tools?d=2021-09-29 <https://sal.toolforge.org/tools?d=2021-09-29>). You may need to run `webservice stop && webservice start` in order to ensure your replica sets have correct label expectations on them going forward. Otherwise you may find confusing states may happen when running webservice restart and similar commands.
When I backfilled the new labels, I missed that you cannot change the label matching rules in a deployment retroactively. I apologize for any inconvenience.
In summary: If you haven’t run a webservice stop since 2021-09-29 on your Kubernetes web service, it would be a good idea to stop and start your webservice now to prevent any confusing behavior from webservice in the future.
--
Brooke Storm
Staff SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Dear all,
It’s time for our third edition of the Coolest Tool Award!
Tools play an essential role at Wikimedia, and so do the many volunteer
developers who experiment with new ideas, develop & maintain local &
global solutions and enhance the experience for Wikimedia communities.
We’d like to invite you all to nominate your favorite & most used tools
and help us celebrate the people who create them!
As no one can possibly know all the cool tools out there, we’re looking
for some help and inspiration: please point us to the tools that you
think are great - for any reason you can think of!
Please go to https://meta.wikimedia.org/wiki/Coolest_Tool_Award
to recommend tools by October 27, 2021. You can nominate as many tools
as you want by filling out the form multiple times.
Thank you very much for your ideas & recommendation(s)!
The award is organized & selected by the Coolest Tool Academy 2021. We
plan to recognize the greatest tools in a variety of categories (for
examples, see last year’s categories). The award ceremony will take
place virtually again this year and we will provide more details soon
about the specific logistics and dates.
We will continue to spread the word over the next week, but if you get
the chance, please feel welcome to share this information with others
too!
Thanks :-)
Andre, for the Coolest Tool Academy 2021
--
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
Next Tuesday we will be upgrading Kubernetes on toolforge. As part of the
upgrade we will need to restart all pods. This will produce a brief
interruption in web services and other tools that use kubernetes. Assuming
your services are able to survive a restart, no action should be needed on
your part.
Thank you,
Michael + the WMCS team
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
TL;DR:
* Let's Encrypt [0] TLS certificates are "signed" by "root"
certificates to create a chain of trust
* The oldest "root" signing certificate for LE certs (DST Root CA X3)
expired on 2021-09-30 [1]
* Deprecated Toolforge Kubernetes containers only knew this root
certificate and not the newer root certificate (ISRG Root X1)
* Update your tool to a newer container to fix
We are starting to hear reports of tools that suddenly stopped working
on 2021-09-30. The common issue is accessing the APIs for Wikimedia
wikis.
The Wikimedia wikis use multiple TLS certificates issued by different
providers for redundancy and protection against a problem with a
single certificate provider. One of the certificate providers that we
use is Let's Encrypt (LE) [0]. LE certificates are themselves signed
by multiple "root" certificates to create a chain of trust that your
web browser or other TLS verifying software can trust. The oldest root
certificate (named "DST Root CA X3") used to sign the LE certificates
expired on 2021-09-30 [1]. Very old operating systems and some
compiled software do not have the newer root certificate (named "ISRG
Root X1") in their trusted certificate collection. These systems are
now rejecting LE certificates.
In Toolforge, we think that this mainly affects tools running on the
Kubernetes cluster inside Debian Jessie based containers. Specifically
the "php5.6", "python", "python2", and "ruby2" containers are expected
to have issues with the LE certificate expiration based on what we
have found so far. Recommended replacement containers are "php7.4",
"python3.9", and "ruby25".
We also have reports of `mono` on the bastions + grid engine failing.
We do not yet have a fix for this. It will require us to compile and
install a newer version of mono for everyone who is using it.
Interested folks can follow progress of our infrastructure updates in
response to this issue at T291387 [3].
[0]: https://letsencrypt.org/
[1]: https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/
[2]: https://phabricator.wikimedia.org/T291387
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hi all,
When I run python3 --version on Toolforge I see version 3.5.3 is installed.
Because python 3.5 reached the end of its life in September 2020, pip is
really unhappy about that.
Is there a way to use a later version of python3 on Toolforge? If not, are
there plans to upgrade the OS and upgrade python with it?
PS: I know from Help:Toolforge/Python
<https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python> that 3.7.3 is
available on Kubernetes, but I am dealing with scripts that are submitted
via jsub and not k8s, and the overhead of converting them is prohibitive.
Thanks!