- Toolhub-dev - lists.wikimedia.org

Welcome Tricia Burmeister to the list
by Bryan Davis 13 Oct '21

13 Oct '21

Hey folks, I approved Tricia Burmeister's application to join this mailing list today. Tricia has recently joined the Foundation as a Technical Writer in the Developer Advocacy subteam of the Technical Engagement team. Welcome to the list Tricia! Everyone, please do remember that this list is a part of the Wikimedia technical spaces and thus covered by the <https://www.mediawiki.org/wiki/Code_of_Conduct>. The archives of the list are also publicly available, so keep that in mind as well when posting here. As long as we stick to the topic of the list, developing Toohub, I think everyone will be fine. :) Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

Volunteers to be early functionaries for Toolhub?
by Bryan Davis 08 Oct '21

08 Oct '21

Hey folks, We are planning on a community wide announcement of Toolhub on Wednesday 2021-10-13. In the days following the announcement we will need to start handing out advanced user rights (<https://meta.wikimedia.org/wiki/Toolhub#User_permission_levels>) to enable patrolling and moderation of content. I was hoping that some of you would be interested in having at least the Bureaucrat right in Toolhub to help in bootstrapping this new sub-community. At some point I expect that the new community will invent its own rules for how rights are asked for and granted, but for now it is a wide open topic. I would love to hear from those of you with on-wiki experience in drafting or following these kinds of policies to help me come up with some guidelines beyond "give rights to those you already trust", which is basically my first idea here. Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-10-08 weekly Toolhub update
by Bryan Davis 08 Oct '21

08 Oct '21

Report on the Toolhub project for the week ending 2021-10-01 See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Crawler working in production * Metawiki help documents * Deploy date picked! We have chosen Wednesday 2021-10-13 as the official launch date for Toolhub. The team is now working on drafting emails and other announcement messages to send to the Wikimedia community on that day and in the days following to promote the project widely. It has been a long journey from a skeleton project in September 2020 to getting everything working in the production Kubernetes cluster this past week. Next week we will announce the official Toolhub URL and it's features to the entire Wikimedia movement. This closes the initial chapter of the project, but is really just the start of making Toolhub the project that the movement has asked for and deserves. Bryan would like to thank everyone, and there have been many of you, who has helped keep the project alive and progressing over the past year and beyond. The team is very excited for the conversations and work to come as we continue to build this project together. To all of you who have been following and participating in this mailing list, thank you. I have not reached out to the list for help too often in the last year, but each time I did folks responded and helped move us forward. I'm not sure what the list or the advisory board will look like in the future, but I hope that we will have more developers join the project and continue to have others around to help the developers and project managers understand the many interesting and complex needs and expectations of the Wikimedia community. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-10-08 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-10-01 weekly Toolhub update
by Bryan Davis 01 Oct '21

01 Oct '21

Report on the Toolhub project for the week ending 2021-10-01 See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Kubernetes deployment progress * Metawiki help documents * Planning launch announcements One step forward! One step to the side. :/ We are inching closer to a public release, but still have some technical challenges to work out before we can be confident in the launch date. The amount of time this is taking is a bit frustrating, but we know that we only get one chance at making a first impression with the larger community and we do not want to waste that with a buggy product. Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-09-24 weekly Toolhub update
by Bryan Davis 24 Sep '21

24 Sep '21

Report on the Toolhub project for the week ending 2021-09-24. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Minor code cleanups * Kubernetes deployment progress Bryan's goal for the week was to complete the technical tasks of creating the initial database tables and test the crawler process from inside the Kubernetes staging cluster. The database tables have been created and some testing done, but that testing revealed missing configuration which took the rest of the week to design and implement. With that configuration now in place, work will continue next week on setting up LVS, DNS, and Varnish for https://toolhub.wikimedia.org. Once that is done we will be able to continue testing OAuth authentication and the crawler. If all goes well we may be ready to announce Toolhub to the movement in the following week! Many thanks to Giuseppe Lavagetto, Manuel Aróstegui, and Kunal Mehta for their help this week in finding and resolving deployment issues. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-09-24 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-09-17 weekly Toolhub update
by Bryan Davis 17 Sep '21

17 Sep '21

Report on the Toolhub project for the two weeks ending 2021-09-17. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * CC0 selected for content license * List editing With the content license selected, the list of critical remaining tasks before we can do the official launch is fairly short: * Create initial database tables and seed data * Create initial Elasticsearch schema and index * Test crawler process from inside Kubernetes staging cluster to validate HTTP proxy configuration Beyond these must do tasks, we would also like to work on end user documentation for using Toolhub on metawiki and to plan our announcement strategy for letting the Wikimedia community know about the project. Bryan expects to work on all of these things in the coming week with a personal goal of having the technical tasks completed by the next weekly report. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-09-17 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-09-03 weekly Toolhub update
by Bryan Davis 03 Sep '21

03 Sep '21

Report on the Toolhub project for the week ending 2021-09-03. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * First deployment into 'staging' cluster Last week's report gave a punch list of known remaining tasks. After this week's progress, that list now looks like: * Select content license and document in Toolhub UI at all data entry locations * {{Done}} Add support for mcrouter to the Helm chart for Toolhub and per-deployment configuration via helmfile settings * {{Done}} Load test API to establish initial container sizing limits for CPU and RAM * {{Done}} Add production OAuth secrets to generated configuration * {{Done}} Deploy into "staging" Kubernetes cluster * Create initial database tables and seed data * Create initial Elasticsearch schema and index * Test crawler process from inside Kubernetes staging cluster to validate HTTP proxy configuration There will not be a weekly report published on 2021-09-10. Bryan is going to be taking the week off to enjoy some late summer/early fall fun away from his laptop. Expect our next report on 2021-09-17, and join Bryan in hoping that the update provided then will include either a soft production launch or a concrete timeline for the official product launch. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-09-03 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-08-27 weekly Toolhub update
by Bryan Davis 27 Aug '21

27 Aug '21

Report on the Toolhub project for the week ending 2021-08-27. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Tool created to collect enwiki userscripts * Production logging * Production memcache Last week's report ended with an expectation that by this report we would have a better understanding of the work left to deploy Toolhub fully into production. Bryan's current punch list of known remaining work is: * Select content license and document in Toolhub UI at all data entry locations * Add support for mcrouter to the Helm chart for Toolhub and per-deployment configuration via helmfile settings * Load test API to establish initial container sizing limits for CPU and RAM * Add production OAuth secrets to generated configuration * Deploy into "staging" Kubernetes cluster * Create initial database tables and seed data * Create initial Elasticsearch schema and index * Test crawler process from inside Kubernetes staging cluster to validate HTTP proxy configuration It seems unlikely that all of these tasks will be Done in the coming week, but it should be possible to work through a number of them. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-27 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-08-20 weekly Toolhub update
by Bryan Davis 20 Aug '21

20 Aug '21

Report on the Toolhub project for the week ending 2021-08-20. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Wikimania outcomes * Importing data from the wikis We started the week with discussions at Wikimania to promote the project generally and continue gathering information on what we could do next. Bryan spent some time doing hackathon-like experiments on using Toolhub as a tool builder. Discussions started last week about the overlooked content licensing issue continued both on Phabricator and with the Foundation's Legal team. A few patches needed for the production deployment were merged, but there is more work remaining there. Bryan expects to return his focus to the production deployment next week, and will ideally be able to make a better forecast of when https://toolhub.wikimedia.org/ will be live as result by next week's report. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-20 Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2021-08-13 weekly Toolhub update
by Bryan Davis 13 Aug '21

13 Aug '21

Report on the Toolhub project for the week ending 2021-08-13. See the on-wiki progress report [0] for additional details and clickable links. == TL;DR == * Security review status update * Kubernetes deployment progress * User experience improvements * Content licensing * Wikimania 2021! All long term watchers of the project will likely realize that we did not achieve our hoped for 2021-08-12 production deployment target. We came very close, but ultimately there were more tasks left to complete in the final week than it was reasonably possible to accomplish. Bryan hopes that everyone is understanding of these delays, but also informed of what remains to be done and progress on those goals. We are so close, but not quite there yet. :) Check out our unconference plans for talking about Toolhub at Wikimania [1] and join a session if you have the chance! We have dropped one Toolhub Introduction session that was listed last week, but if you are interested in that content you can still watch the lightning talk or unconference session covering it on Saturday 2021-08-14. [0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-06 [1]: https://wikimania.wikimedia.org/wiki/2021:Unconference/Toolhub Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 0

2024

2023

2022

2021

2020

Toolhub-dev