Report on the Toolhub project for the week ending 2021-08-27.
See the on-wiki progress report [0] for additional details and clickable links.
== TL;DR ==
* Tool created to collect enwiki userscripts
* Production logging
* Production memcache
Last week's report ended with an expectation that by this report we
would have a better understanding of the work left to deploy Toolhub
fully into production. Bryan's current punch list of known remaining
work is:
* Select content license and document in Toolhub UI at all data entry locations
* Add support for mcrouter to the Helm chart for Toolhub and
per-deployment configuration via helmfile settings
* Load test API to establish initial container sizing limits for CPU and RAM
* Add production OAuth secrets to generated configuration
* Deploy into "staging" Kubernetes cluster
* Create initial database tables and seed data
* Create initial Elasticsearch schema and index
* Test crawler process from inside Kubernetes staging cluster to
validate HTTP proxy configuration
It seems unlikely that all of these tasks will be Done in the coming
week, but it should be possible to work through a number of them.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-27
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
Report on the Toolhub project for the week ending 2021-08-20.
See the on-wiki progress report [0] for additional details and clickable links.
== TL;DR ==
* Wikimania outcomes
* Importing data from the wikis
We started the week with discussions at Wikimania to promote the
project generally and continue gathering information on what we could
do next. Bryan spent some time doing hackathon-like experiments on
using Toolhub as a tool builder. Discussions started last week about
the overlooked content licensing issue continued both on Phabricator
and with the Foundation's Legal team. A few patches needed for the
production deployment were merged, but there is more work remaining
there. Bryan expects to return his focus to the production deployment
next week, and will ideally be able to make a better forecast of when
https://toolhub.wikimedia.org/ will be live as result by next week's
report.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-20
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
Report on the Toolhub project for the week ending 2021-08-13.
See the on-wiki progress report [0] for additional details and clickable links.
== TL;DR ==
* Security review status update
* Kubernetes deployment progress
* User experience improvements
* Content licensing
* Wikimania 2021!
All long term watchers of the project will likely realize that we did
not achieve our hoped for 2021-08-12 production deployment target. We
came very close, but ultimately there were more tasks left to complete
in the final week than it was reasonably possible to accomplish. Bryan
hopes that everyone is understanding of these delays, but also
informed of what remains to be done and progress on those goals.
We are so close, but not quite there yet. :)
Check out our unconference plans for talking about Toolhub at
Wikimania [1] and join a session if you have the chance! We have
dropped one Toolhub Introduction session that was listed last week,
but if you are interested in that content you can still watch the
lightning talk or unconference session covering it on Saturday
2021-08-14.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-06
[1]: https://wikimania.wikimedia.org/wiki/2021:Unconference/Toolhub
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
Do any of you have past experience in choosing a content license for a
new Wikimedia project?
I had the horrible realization today that this has not been done for
Toolhub. I have filed https://phabricator.wikimedia.org/T288832 with
my initial ideas. I have also reached out via private message (slack)
to the Foundation's Legal team for their advice on policies and
procedures that need to be followed for this kind of thing.
Regardless of your past experience selecting licenses for a whole
project, if you have strong and informed opinions on CC0 vs CC-BY for
the type of data that Toolhub will aggregate I welcome your feedback
here or on the Phabricator task.
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
Report on the Toolhub project for the week ending 2021-08-06.
See the on-wiki progress report [0] for additional details and clickable links.
== TL;DR ==
* Kubernetes deployment configuration in progress
* Production database created
* Production Elasticsearch discussed
* Security review status update
* Toolhub at Wikimania
Wikimania is next week! And so is our target deployment date. There is
still some uncertainty about the hoped for deployment date. The
initial recommendations from the security review are still pending,
the Helm charts for the deployment need some additional work, and
typical August holidays plus Wikimania prep may make it difficult to
receive all the necessary approvals in time for a Thursday August 12th
deployment. We do have a fully functional demo server in place and
will be able to use that to show off live features during Wikimania if
necessary. At this point Bryan is considering the production release
prior to Wikimania a "nice to have" outcome, but cautions everyone
that it may not happen. A release in the two weeks following Wikimania
is a higher probability outcome.
Check out our unconference plans for talking about Toolhub at
Wikimania [1] and join a session if you have the chance!
[0]: https://meta.wikimedia.org/wiki/Toolhub/Progress_reports/2021-08-06
[1]: https://wikimania.wikimedia.org/wiki/2021:Unconference/Toolhub
Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808