Welcome to the first weekly project update for Toolhub! In an attempt to keep our loosely connected group informed about happenings across Toolhub we will try to establish a regular practice of collecting updates and sending them out once a week. This week things reported will be off the top of Bryan's head, but work will happen soon™ to create some place (probably on metawiki) to record notes during the week so that they can be summarized and distributed periodically.
Since this is the first "weekly" summary, and the project has been doing things for about 2 months, this first update will cover a bit more than just one week of events. :)
== Advisory council "complete" == T258185 was closed by Bryan on 2020-09-14 after Risker agreed to join the advisory council. We now have representation on the advisory committee of folks with experience in SRE, Security, Community Relations, technical volunteering, and on-wiki contributions. More folks may be invited to the committee if we find large gaps during discussions of features and needs of the system, but this completes the list of skills that was previously identified as critical to initial work.
== Vue.js picked as UI framework == T261029 was closed by Bryan on 2020-09-14 after Srishti selected Vue.js. Srishti initially narrowed the choices for a Javascript UI framework down to React and Vue.js. Moriel and others made a good case of adopting Vue.js in alignment with the relatively recent decision by the now-disbanded Front-end Architecture Working Group (FAWG) to recommend Vue.js as the future replacement for OOJS and other component frameworks in MediaWiki. Ultimately Srishti and Bryan decided that using Vue.js would be technically possible and politically advantageous.
== Decision record created on metawiki == Bryan has started https://meta.wikimedia.org/wiki/Toolhub/Decision_record as a location to collect evidence of decisions made duing the course of designing and implementing Toolhub. The intent is to create a document that will help anyone joining the advisory council or implementation team catch up on the important bits of past discussions. Currently it documents the UI framework selection and a prior fiat decision to use an "API first" design process when building application features.
== Dynamic translation research == Toolhub will not be attempting to establish its own translator community. Instead we are hoping to integrate with the existing translatewiki.net (TWN) community. Currently how that integration will actually work is undefined, but there are 2 ideas along with some pros and cons outlined in T259838. We expect to do some tech spike work in the coming quarter to get a better idea of the feasibility of a web API integration approach. If that does not seem viable after some testing we are likely to go with the idea of exporting strings to a TWN supported catalog format, shipping that catalog to TWN via git repos (their normal workflow), and ultimately getting translations back for integration with the application.
Bryan documented an afternoon of research at T259838#6460927 existing Django apps intended to help with the technical issues of translating user provided content stored in the application's database. High level summaries of the functionality of each app are available on the Phabricator task.
== Toolinfo v1.2.0-draft01 json schema published for review == Bryan did a review of the data model documented by James Hare during the 2018 work on Toolhub to refresh his understanding of the design. During this review, the lack of toolinfo.json support for declaring the URL of end-user facing documents seemed like something that should be revisited. This had been brought up in 2018 by Kunal on the data model talk page [0]. James explained that he had set this field as an "annotation" value to be added thorough Toolhub rather than a toolinfo.json publisher controlled value in a hope of supporting links to both 'official' and community created documentation. Bryan agrees with the need to allow documenting community created documentation links, but felt that we can figure out how to deal with competing links in the user interfaces.
A new v1.2.0-draft01 json schema is now on metawiki for review [1]. A "final" v1.2.0 schema will probably not be released until Toolhub reaches a milestone of a working toolinfo.json submission and storage system to reduce the documentation churn that theory meeting practice is likely to expose in the current draft.
== Prototype development environment == Bryan has been poking at a local development environment [2] for the project over the last few weeks. This week he reached a point where he shared the system with Srishti to see if it could be made to work anywhere other than Bryan's laptops. That test turned out to be successful! Srishti reported back that she was able to get the system booted up on her laptop and returning the expected empty Django project landing page.
The current system is very experimental and the tooling it uses may change significantly before we hit any deployment milestones. The current system is using Docker Compose to manage multiple Docker containers (initial one for the Django app and one for a MariaDB database), Poetry for Python dependency management and versioning, and a custom Makefile to make using it all a bit easier for the developer. Today only git, Docker, Docker Compose, and GNU make are needed on the developer's laptop. All of the Python and other tooling requirements are being packed into the development mode Docker containers. This may or may not be sustainable as we progress out of experimental mode and start to integrate with the Wikimedia development ecosystem (Gerrit, Jenkins, etc), but it has been a useful experiment for Bryan who has not worked locally with Docker extensively.
== Wrap up == That's all the news that Bryan can remember today. :)
[0]: https://meta.wikimedia.org/wiki/Talk:Toolhub/Data_model#1.1.0_feedback [1]: https://meta.wikimedia.org/wiki/Toolhub/Data_model#Version_1.2.0 [2]: https://github.com/bd808/toolhub
Bryan
All these updates sound promising! Thanks for sharing, Bryan :)
Cheers, Srishti
On Fri, Sep 18, 2020 at 2:47 PM Bryan Davis bd808@wikimedia.org wrote:
Welcome to the first weekly project update for Toolhub! In an attempt to keep our loosely connected group informed about happenings across Toolhub we will try to establish a regular practice of collecting updates and sending them out once a week. This week things reported will be off the top of Bryan's head, but work will happen soon™ to create some place (probably on metawiki) to record notes during the week so that they can be summarized and distributed periodically.
Since this is the first "weekly" summary, and the project has been doing things for about 2 months, this first update will cover a bit more than just one week of events. :)
== Advisory council "complete" == T258185 was closed by Bryan on 2020-09-14 after Risker agreed to join the advisory council. We now have representation on the advisory committee of folks with experience in SRE, Security, Community Relations, technical volunteering, and on-wiki contributions. More folks may be invited to the committee if we find large gaps during discussions of features and needs of the system, but this completes the list of skills that was previously identified as critical to initial work.
== Vue.js picked as UI framework == T261029 was closed by Bryan on 2020-09-14 after Srishti selected Vue.js. Srishti initially narrowed the choices for a Javascript UI framework down to React and Vue.js. Moriel and others made a good case of adopting Vue.js in alignment with the relatively recent decision by the now-disbanded Front-end Architecture Working Group (FAWG) to recommend Vue.js as the future replacement for OOJS and other component frameworks in MediaWiki. Ultimately Srishti and Bryan decided that using Vue.js would be technically possible and politically advantageous.
== Decision record created on metawiki == Bryan has started https://meta.wikimedia.org/wiki/Toolhub/Decision_record as a location to collect evidence of decisions made duing the course of designing and implementing Toolhub. The intent is to create a document that will help anyone joining the advisory council or implementation team catch up on the important bits of past discussions. Currently it documents the UI framework selection and a prior fiat decision to use an "API first" design process when building application features.
== Dynamic translation research == Toolhub will not be attempting to establish its own translator community. Instead we are hoping to integrate with the existing translatewiki.net (TWN) community. Currently how that integration will actually work is undefined, but there are 2 ideas along with some pros and cons outlined in T259838. We expect to do some tech spike work in the coming quarter to get a better idea of the feasibility of a web API integration approach. If that does not seem viable after some testing we are likely to go with the idea of exporting strings to a TWN supported catalog format, shipping that catalog to TWN via git repos (their normal workflow), and ultimately getting translations back for integration with the application.
Bryan documented an afternoon of research at T259838#6460927 existing Django apps intended to help with the technical issues of translating user provided content stored in the application's database. High level summaries of the functionality of each app are available on the Phabricator task.
== Toolinfo v1.2.0-draft01 json schema published for review == Bryan did a review of the data model documented by James Hare during the 2018 work on Toolhub to refresh his understanding of the design. During this review, the lack of toolinfo.json support for declaring the URL of end-user facing documents seemed like something that should be revisited. This had been brought up in 2018 by Kunal on the data model talk page [0]. James explained that he had set this field as an "annotation" value to be added thorough Toolhub rather than a toolinfo.json publisher controlled value in a hope of supporting links to both 'official' and community created documentation. Bryan agrees with the need to allow documenting community created documentation links, but felt that we can figure out how to deal with competing links in the user interfaces.
A new v1.2.0-draft01 json schema is now on metawiki for review [1]. A "final" v1.2.0 schema will probably not be released until Toolhub reaches a milestone of a working toolinfo.json submission and storage system to reduce the documentation churn that theory meeting practice is likely to expose in the current draft.
== Prototype development environment == Bryan has been poking at a local development environment [2] for the project over the last few weeks. This week he reached a point where he shared the system with Srishti to see if it could be made to work anywhere other than Bryan's laptops. That test turned out to be successful! Srishti reported back that she was able to get the system booted up on her laptop and returning the expected empty Django project landing page.
The current system is very experimental and the tooling it uses may change significantly before we hit any deployment milestones. The current system is using Docker Compose to manage multiple Docker containers (initial one for the Django app and one for a MariaDB database), Poetry for Python dependency management and versioning, and a custom Makefile to make using it all a bit easier for the developer. Today only git, Docker, Docker Compose, and GNU make are needed on the developer's laptop. All of the Python and other tooling requirements are being packed into the development mode Docker containers. This may or may not be sustainable as we progress out of experimental mode and start to integrate with the Wikimedia development ecosystem (Gerrit, Jenkins, etc), but it has been a useful experiment for Bryan who has not worked locally with Docker extensively.
== Wrap up == That's all the news that Bryan can remember today. :)
Bryan
Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808
Toolhub-dev mailing list Toolhub-dev@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/toolhub-dev
On Fri, Sep 18, 2020 at 11:47 PM Bryan Davis bd808@wikimedia.org wrote:
Welcome to the first weekly project update for Toolhub!
Hello!
Some small comments inline
[CUT]
== Prototype development environment == Bryan has been poking at a local development environment [2] for the project over the last few weeks. This week he reached a point where he shared the system with Srishti to see if it could be made to work anywhere other than Bryan's laptops. That test turned out to be successful! Srishti reported back that she was able to get the system booted up on her laptop and returning the expected empty Django project landing page.
This is a great result! And while I think we'll use kubernetes and helm to deploy this software in production, I prefer docker-compose for local development to using minikube.
The current system is very experimental and the tooling it uses may change significantly before we hit any deployment milestones. The current system is using Docker Compose to manage multiple Docker containers (initial one for the Django app and one for a MariaDB database), Poetry for Python dependency management and versioning, and a custom Makefile to make using it all a bit easier for the developer. Today only git, Docker, Docker Compose, and GNU make are needed on the developer's laptop. All of the Python and other tooling requirements are being packed into the development mode Docker containers. This may or may not be sustainable as we progress out of experimental mode and start to integrate with the Wikimedia development ecosystem (Gerrit, Jenkins, etc), but it has been a useful experiment for Bryan who has not worked locally with Docker extensively.
Regarding this, I think it would be advisable to convert the system to use the deployment pipeline sooner rather than later, in particular given it will let you use in development basically the same images you'll use in production (including the same versions of all dependencies).
You can refer to the two very good tutorials here:
https://wikitech.wikimedia.org/wiki/Blubber/Tutorial https://wikitech.wikimedia.org/wiki/PipelineLib/Tutorial
This of course makes even more sense if you move development to gerrit.
Cheers,
Giuseppe
toolhub-dev@lists.wikimedia.org