Hello Admins,
The build service is ready for the next phase, and we want to get some early users to test it and give early feedback!
This next step is going to start this Monday, when around 100 tool maintainers will kindly receive an email asking them to try and test the service.
The full list will be made available in the task [1]
You can read a draft of the email here[2]
Later next week, there will also be a session at the Athens Hackathon (thanks Slavina!) [3] introducing the build service to some new users too, as new user experiences give an important perspective too.
We will tentatively stay in this period of feedback for ~1 month. At that point we will sit back, reflect on the feedback (will share the thoughts with you too), and decide if we are ready for a broader announcement (cloud-announcement, blog post, …, to be defined), to start getting wider input.
In preparation for this, the team (including volunteers) has been working on setting up some minimal information on wikitech [4] and phabricator [5].
Feel free to add comments to the talk page [6] and/or make changes and fixes to those pages.
There's still many things to figure out, and many more will show up during this phase, your help and input will be critical in shaping this service.
I encourage you to try it out yourself if you have not, and open any bugs that you find or feature requests that you think would be useful, find the links for those in the feedback page [7].
You can see a more detailed planning in the task [1] (feel free to add comments there too)
Thanks for your continued support!
[1]: https://phabricator.wikimedia.org/T335249
[2]: https://etherpad.wikimedia.org/p/tQr7TQr20xorkEXXk6Pj
[3]: https://phabricator.wikimedia.org/T336055
[4]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service
[5]: https://phabricator.wikimedia.org/project/profile/6529/
[6]: https://wikitech.wikimedia.org/wiki/Help_talk:Toolforge/Build_Service
[7]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Feedback
Hi,
On 5/11/23 01:07, Seyram Komla Sapaty wrote:
Hello Admins,
The build service is ready for the next phase, and we want to get some early users to test it and give early feedback!
This next step is going to start this Monday, when around 100 tool maintainers will kindly receive an email asking them to try and test the service.
The full list will be made available in the task [1]
This is very different from what I was told at the Toolforge Council meeting last week. My impression was that you'd be reaching to a handful of maintainers for tools we know that can be migrated to the currently very "Python web service without any database migrations or other complications" centric system. How will those maintainers be selected?
You can read a draft of the email here[2]
Please clarify in the email which features are actually available today and which are not. Right now the draft gives an impression that push to deploy is currently supported, which is not true.
There's still many things to figure out, and many more will show up during this phase, your help and input will be critical in shaping this service.
This is unfortunately not the impression I have about this project. Most of the decision-making and coordination still seems to happen in private and undocumented spaces despite my multiple requests to get involved in those conversations.
Taavi
On Thu, 2023-05-11 at 01:40 +0300, Taavi Väänänen wrote:
Hi,
On 5/11/23 01:07, Seyram Komla Sapaty wrote:
Hello Admins,
The build service is ready for the next phase, and we want to get some early users to test it and give early feedback!
This next step is going to start this Monday, when around 100 tool maintainers will kindly receive an email asking them to try and test the service.
The full list will be made available in the task [1]
This is very different from what I was told at the Toolforge Council meeting last week. My impression was that you'd be reaching to a handful of maintainers for tools we know that can be migrated to the currently very "Python web service without any database migrations or other complications" centric system. How will those maintainers be selected?
The communications regarding the release so far have been a bit vague, this is the first one in which we aim to add more definition to it. And yes, that means that different people understood different things, sorry for that.
I'll let Komla reply about the process of selection of the maintainers, but there's a few goals in mind here:
* Feedback on the build process itself * Feedback on migration paths (so we can iterate on adding support for those) * Feedback on other new/unexpected use cases (so we can adapt where the project is going) * Help finding bugs
You can read a draft of the email here[2]
Please clarify in the email which features are actually available today and which are not. Right now the draft gives an impression that push to deploy is currently supported, which is not true.
Thanks for the feedback, will do!
There's still many things to figure out, and many more will show up during this phase, your help and input will be critical in shaping this service.
This is unfortunately not the impression I have about this project. Most of the decision-making and coordination still seems to happen in private and undocumented spaces despite my multiple requests to get involved in those conversations.
I'm sad to hear that, we have made a huge effort in the last few quarters to make sure that coordination and decisions are made in the open and collaboratively (two different things).
That was one of the main drivers for the Toolforge Workgroup[1], the Toolforge Build Service project page[2], and the phabricator board[3] with iterations on it.
The only communication specific to this project that currently happens in private (that I'm aware of) is the bi-weekly sync meeting, in which we decide the tasks each will take for the next iteration, and then is reflected publicly on the project board. We can consider opening it for anyone if you are interested.
Any major decision made has been going through the public decision request process[4], and minor decisions are either directly on-task or on code review.
There's also twice a week "co-working space" meeting that is mostly to pair on a specific issue or just hang out in a video call, it's more of a social space than technical one, if you want to join you are more than welcome to, just let me know and I'll send an invite. This one is not related to the project though.
The use of slack channels has been reduced to daily syncs, that we are working on moving to IRC soon enough (I raised it a couple team meetings back[5]), but want to avoid the noise on the main channel, so will have to figure out an alternative.
That one is not specific to the project either, and not everyone gives updates, it's a voluntary "I'm doing this today, I need help with that" daily message.
If there's any particular decision you think you were left out of, please share and we'll figure out a way to make them more accessible. Everyone makes mistakes, the important part is that we continue trying to fix them.
[1] https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Workgroup [2] https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Ongoing_Efforts/Toolfor... [3] https://phabricator.wikimedia.org/project/board/5596/ [4] https://phabricator.wikimedia.org/tag/cloud_services_proposals/ [5] https://etherpad.wikimedia.org/p/WMCS-2023-05-03
Taavi
Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
Thank you for the detailed response!
On 5/11/23 10:38, David Caro wrote:
The only communication specific to this project that currently happens in private (that I'm aware of) is the bi-weekly sync meeting, in which we decide the tasks each will take for the next iteration, and then is reflected publicly on the project board. We can consider opening it for anyone if you are interested.
Could we start from making the notes from those meetings public?
Also, can I ask which communication channels were used to decide on the details of the upcoming release and user communications relating to that? Was it this meeting or something else?
Taavi
On Thu, 2023-05-11 at 18:18 +0300, Taavi Väänänen wrote:
Thank you for the detailed response!
On 5/11/23 10:38, David Caro wrote:
The only communication specific to this project that currently happens in private (that I'm aware of) is the bi-weekly sync meeting, in which we decide the tasks each will take for the next iteration, and then is reflected publicly on the project board. We can consider opening it for anyone if you are interested.
Could we start from making the notes from those meetings public?
Done :) https://docs.google.com/document/d/1oVCRlctENWp_kaCLDTuVg7qU4rtnsCCxvGG23Jwj...
Though the idea was to add the noes in the page (did not get to it yet): https://docs.google.com/document/d/1oVCRlctENWp_kaCLDTuVg7qU4rtnsCCxvGG23Jwj...
We started (but stopped by mistake) to put it in the wiki, where it should have been: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Ongoing_Efforts/Toolfor...
Note that anything noted there was added after the decisions were made public.
Also, can I ask which communication channels were used to decide on the details of the upcoming release and user communications relating to that? Was it this meeting or something else?
There was not really a common decision made, just communicated to the rest of us and shared both in the toolforge build sync, and the following toolforge workgroup meeting (where you were also).
Taavi
Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
Quick update, we are reviewing the list of candidate tools to invite to test the service. We are thinking of reducing the number to ~30 tools that already use flask, python and git, and prioritizing those that still have to migrate off the grid (~12).
Here's a quick preliminary list[1].
Anyone knows of users that might be interested too? (even if they don't really match that criteria). We can add those to the list also.
We will re-check and expand the list after ~2 weeks.
I'll update the task once we have something a bit more defined.
[1]https://etherpad.wikimedia.org/p/first_round
On Thu, May 11, 2023 at 4:40 PM David Caro dcaro@wikimedia.org wrote:
On Thu, 2023-05-11 at 18:18 +0300, Taavi Väänänen wrote:
Thank you for the detailed response!
On 5/11/23 10:38, David Caro wrote:
The only communication specific to this project that currently happens in private (that I'm aware of) is the bi-weekly sync meeting, in which we decide the tasks each will take for the next iteration, and then is reflected publicly on the project board. We can consider opening it for anyone if you are interested.
Could we start from making the notes from those meetings public?
Done :)
https://docs.google.com/document/d/1oVCRlctENWp_kaCLDTuVg7qU4rtnsCCxvGG23Jwj...
Though the idea was to add the noes in the page (did not get to it yet):
https://docs.google.com/document/d/1oVCRlctENWp_kaCLDTuVg7qU4rtnsCCxvGG23Jwj...
We started (but stopped by mistake) to put it in the wiki, where it should have been:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Ongoing_Efforts/Toolfor...
Note that anything noted there was added after the decisions were made public.
Also, can I ask which communication channels were used to decide on the details of the upcoming release and user communications relating to that? Was it this meeting or something else?
There was not really a common decision made, just communicated to the rest of us and shared both in the toolforge build sync, and the following toolforge workgroup meeting (where you were also).
Taavi
Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information:
https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
Hi Taavi,
So far, the selected users I've reached out to are maintainers who have their tools sorted in the "Needs Custom Image" column(excluding those needing multistack support)on the grid migration project board[0].
In this next phase, we are slowly expanding the selected users from the hundreds of tools that are still in the 'Backlog' column on the migration board, with emphasis on the ones maintaining Python tools.
The goal is to inform them that there is one more lever available to them for their migration needs and also to let them start getting familiar with the tool they will probably have to use to migrate.
[0]: https://phabricator.wikimedia.org/tag/grid-engine-to-k8s-migration/
On Wed, May 10, 2023 at 10:40 PM Taavi Väänänen hi@taavi.wtf wrote:
Hi,
On 5/11/23 01:07, Seyram Komla Sapaty wrote:
Hello Admins,
The build service is ready for the next phase, and we want to get some early users to test it and give early feedback!
This next step is going to start this Monday, when around 100 tool maintainers will kindly receive an email asking them to try and test the service.
The full list will be made available in the task [1]
This is very different from what I was told at the Toolforge Council meeting last week. My impression was that you'd be reaching to a handful of maintainers for tools we know that can be migrated to the currently very "Python web service without any database migrations or other complications" centric system. How will those maintainers be selected?
You can read a draft of the email here[2]
Please clarify in the email which features are actually available today and which are not. Right now the draft gives an impression that push to deploy is currently supported, which is not true.
There's still many things to figure out, and many more will show up during this phase, your help and input will be critical in shaping this service.
This is unfortunately not the impression I have about this project. Most of the decision-making and coordination still seems to happen in private and undocumented spaces despite my multiple requests to get involved in those conversations.
Taavi
Hi,
On 5/11/23 15:36, Seyram Komla Sapaty wrote:
So far, the selected users I've reached out to are maintainers who have their tools sorted in the "Needs Custom Image" column(excluding those needing multistack support)on the grid migration project board[0].
My understanding is that most of the tools in the "needs custom image" column need either multistack support or custom Apt packages installed in the image, neither of which are currently supported. So starting from that column seems like a very strange decision to me.
In this next phase, we are slowly expanding the selected users from the hundreds of tools that are still in the 'Backlog' column on the migration board, with emphasis on the ones maintaining Python tools.
Your previous message indicates that you were planning to send a large amount of invites next Monday (May 15th). What does "slowly" mean here?
I still believe that sending any non-small amount of invites should not happen at this stage. There is a large amount of known issues that make debugging very difficult[0], cause confusing and unexplainable failures[1] or completely break support for the most popular[2] runtime on Toolforge[3].
[0]: https://phabricator.wikimedia.org/T336225 [1]: https://phabricator.wikimedia.org/T336360 [2]: https://k8s-status.toolforge.org/images/ [3]: https://phabricator.wikimedia.org/T335865
The only thing that's going to happen if we unnecessarily rush this release is that people are going to get a mental image where Kubernetes, our tooling around it and the Build Service are difficult to use and are full of bugs and annoying limitations, which is going to make it even harder and slower to get everyone to migrate off the Grid. We've made this mistake before and I would really like to avoid it this time.
Taavi
cloud-admin@lists.wikimedia.org