Sorry that was not complete xd, still getting used to the gmail UI

On Tue, Oct 24, 2023 at 2:13 PM David Caro <dcaro@wikimedia.org> wrote:

Hi Maciej!

Replying inline

On Mon, Oct 23, 2023 at 9:03 PM Maciej Jaros <egil@wp.pl> wrote:

Hi

Build tool sounds like an interesting idea. What is the intended use case here?

Using for complex tools that might require complex or too specific tools on the server side?
Using for small tools that has some build steps which produce static files? (aka serverless server)
Both, to some extent, though it's main aim is to simplify the build process altogether.

It enables you to not have to manually build you tool, and still use the high level package managers of your language of choice (pip/golang/bundle/npm/...).
For some complex tools, you can also use the package manager (using the Aptfile file, ubuntu based base image), though it might not be suitable for very complex tools that might need a different kind of dependencies installed.

For tools that generate static files, there's some options:

* Use buildservice to build an image that has everything on it (npm seems the right one there), you might have to add a "build" script to build the webpack for production:

https://devcenter.heroku.com/articles/node-best-practices#hook-things-up

* Use buildservice to setup the environment (npm install and such, but not build the assets), and then use that buildservice image to run generate the assets in a job, and serve them though the static files folder

https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Job

https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Serving_static_files

I think the first might be simpler, but might require you to change a bit the package.json you have, and seems to complain also about the duplicated lockfile (yarn.lock + package-lock.json) .

Please open a task to follow up on the effort if you need any help or have any issues when you start doing so.

An example for the serverless tool (2).

I have a map tool hosted on Github.

The map tool is built with Node.js with Webpack.

The output of webpack is basically a html with JS (and some other static assets).

The tool is hosted on Toolforge now (zabytki.toolforge.org).

Can I... Or, to be more exact... Should I eventually move that tool to a build service?

Essentially, what I'm asking is whether building a tool with this service is a small or large burden? Is it a small enough burden to use even with very small tools?"

Definitely yes, you can use it to build very small tools, for us, it's simpler to maintain tools based on the buildservice than grid ones or basic webservice ones. Feel free to open a task if you find any issues or get stuck, there's still some flows that we have not figured out, but we are eager to help people get things done :)

Serving static files might be a good use case too, so we might create a tutorial out of it.

Regards,
Maciej Nux.

Seyram Komla Sapaty (2023-10-18 16:16):
Hello,

The Toolforge admin team is happy to announce that the Toolforge Build Service[0] is now available in open beta.

The Build Service is intended to allow more tools to migrate off the Grid Engine and to make the process for deploying code to

Toolforge easier and more flexible, by building container images with the specific dependencies for each tool.

Here are quick highlights of some of the current key features:

1. Build your tool from source code, using you language's dependency management tool, no dockerfiles, no scripts, no manual steps

2. Use industry-wide standards[1] no vendor lock-in by using upstream buildpacks

3. Support for many languages out of the box[2]

4. Envvars - Create and manage environment variables and secrets that are available at runtime.[3]

5. Ability to install packages from the Ubuntu repositories[4]

6. Improved resiliency and resource usage by allowing NFS-less webservices[5], if you don't need NFS

7. Test your image locally, or anywhere[6]

Please review the current known limitations here[7]

We also have a growing list of tutorials for various languages[8]

During this open beta, we invite you to actively participate and share your feedback replying to this thread or through irc, and if you

find any issues or have any feature suggestions, you can use this task template[9].

Your insights will help us enhance and tailor the Build Service to meet the needs of your tools.

The plan is to have this phase run for the next months, and if no big issues are found, promote it to global availability phase 1 (GA1)

while we work on adding automatic triggering and deployment, for which we will do a second round of beta testing for those specific features.

This unblocks the last step to migrate out of the grid, so we request all grid users to give it a try and report any issues they might find,

there's no big changes expected for the currently implemented features, so any work done now will help later.

Thank you for being a part of this journey. We look forward to your invaluable feedback and collaboration as we strive to provide a better developer experience.

[0]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service

[1]: https://buildpacks.io/

[2]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Supported_languages
[3]:https://wikitech.wikimedia.org/wiki/Help:Toolforge/Envvars_Service

[4]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Installing_Apt_packages
[5]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Using_NFS_shared_storage

[6]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Testing_locally_(optional)

[7]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Known_current_limitations

[8]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Tutorials_for_popular_languages

[9]: https://w.wiki/7kpi

--

Seyram Komla Sapaty
Developer Advocate

Wikimedia Cloud Services
_______________________________________________
Cloud-announce mailing list -- cloud-announce@lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.org/
_______________________________________________
Cloud mailing list -- cloud@lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
_______________________________________________
Cloud mailing list -- cloud@lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/