Sorry that was not complete xd, still getting used to the gmail UI
On Tue, Oct 24, 2023 at 2:13 PM David Caro <dcaro(a)wikimedia.org> wrote:
Hi Maciej!
Replying inline
On Mon, Oct 23, 2023 at 9:03 PM Maciej Jaros <egil(a)wp.pl> wrote:
Hi
Build tool sounds like an interesting idea. What is the intended use case
here?
1. Using for complex tools that might require complex or too specific
tools on the server side?
2. Using for small tools that has some build steps which produce
static files? (aka serverless server)
Both, to some extent, though it's main aim is to simplify the build
process
altogether.
It enables you to not have to manually build you tool, and still use the
high level package managers of your language of choice
(pip/golang/bundle/npm/...).
For some complex tools, you can also use the package manager (using the
Aptfile file, ubuntu based base image), though it might not be suitable for
very complex tools that might need a different kind of dependencies
installed.
For tools that generate static files, there's some options:
* Use buildservice to build an image that has everything on it (npm seems
the right one there), you might have to add a "build" script to build the
webpack for production:
https://devcenter.heroku.com/articles/node-best-practices#hook-things-up
* Use buildservice to setup the environment (npm install and such, but not
build the assets), and then use that buildservice image to run generate the
assets in a job, and serve them though the static files folder
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Job
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Serving_static_files
I think the first might be simpler, but might require you to change a bit
the package.json you have, and seems to complain also about the duplicated
lockfile (yarn.lock + package-lock.json) .
Please open a task to follow up on the effort if you need any help or have
any issues when you start doing so.
An example for the serverless tool (2).
- I have a map tool hosted on Github
<https://github.com/Eccenux/wlm-zabytki-mapa/>.
- The map tool is built with Node.js with Webpack
<https://github.com/Eccenux/wlm-zabytki-mapa/actions/runs/6018297452/job/16326215483>
.
- The output of webpack is basically a html with JS (and some other
static assets).
- The tool is hosted on Toolforge now (
zabytki.toolforge.org).
Can I... Or, to be more exact... *Should* I eventually move that tool to
a build service?
Essentially, what I'm asking is whether building a tool with this service
is a small or large burden? Is it a small enough burden to use even with
very small tools?"
Definitely yes, you can use it to build very small tools, for us, it's
simpler to maintain tools based on the buildservice than grid ones or basic
webservice ones. Feel free to open a task if you find any issues or get
stuck, there's still some flows that we have not figured out, but we are
eager to help people get things done :)
Serving static files might be a good use case too, so we might create a
tutorial out of it.
Regards,
Maciej Nux.
Seyram Komla Sapaty (2023-10-18 16:16):
Hello,
The Toolforge admin team is happy to announce that the Toolforge Build
Service[0] is now available in open beta.
The Build Service is intended to allow more tools to migrate off the Grid
Engine and to make the process for deploying code to
Toolforge easier and more flexible, by building container images with the
specific dependencies for each tool.
Here are quick highlights of some of the current key features:
1. Build your tool from source code, using you language's dependency
management tool, no dockerfiles, no scripts, no manual steps
2. Use industry-wide standards[1] no vendor lock-in by using upstream
buildpacks
3. Support for many languages out of the box[2]
4. Envvars - Create and manage environment variables and secrets that are
available at runtime.[3]
5. Ability to install packages from the Ubuntu repositories[4]
6. Improved resiliency and resource usage by allowing NFS-less
webservices[5], if you don't need NFS
7. Test your image locally, or anywhere[6]
Please review the current known limitations here[7]
We also have a growing list of tutorials for various languages[8]
During this open beta, we invite you to actively participate and share
your feedback replying to this thread or through irc, and if you
find any issues or have any feature suggestions, you can use this task
template[9].
Your insights will help us enhance and tailor the Build Service to meet
the needs of your tools.
The plan is to have this phase run for the next months, and if no big
issues are found, promote it to global availability phase 1 (GA1)
while we work on adding automatic triggering and deployment, for which we
will do a second round of beta testing for those specific features.
This unblocks the last step to migrate out of the grid, so we request all
grid users to give it a try and report any issues they might find,
there's no big changes expected for the currently implemented features, so
any work done now will help later.
Thank you for being a part of this journey. We look forward to your
invaluable feedback and collaboration as we strive to provide a better
developer experience.
[0]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service
[1]:
https://buildpacks.io/
[2]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Supported_…
[
3]:https://wikitech.wikimedia.org/wiki/Help:Toolforge/Envvars_Service
[4]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Installing…
[5]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Using_NFS_…
[6]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Testing_lo…
[7]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Known_curr…
[8]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Tutorials_…
[9]:
https://w.wiki/7kpi
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
_______________________________________________
Cloud mailing list -- cloud(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
_______________________________________________
Cloud mailing list -- cloud(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/