Hi there,
The Toolforge jobs framework just got upgraded with a few new features:
* support for custom logs
* support for job failure retry policy
* new behavior with job image listing
* some initial validation of YAML files
The documentation should be mostly up-to-date in wikitech:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework
You can stop reading here unless you want more details :-)
The custom log files feature will allow you do things like:
* using a custom directory to store log files
* merging stdout/stderr logs together into a single file
* ignoring one of the two log streams
The job retry policy allows to instruct the computing engine to restart jobs
that failed, up to 5 times.
Job images are now listed in a different format, and deprecated images are
hidden by default, to encourage usage of newer ones.
Regarding the YAML validation, the toolforge-jobs utility will now emit a
warning if some key is unknown. We plan to make this more robust in the future,
also providing a schema file.
We don't usually announce upgrades, but this one in particular contained much
awaited features. This is the result of hard work by several folks, in
particular Taavi (community member) and Raymond (WMF contractor).
Happy `toolforging`. Regards.
--
Arturo Borrero Gonzalez
Senior SRE / Wikimedia Cloud Services
Wikimedia Foundation