Sorry about the noise with the alerts. After deploying, I discovered a few small errors in the timer/service definitions that have been fixed.

Finally, there is an error in Wikitech that is causing `mwscript runJobs.php` to fail frequently (T218935). That was being hidden in the noise with cronjobs as was surfaced by adopting timers, so I think that's a win. That job has been downtimed in Icinga for now.

On Thu, Mar 21, 2019 at 11:00 AM Giovanni Tirloni gtirloni@wikimedia.org wrote:

Hi,

Just a heads up, I've merged a change that converts OpenStack admin cronjobs to systemd timers.

List all timers:

root@cloudcontrol1003:~# systemctl list-timers --no-pager
NEXT                         LEFT       LAST                         PASSED       UNIT                                            ACTIVATES
Thu 2019-03-21 14:00:00 UTC  6min left  Thu 2019-03-21 13:50:01 UTC  3min 31s ago designate_floating_ip_ptr_records_updater.timer designate_floating_ip_ptr_records_updater.service
Thu 2019-03-21 14:15:00 UTC  21min left n/a                          n/a          glance_rsync_images.timer                       glance_rsync_images.service
Thu 2019-03-21 14:20:00 UTC  26min left n/a                          n/a          keystone_delete_expired_tokens.timer            keystone_delete_expired_tokens.service
Thu 2019-03-21 14:30:00 UTC  36min left n/a                          n/a          keystone_novaobserver_delete_tokens.timer       keystone_novaobserver_delete_tokens.service
Thu 2019-03-21 14:40:00 UTC  46min left n/a                          n/a          keystone_novaadmin_delete_tokens.timer          keystone_novaadmin_delete_tokens.service
Fri 2019-03-22 13:30:52 UTC  23h left   Thu 2019-03-21 13:30:52 UTC  22min ago    systemd-tmpfiles-clean.timer                    systemd-tmpfiles-clean.service

6 timers listed.

Now all logs can be accessed by `journalctl -u $unit` and Icinga can monitor failures as well.

If you find any issues, please let me know.

Regards,

-- Giovanni Tirloni Operations Engineer Wikimedia Cloud Services