Hello,
Since I have to re-do a lot of setup for the project anyway, I am considering using Puppet to make it easier to maintain in the future.
Is running own puppetserver (as described in the docs) the only option or is there something easier?
Marcin User:Saper
Hi, It may be an unpopular opinion but I suggest not doing a stand-alone puppetmaster or any of that. It makes sense for production where you have a central system and you need to keep an inventory of thousands of servers and keep them in check.
For WMCS projects (and my projects) something simpler and not requiring a centralized authority (so Chef is also not a good idea IMHO) would make more sense, Ansible is the standard, you can create a playbook repo and just run it but there are also more client-based ones IIRC, there were also Consul, Fabric (Fabrik?), etc. And they wouldn't need you to set up anything specific on the VMs either. They just ssh and do the work.
my 2c
Am Di., 1. Sept. 2026 um 14:02 Uhr schrieb Marcin Cieslak via Cloud < cloud@lists.wikimedia.org>:
Hello,
Since I have to re-do a lot of setup for the project anyway, I am considering using Puppet to make it easier to maintain in the future.
Is running own puppetserver (as described in the docs) the only option or is there something easier?
Marcin User:Saper_______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
On Tue, 1 Sep 2026, Amir Sarabadani via Cloud wrote:
Hi, It may be an unpopular opinion but I suggest not doing a stand-alone puppetmaster or any of that. It makes sense for production where you have a central system and you need to keep an inventory of thousands of servers and keep them in check.
For WMCS projects (and my projects) something simpler and not requiring a centralized authority (so Chef is also not a good idea IMHO) would make more sense, Ansible is the standard, you can create a playbook repo and just run it but there are also more client-based ones IIRC, there were also Consul, Fabric (Fabrik?), etc. And they wouldn't need you to set up anything specific on the VMs either. They just ssh and do the work.
Thank you I agree with everything you say (I';l keep my opinion on Ansible off this list, it might be good for simple things).
There is one thing I wanted to do which could be upstreamed if anyone is interested - I want to decouple the PostgreSQL version from the Debian version by using PGDG packages. Also this could allow me to run multiple major version of the database in parallel which is good for certain upgrade scenarios.
Would that be interesting for the operations/puppet repo?
Marcin User:Saper
This thread illustrates what I consider one of the fundamental problems with our cloud infrastructure: the siloization of the production and volunteer sides of the house. We already have a system in place (puppet) to do the sorts of things Marcin wants to do. It is professionally run. It is monitored. Upgrades are done to it as needed. It is well understood. The obvious solution is to just let Marcin add his project to the existing puppet server and take advantage of all those good things.
Asking him to set up his own puppet (or chef, or ansible, or ...) server means he will have to expend effort that would otherwise be put to doing more useful things. Undoubtedly, once he gets it running, it will be less well managed than the production puppet system. It's just a pointless waste of human effort, and human effort is our most valuable asset.
In the two large-scale hosting systems that I'm familiar with (Amazon and Google), internal production and external customer workloads run on the same infrastructure, taking advantage of the economies of scale. It's sad that we can't do the same thing here.
On Sep 1, 2026, at 8:09 AM, Amir Sarabadani via Cloud cloud@lists.wikimedia.org wrote:
Hi, It may be an unpopular opinion but I suggest not doing a stand-alone puppetmaster or any of that. It makes sense for production where you have a central system and you need to keep an inventory of thousands of servers and keep them in check.
For WMCS projects (and my projects) something simpler and not requiring a centralized authority (so Chef is also not a good idea IMHO) would make more sense, Ansible is the standard, you can create a playbook repo and just run it but there are also more client-based ones IIRC, there were also Consul, Fabric (Fabrik?), etc. And they wouldn't need you to set up anything specific on the VMs either. They just ssh and do the work.
my 2c
Am Di., 1. Sept. 2026 um 14:02 Uhr schrieb Marcin Cieslak via Cloud <cloud@lists.wikimedia.org mailto:cloud@lists.wikimedia.org>: Hello,
Since I have to re-do a lot of setup for the project anyway, I am considering using Puppet to make it easier to maintain in the future.
Is running own puppetserver (as described in the docs) the only option or is there something easier?
Marcin User:Saper_______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org mailto:cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
-- Amir (he/him)
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
I think it's a bit different on puppet vs ansible, the overhead of puppet/chef is quite large but ansible and similar are basically glorified bash scripts that you run from your laptop and it ssh'es into the vm and does things. On top of that, there are many playbooks ready in the wild, so for example if you want to set up an apache service, you can just find an small playbook from the internet.
But I understand that the overhead could be a lot, I think that's why there is also toolforge. IMHO, it's just the trade-off of "ease to deploy" vs "scale/security/reliability". On one end of the spectrum you have toolforge and at the other end you have puppet-managed, helm-deployed mediawiki k8s pods in production that resemble closer to a nuclear reactor than a toolforge tool. Cloud VPS sits in the middle.
On top of that, OpenStack provides some services to VM like trove that provides a db to you out of the box. So that should also take away a bit of the burden.
Am Di., 1. Sept. 2026 um 14:33 Uhr schrieb Roy Smith roy@panix.com:
This thread illustrates what I consider one of the fundamental problems with our cloud infrastructure: the siloization of the production and volunteer sides of the house. We already have a system in place (puppet) to do the sorts of things Marcin wants to do. It is professionally run. It is monitored. Upgrades are done to it as needed. It is well understood. The obvious solution is to just let Marcin add his project to the existing puppet server and take advantage of all those good things.
Asking him to set up his own puppet (or chef, or ansible, or ...) server means he will have to expend effort that would otherwise be put to doing more useful things. Undoubtedly, once he gets it running, it will be less well managed than the production puppet system. It's just a pointless waste of human effort, and human effort is our most valuable asset.
In the two large-scale hosting systems that I'm familiar with (Amazon and Google), internal production and external customer workloads run on the same infrastructure, taking advantage of the economies of scale. It's sad that we can't do the same thing here.
On Sep 1, 2026, at 8:09 AM, Amir Sarabadani via Cloud < cloud@lists.wikimedia.org> wrote:
Hi, It may be an unpopular opinion but I suggest not doing a stand-alone puppetmaster or any of that. It makes sense for production where you have a central system and you need to keep an inventory of thousands of servers and keep them in check.
For WMCS projects (and my projects) something simpler and not requiring a centralized authority (so Chef is also not a good idea IMHO) would make more sense, Ansible is the standard, you can create a playbook repo and just run it but there are also more client-based ones IIRC, there were also Consul, Fabric (Fabrik?), etc. And they wouldn't need you to set up anything specific on the VMs either. They just ssh and do the work.
my 2c
Am Di., 1. Sept. 2026 um 14:02 Uhr schrieb Marcin Cieslak via Cloud < cloud@lists.wikimedia.org>:
Hello,
Since I have to re-do a lot of setup for the project anyway, I am considering using Puppet to make it easier to maintain in the future.
Is running own puppetserver (as described in the docs) the only option or is there something easier?
Marcin User:Saper_______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
-- Amir (he/him)
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Using puppet in cloud VPS does NOT mean you have to run your own puppet server.
You can simply use the existing global puppet server.
Running your own local puppet server is only needed for advanced use cases like wanting to test puppet changes before they get merged in production.
On Wed, Sep 2, 2026 at 8:45 AM Amir Sarabadani via Cloud < cloud@lists.wikimedia.org> wrote:
I think it's a bit different on puppet vs ansible, the overhead of puppet/chef is quite large but ansible and similar are basically glorified bash scripts that you run from your laptop and it ssh'es into the vm and does things. On top of that, there are many playbooks ready in the wild, so for example if you want to set up an apache service, you can just find an small playbook from the internet.
But I understand that the overhead could be a lot, I think that's why there is also toolforge. IMHO, it's just the trade-off of "ease to deploy" vs "scale/security/reliability". On one end of the spectrum you have toolforge and at the other end you have puppet-managed, helm-deployed mediawiki k8s pods in production that resemble closer to a nuclear reactor than a toolforge tool. Cloud VPS sits in the middle.
On top of that, OpenStack provides some services to VM like trove that provides a db to you out of the box. So that should also take away a bit of the burden.
Am Di., 1. Sept. 2026 um 14:33 Uhr schrieb Roy Smith roy@panix.com:
This thread illustrates what I consider one of the fundamental problems with our cloud infrastructure: the siloization of the production and volunteer sides of the house. We already have a system in place (puppet) to do the sorts of things Marcin wants to do. It is professionally run. It is monitored. Upgrades are done to it as needed. It is well understood. The obvious solution is to just let Marcin add his project to the existing puppet server and take advantage of all those good things.
Asking him to set up his own puppet (or chef, or ansible, or ...) server means he will have to expend effort that would otherwise be put to doing more useful things. Undoubtedly, once he gets it running, it will be less well managed than the production puppet system. It's just a pointless waste of human effort, and human effort is our most valuable asset.
In the two large-scale hosting systems that I'm familiar with (Amazon and Google), internal production and external customer workloads run on the same infrastructure, taking advantage of the economies of scale. It's sad that we can't do the same thing here.
On Sep 1, 2026, at 8:09 AM, Amir Sarabadani via Cloud < cloud@lists.wikimedia.org> wrote:
Hi, It may be an unpopular opinion but I suggest not doing a stand-alone puppetmaster or any of that. It makes sense for production where you have a central system and you need to keep an inventory of thousands of servers and keep them in check.
For WMCS projects (and my projects) something simpler and not requiring a centralized authority (so Chef is also not a good idea IMHO) would make more sense, Ansible is the standard, you can create a playbook repo and just run it but there are also more client-based ones IIRC, there were also Consul, Fabric (Fabrik?), etc. And they wouldn't need you to set up anything specific on the VMs either. They just ssh and do the work.
my 2c
Am Di., 1. Sept. 2026 um 14:02 Uhr schrieb Marcin Cieslak via Cloud < cloud@lists.wikimedia.org>:
Hello,
Since I have to re-do a lot of setup for the project anyway, I am considering using Puppet to make it easier to maintain in the future.
Is running own puppetserver (as described in the docs) the only option or is there something easier?
Marcin User:Saper_______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
-- Amir (he/him)
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
-- Amir (he/him)
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/