I’m all for option two, because it is what we’ve been doing in general.  Also note: I’ve been segregating anything “grid” into a sub-profile and sub-role because we dream of deprecating it one day, and it will be much easier if it all lives in one folder.  Additionally, the profile classes end up quite long no matter what we do.  I think it is just as well if one more folder isn’t in the list.  I think this is both what we are doing now (since I refactored my roles for grid quick) and pretty darn good.

I think refactoring to option 1 may be more consistent, but we would have to redo a lot of work for both the openstack profiles (which really should follow the same model) and working toolforge sonofgridengine stuff.  I think that, for now, that would be a distraction because of how interwoven the files, templates and profiles are for both gridengine and openstack (and I’ve simplified the gridengine ones a bit!).  Option 2 is what openstack does, and sticking to that (and what has already been started) seems great to me.  If prod started up an openstack project that clashed, we’d have to revisit.  The same if someone else started using the name toolforge in puppet, but that seems very unlikely.

On option 3:  As is, each grid role includes about three profiles.  These profiles include one another to construct what the old toollabs modules used to.  This allows flexibility and reuse of code there.  The services modules are a bit simpler in some ways, and it would be easier to try option 3 with them, but it also feels less consistent overall.  It adds a level of obfuscation and abstraction.  To find a functional profile, I’d have to look from role, to profile to yet another profile.  As is, finding anything in puppet typically requires more than one search because things are so organized.  Coming from a saltstack and chef background, I couldn’t read our puppet at all for at least a month when I started here :)  I feel like option 3 adds a small, but additional step in that process.

Brooke Storm
Operations Engineer
Wikimedia Cloud Services
bstorm@wikimedia.org
IRC: bstorm_

On Oct 26, 2018, at 8:19 AM, Arturo Borrero <aborrero@wikimedia.org> wrote:

Hi,

Brooke already started refactoring puppet code for toolforge-stretch,
and both Giovanni and me are following with other side of the code
(currently, service nodes T207591).

I'm starting with the refactor myself and I have some doubts on how to
better organize the code.
I think it's really important we agree on how to do this beforehand,
and also I may lack some knowledge on best practices etc, so I'm
sharing some ideas for you to comment on.

The following wikitech page contains 3 model options I think can be applied:

https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/puppet_refactor

Brooke has probably some ideas on this, since she already started.

Please comment, and forgive my ignorance if the answer to my questions
are obvious.
Regards.

PD: CC'ing Chase, because he did the massive CloudVPS refactor in the past.

_______________________________________________
Cloud-admin mailing list
Cloud-admin@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/cloud-admin