When I first started doing this for beta I did use a local tracking branch. After about a week I switched to the cherry-pick and rebase model. Keeping the changes as cherry-picks with the cron'd rebase script has a few advantages in my opinion:

* It's easy to see what the delta from upstream is using the log command I showed before (git log --color --pretty=oneline --abbrev-commit origin/HEAD..HEAD). * As things are merged upstream the rebase process drops the cherry-pick. * People should be less tempted to make permanent "local hack" patches.

Ideally the only reason for having the custom puppet master in beta at all is to allow testing of patches that are work in progress before having them merged into the operations/puppet.git production branch and to let us fix things quickly in beta without waiting for ops +2 on the changes. The two cruddy local only patches that are/were there were the result of a need to refactor that upstream wasn't ready for at the time. Now that we have puppet3 and hiera available both of these patches should be resolvable by introducing hiera vars.

Bryan

On Tue, Dec 2, 2014 at 5:53 AM, Mukunda Modell mmodell@wikimedia.org wrote:

Is there any reason we don't have a branch on the git repo to hold these commits so that the beta environment doesn't need a bunch of transient state? This is exactly what branches are good for isn't it?

On Monday, December 1, 2014, Bryan Davis bd808@wikimedia.org wrote:

...

On Mon, Dec 1, 2014 at 4:17 PM, Greg Grossmeier greg@wikimedia.org wrote:

...

Adding wikitech-l

On Mon, Dec 1, 2014 at 2:59 PM, Greg Grossmeier greg@wikimedia.org wrote:

...

tl;dr: Testing on deployment-salt (or other Beta Cluster/deployment-prep vms?): please watch out for "[LOCAL HACK]" commits and don't lose them.

See: https://phabricator.wikimedia.org/T75947

Basically: Some local puppet changes were committed on deployment-salt with "[LOCAL HACK]" in the summary (as is the practice to test/do things before they're merged in ops/puppet for a number of reasons, see: https://phabricator.wikimedia.org/T76392 for reducing that). They were, unfortunately lost somehow. Current theory is that someone accidentally clobbered them while doing their own testing.

If it was you, no worries (Bryan's already fixed this specific case, thanks Bryan!) but please don't do it again. :)

Feel free to talk with the QA mailing list if you have any questions on how to do testing of puppet changes in Beta Cluster: https://lists.wikimedia.org/mailman/listinfo/qa

Best,

Greg

On any given day there are generally quite a few cherry-picked commits on deployment-salt that are being tested and used in the beta cluster prior to being merged into the shared operations/puppet.git repo that powers puppet for labs in general, the beta project (deployment-prep) and our production hosts. At the moment I'm writing this, the list looks like this:

deployment-salt:/var/lib/git/operations/puppet (git production $) bd808$ git log --color --pretty=oneline --abbrev-commit origin/HEAD..HEAD 3353137 logstash: Forward syslog events for apache2 + hhvm ace40a2 Use hiera to configure udp2log endpoint for ::mediawiki dc11da3 logstash: Rules for processing MW input via Redis 22837cd Configure Logstash and Elasticsearch for ApiFeatureUsage 53d8b58 Change eventlogging log dir bbaa10b eventlogging: couple less tightly to ganglia 48f60fa Fix Parsoid in beta 1d58f04 Get betalabs localsettings.js file from deploy repo (just like prod) 4e03e67 Allow puppetmaster to send reports to logstash caef989 [LOCAL HACK] T67591: User['mwdeploy'] shell => /bin/bash a22bbec [LOCAL HACK] T47706 Change MySQL admin user in sql script

There is a pretty good write up on wikitech [0] on the safe process for adding a new cherry-pick to the beta puppet master and removing one that is no longer wanted. I'd be glad to help anyone who has questions or problems with the process as well.

Bryan

Bryan Davis Wikimedia Foundation bd808@wikimedia.org [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

---

Engineering mailing list Engineering@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/engineering