Woo!  Ok I think we are done.  Sorry I didn't email about each machine...it didn't quite work like that.  Puppet is happy, and all 3 stat nodes now have internal IPs on the Analytics VLAN.

  stat1001.eqiad.wmnet 10.64.21.101
  stat1002.eqiad.wmnet 10.64.5.102
  stat1003.eqiad.wmnet 10.64.36.103

Note that you will now need to addess stat1003 (and stat1001) via their backend network names.  the *.wikmedia.org addresses no longer exist.  You will need to ssh into them via a bastion now.

There will likely be problems that crop up in the next week or two.  If you notice any jobs that aren't able to connect to something, or don't work anymore like they used to, this change is probably the cause.  Let me know asap and I'll see what I can do.

Note that by being inside the Analytics VLAN, these nodes are now firewalled off from the rest of the WMF production networks.  If you need access to something you don't already have access to, we'll have to explicitly whitelist it.

Many thanks to Brandon Black for all his help today!

-Ao



On Thu, Dec 18, 2014 at 9:58 AM, Andrew Otto <otto@wikimedia.org> wrote:
Hi everybody!  Reminder that this is happening today!

I will try to make sure downtime is as small as possible.  I will email as each machine is back up and ready for use.

On Mon, Dec 8, 2014 at 11:20 AM, Andrew Otto <aotto@wikimedia.org> wrote:
Ok, I haven’t heard any strong objections, and unless I get any before the ops meeting in 3 hours, I will schedule this for Thursday, December 18th.

Also, there was some discussion in another thread about the usefulness of stat1003’s public IP.  Ops wants this go to away, and I tend to agree with them.  It still possible to reach out to the internet on machines with private IP using the webproxy:

https://wikitech.wikimedia.org/wiki/Http_proxy

The only trouble I have had is when trying to reach HTTPS resources.  I will ask ops about how to get around this.

That said, are there any strong objections to removing stat1003’s public IP?

-Ao




> On Dec 5, 2014, at 11:10, Andrew Otto <aotto@wikimedia.org> wrote:
>
> Hi all!
>
> Ops would like us to move the stat* boxes inside of the analytics VLAN.  I need to just pick a date for this to happen.
>
> I’m not entirely sure how long this will all take, so I’d like to schedule an entire day for these to potentially be offline.  How about Thursday December 18th?  If there are objections, I can find another day.
>
> Thanks!
> -Andrew Otto
>
>