Hi Arzhel,
I'm working on writing down required changes to our setup to introduce BGP routing in CloudVPS transport network (between the Neutron virtual router and the Core Router).
It would be great if you can write details of what you need from us, and a recommended procedure/timeline for doing the changes. Example of stuff I'm expecting:
* Neutron would need to use BGP protocol version X (I ignore this bit) * We can only recv x.x.x.x ranges, and not other ranges * we'd need to do the switch from static to BGP only on a concrete day * You need to use AS # NNN * Make sure protocol X is allowed in any firewalling for the session to be correctly established (or monitoring or whatever)
My idea is to merge your ideas/requirements with what Neutron docs say. The docs will end up here:
https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhancemen...
For now, let's assume everything will happen in codfw (in our codfw1dev deployment).
Hello!
Let me know if this answers all your questions, don't hesitate to ask if not.
Wikinetwork codfw side: AS65002 cr1-codfw IP: 208.80.153.186 cr2-codfw IP: 208.80.153.187
Will import (accept) prefixes: - 185.15.57.0/24 - 172.16.128.0/21 (can be adjusted as you wish)
Will export a default route
WMCS codfw side: AS64711 cloudnetX IP: cloudnetY IP:
BFD if availlable: 3*300ms
The transition from static to BGP is impact-less and can be done anytime (with a small notice to prepare the config). Steps are: 1/ configure BGP and verify it's behaving as expected At this point the static routes will still have the priority 2/ Remove the static routes and VRRP/VIP config Need BGP (and optionally BFD) ports open and listening to the routers IPs.
Cheers
On Tue, Feb 18, 2020 at 12:38 PM Arturo Borrero Gonzalez < aborrero@wikimedia.org> wrote:
Hi Arzhel,
I'm working on writing down required changes to our setup to introduce BGP routing in CloudVPS transport network (between the Neutron virtual router and the Core Router).
It would be great if you can write details of what you need from us, and a recommended procedure/timeline for doing the changes. Example of stuff I'm expecting:
- Neutron would need to use BGP protocol version X (I ignore this bit)
- We can only recv x.x.x.x ranges, and not other ranges
- we'd need to do the switch from static to BGP only on a concrete day
- You need to use AS # NNN
- Make sure protocol X is allowed in any firewalling for the session to be
correctly established (or monitoring or whatever)
My idea is to merge your ideas/requirements with what Neutron docs say. The docs will end up here:
https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhancemen...
For now, let's assume everything will happen in codfw (in our codfw1dev deployment).
-- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation
On 2/18/20 8:48 PM, Arzhel Younsi wrote:
Wikinetwork codfw side: AS65002 cr1-codfw IP: 208.80.153.186 cr2-codfw IP: 208.80.153.187
WMCS codfw side: AS64711 cloudnetX IP: cloudnetY IP:
We might want to establish the BGP session in the transport network, between the neutron virtual router and the CRs.
Diagram of the current setup for the transport network:
https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Neutron#/media/Fi...
(the diagram is for eqiad1, but is mostly the same for codfw1dev, with subnet being 208.80.153.184/29)
As you can see, there are just a couple of addresses on that subnet: * 208.80.153.185 (cr-codfw virtual address of some sort) * 208.80.153.190 (cloudinstances2b-gw.openstack.codfw1dev.wikimediacloud.org)
Shall we just establish the BGP session using these 2 addresses? Honestly on the neutron side I don't have any more addresses, so at least the CR should be configured to peer with 208.80.153.190 only (unless in future tests I discover there are other ways of doing this).
regards.
On 2/20/20 7:18 PM, Arturo Borrero Gonzalez wrote:
Shall we just establish the BGP session using these 2 addresses? Honestly on the neutron side I don't have any more addresses, so at least the CR should be configured to peer with 208.80.153.190 only (unless in future tests I discover there are other ways of doing this).
My assumptions were wrong. Apparently neutron ignores the software defined transport network and uses the physical network for doing the BGP stuff.
This is the neutron BGP agent trying to contact the core router:
11:05:18.615101 IP 10.192.20.10.34716 > 208.80.153.185.179: Flags [S], seq 2498112052, win 29200, options [mss 1460,sackOK,TS val 1130299297 ecr 0,nop,wscale 9], length 0
For tests, I configured the BGP peer to contact 208.80.153.185 which is vrrp-gw-2120.wikimedia.org (i.e, the core router VIP in the cloud transport network). Note how the source address is 10.192.20.10 (cloudnet2002-dev.codfw.wmnet).
So I'm discarding my approach and creating the BGP session using the physical network. You should allow BGP from the following addresses:
* cloudnet2002-dev.codfw.wmnet: 10.192.20.10 * cloudnet2003-dev.codfw.wmnet: 10.192.20.12
And I will contact:
* cr1-codfw IP: 208.80.153.186 * cr2-codfw IP: 208.80.153.187
as you originally suggested.
regards.
cloud-admin@lists.wikimedia.org