I this is both expected and a bug :D

There is an option to disable port security for a Network.  This is potentially the right thing in a world where Tooling here is really Toolforge and we want ferm/iptables to manage rulesets and not be overlapping with security groups.  If this is its own VXLAN network that could make sense.

So to test this I issued:

# optional mass turning off of port-security mode for a network

neutron net-update --port-security-enabled=false <net uuid>

This documented in modules/openstack/templates/bootstrap/neutron/neutron_seed.sh.erb

port-security (security group application) can be disabled per-port or per-network.  Per-port seems to be fairly well worked out, but per network seems to come with confusion between nova and neutron.  Nova assumes in some cases that every port will have a security group and so balks.  I see https://review.openstack.org/#/c/59578/ in relation but haven't found a similar bug for disabled for an entire network but I'm sure I've seen one I just can't find it.  If only we could read redhat's magic docs at https://access.redhat.com/solutions/2143121.  Possibly https://bugzilla.redhat.com/show_bug.cgi?id=1291210

I don't recall doing anything special to create the existing instances so that's interesting, and this seems not to happen if we first make a port with security groups disabled and then attach an instance.  So in general I think this is just mismatched expectations between nova and neutron when security groups are disabled for an entire network.  There a few known upstream reports of similar things.  

On Fri, Jun 1, 2018 at 7:43 AM, Arturo Borrero Gonzalez <aborrero@wikimedia.org> wrote:

Hi,

I've been working on some sanity checks for our labtestn
(mitaka/neutron) deployment [0].

I found a weird issue when trying to create an instance in the 'tooling'
network (the vxlan based network):

<<
Network requires port_security_enabled and subnet associated in order to
apply security groups.
>>

This error is seen in the /var/log/nova/nova-conductor.log file when I
create an instance with this cmdline:

% openstack server create --flavor 2 \
--image 66e544e8-fe4f-41f7-9809-6723e53b5a99 aborrero-test1 \
--nic port-id=9389a984-d58e-4776-8b7a-30ff93073917 \
--property subnet=3ec06de7-3b9e-4de3-86c6-67ba1895b253

However:

% neutron port-show 9389a984-d58e-4776-8b7a-30ff93073917 \
| grep port_security_enabled

| port_security_enabled | True

(this port was created manually by me)

Not sure if the 'server create' command is lacking some additional
option. I generated it following what I saw in our bootstrap docs +
labtestcontrol2003 history

I also tried with this command:

% openstack server create --flavor 2 \
--image 66e544e8-fe4f-41f7-9809-6723e53b5a99 aborrero-test1 \
--nic net-id=60aa9467-253c-4fdf-9fa0-eba42dafc975

with the same result (i.e, a net instead of a port)

I'm probably misunderstanding some openstack concepts: nics, ports,
subnets, nets, etc.

[0]
https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Deployment_sanity_checklist

--

Chase Pettet

chasemp on phabricator and IRC