In my daily reboots, I'm coming across a few weird VMs that should be on ceph but instead are using local storage on hypervisors. I've tracked this down to particular flavors which don't specify the Ceph backend. For example, here's a 'good' flavor:
root@cloudcontrol1003:~# openstack flavor show g3.cores4.ram8.disk20 +----------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +----------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 20 | | id | c14b5856-5e6a-4f7a-8125-ace4f616c299 | | name | g3.cores4.ram8.disk20 | | os-flavor-access:is_public | True | | properties | aggregate_instance_extra_specs:ceph='true', quota:disk_read_iops_sec='5000', quota:disk_total_bytes_sec='200000000', quota:disk_write_iops_sec='500' | | ram | 8192 | | rxtx_factor | 1.0 | | swap | | | vcpus | 4 | +----------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
...and here's a bad one...
root@cloudcontrol1003:~# openstack flavor show g3.cores24.ram122.disk20 +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | ['wikiwho'] | | description | None | | disk | 20 | | id | 6207fade-5517-4ac4-a147-6ae0c8fa1384 | | name | g3.cores24.ram122.disk20 | | os-flavor-access:is_public | False | | properties | | | ram | 124928 | | rxtx_factor | 1.0 | | swap | | | vcpus | 24 | +----------------------------+--------------------------------------+
Note the lack of properties set on the latter.
We should definitely investigate whether to have flavors without properties have good defaults -- at this point we're setting those same properties for almost every flavor which seems silly. In the meantime, though, it's important that those properties be specified on new flavors, otherwise VMs with those flavors can be consigned to purgatory and weird behavior.
The docs about flavor creation are here:
https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/VM_flavors#Genera...
most likely that should be duplicated or linked from other places though.
Thanks!
-A