###How to remove OSD from Ceph cluster It is not well described in the docs.
- Look into OSD list (ceph osd tree). Select one you want to remove. Let’s say it is an ‘osd.11’.
- Mark it ‘out’: ceph osd out osd.11 If you see “osd.11 is already out” — it’s ok.
- Mark it as ‘down’: ceph osd down osd.11
- Remove it: ceph osd rm osd.11. If it says ‘Error EBUSY: osd.11 is still up; must be down before removal.’ that means OSD is not dead yet. Go to the host it resides on and kill it (systemctl stop ceph-osd@11), and repeat rm operation.
- Now it would list in ceph osd tree with ‘DNE’ status (DNE = do not exists). To clean up this status, remove it from CRUSH map: ceph osd crush rm osd.11
- Last step: remove it authorization (it should prevent problems with 'couldn’t add new osd with same number’): ceph auth del osd.11. Please note: in the real life you should be really careful not to damage data on the OSD, so this instruction should be used only for dead OSDs without any hope for recover.