Vault 2015

Ceph is a massively scalable, open source, software-defined distributed storage stack, which uniquely provides object, block and file access interfaces in one unified system. Most of the current Ceph deployments use SSDs for Ceph journal and hard drives as Ceph data store. But when the hard disk backend was replaced by an all-SSD infrastructure, multiple CPU bottlenecks in Ceph code were exposed. Major improvements were done to the Ceph OSD code to eliminate these bottlenecks. Some of the notable changes were removing a single dispatch queue bottleneck, a new sharded worker thread pool for efficient request processing, improved lock granularity, sharded file descriptor cache etc which resulted in performance improvements of 6X for block reads and 5X for object reads.