During the latest OpenStack Summit in Portland, it finally felt like the OpenStack object storage project called Swift had successfully climbed over the wall-of-worry to become a real project. It is no secret that Swift has, so far, been vastly overshadowed by its cousin Nova, the cloud compute project. However with the Grizzly release, Swift has finally come of age and it’s heartening to see broad and strong interest in the project. Meeting rooms were filled with sophisticated audiences asking deep and insightful questions, while the speakers were treated like celebrities after their presentations, which were rich and diverse. There was broad participation on Swift by Seagate, HP, IBM, CloudScaling, SwiftStack, Intel, Korea Telecom, NTT, Nebula, Rackspace, etc.
Highlights of the presentations I attended included:
- Korea Telecom (KT) discussed a set of real world issues they are facing with Swift when depoyed at scale – KT talked about how to improve S3 API compatibility in a module called Swift3 (the middleware that bridges between AWS S3 like APIs to native Swift APIs), data-at-rest encryption, large container DB, and container-container sync issues.
- Intel talked about CoS bench, which is turning out to be the IO meter for object storage. This tool is maturing well and is useful for performance benchmarking.
- We did a joint presentation with Seagate on drive traces and factors that affect Swift performance. I have to admit, we were skeptical about how many people would care about this topic. To our collective surprise, this session had a huge attendance and proved to be a very popular. Of course Tim Feldman, our counterpart from Seagate, is an exceptional speaker, and that might have had something to do with it!
- HP’s presentation on federation of Swift clusters was very interesting about an issue encountered only at a mega-scale. The problem HP is trying to solve was to make multiple clusters still look like one name space.
- IBM discussed ACL and meta-data improvements with the idea to make both sets of functionality a lot richer than today. ACL improvements would allow object level controls which would bridge the gap with S3; while meta-data improvements would help on better rationalization with CDMI (SNIA API for cloud storage).
- Erasure coding for Swift is important for very large scale deployments because it can effectively reduce the 3x overhead of Swift (assuming 3 copies) to ~1.5x with the same or improved durability. Of course, erasure coding comes at a price in that the compute requirements go up – but for very large scale the savings in storage greatly overshadow the need for extra compute. Intel also talked about their erasure coding efforts with regards to Swift.
Net-net Swift is real! More and more companies have put it into production to make sure that the core code-base is mature and solid.
While I can’t reveal specific EVault plans, if you are looking to work on OpenStack projects, EVault is definitely a great place. Our OpenStorage project has the energy of a startup and yet is backed by a multi-billion dollar corporation, our parent company, Seagate. Plus our location could not be more exciting, in the heart of the SoMA district in San Francisco. Email us at firstname.lastname@example.org or check out our careers page.