Software is eating the world: software defined storage edition

"Is Your Storage Vendor Heading Towards the Elephant Graveyard?" was the provocative title of a presentation given by research directors Gene Ruth and Arun Chandrasekaran at the recent Gartner Data Center Conference in Las Vegas. I couldn't make it out to the show myself but one of my colleagues was sufficiently struck by this preso to share it with me.

So what struck us about the presentation?

Certainly not everything discussed was unexpected. Flash memory has finally emerged as an important component of the enterprise storage market. Full stop. Gartner noted that it can even lower storage costs by allowing IT shops to avoid array upgrades and by lowering spindle count. (I'd argue that new technologies coming down the road such as persistent memory are also going to have a big impact but the bottom line is that we're finally moving beyond using essentially one class of storage media for everything besides backup/archive.)

What was less expected was the emphasis on open source and software defined storage (SDS). Now, don't get me wrong. It should hopefully be obvious that I don't need any convincing about the importance of these trends. I do after all work for Red Hat and we bought Gluster, the developers of GlusterFS, a couple of years back. GlusterFS (Red Hat Storage Server is the name of the commercial product) is an open source, high-scale, distributed filesystem that runs on volume x86 hardware. In other words, software defined storage.

But Gartner is often seen as taking a wait-and-see approach to disruptive new technology approaches. Its clients after all tend towards the mainstream to late adopters of technology as another Gartner analyst, Lydia Leong, told me a while back. Therefore, that Gartner is advising clients to put plans in place around storage trends such as SDS, open source, and hybrid clouds is noteworthy.

To be sure, Gartner makes it clear that the full promise of SDS isn't here today. That's fair. SDS is still relatively new even though Red Hat Storage Server is in production use at companies such as Intuit and Pandora. At the same time, though, Gartner doesn't equivocate about SDS's future. It recommends: "Begin evaluating SDS — it's a nascent concept today, but its time is coming."

Among the advantages of SDS that Gartner recognizes are:

• Enables a vendor agnostic hardware infrastructure
• Moves operations toward an SLA delivery model
• Enables new hiring practices and skill profiles
• Challenges conventional data placement thinking

Gartner also recommends that IT shops "insist that incumbent storage vendors develop and enable storage technologies that support hybrid cloud infrastructures."

Venture capitalist and co-founder of Netscape Marc Andreessen famously wrote about how software is eating the world. Software defined storage (like software defined networking) is just another aspect of this trend. The hardware doesn't go away of course. The bits have to sit somewhere. But the intelligence that places, replicates, and mediates the access to those bits is increasingly in open source software rather than in custom circuitry and silicon or the microcode of a proprietary vendor's array controller.