Open source storage is a solution that solves vertical and pain point challenges.
There are many scenarios where it is not necessary. Organisations might just want to hold onto the data they have. In this instance, an S3 bucket or Blobstore API will cover them.
But, implementing an open source storage solution makes sense for targeted use cases. Lustre is a good example if your organisation needs to do some aggressive high performance computing. “For those sorts of workloads, Lustre is extremely mature,” said Stephen Manley, chief technologist at Druva. “It’s extremely well integrated into the ecosystem.”
And, in general, in terms of the maturity curve, the open source storage system itself is quite robust.
Implementing open source storage
Where there are targeted use cases, open source storage can help gain value from data. But, it’s not without its challenges.
Below, Manley outlines his advice to CIOs and CTOs on how they can implement an open source storage solution.
1. It’s not a simple, just plug and play storage
“You do want to make sure that you have at least some expertise or background with it, so that you can configure it properly. Otherwise, you won’t get the performance you expect,” he said.
2. Understand what you’re trying to do
“When looking at different open source storage solutions understand what you’re trying to do,” he continued.
“If you’re trying to run a high performance Kubernetes cluster, Ceph — for example — is probably not a great choice. But, if you’re looking for a bulk storage platform, that makes a lot more sense. The horses for courses still fits here.
“That’s what I’d caution the CIOs and CTOs on.”
A reliance on open source in enterprise: Necessary for digital transformation
3. It’s specific
“Open source storage file systems are very much oriented around solving specific production workloads and not only are they not as general as more traditional file systems, but they also don’t have some of traditional data management functionality built in,” he said. “Things like snapshots and replication are not as mature.”
4. Protect it
Manley summed up, “You have to ask, how am I protecting it? What am I going to do if something goes wrong?
“You’re moving into a world with open source where the ecosystem around it is not as mature than traditional file systems. You’ve got to be much more intentional about making sure that you’ve covered this ecosystem piece.”
Manley’s top open source storage pick
“By far, Lustre (for what it does) is fantastic. It’s not trying to be your file system for every piece of data that you store. But for high performance computing it can’t be beat. There’s a reason why AWS integrated it.
“It’s a well written, architected and designed file system. and I’ve not seen a workload that does better.”
