The web sites of technology industry analysts and consultants are awash with surveys of disk storage usage. All tell the same story: most businesses barely make use of half their disk storage capacity; in some cases it’s as low as 10% to 20%.

With the call for better ‘IT governance’ becoming ever louder, and with user demand for storage as insatiable as ever, those kind of utilisation rates look downright negligent.

The historical reasons for the situation are well-documented: in most cases, businesses have grown their storage systems in a piecemeal way. As new applications are introduced, or as

existing applications look in danger of running out of space, system managers have simply added standalone storage units.

Moreover, storage capacity has been geared to the peak loads that each application requires – when a finance package is being used to roll up year-end numbers or when a retail package is having to deal with a seasonal sales spike, for example. In case such applications even come close to saturation point, more storage capacity is factored in to provide ‘headroom’. The rule of thumb, which has become something of an industry joke, is ‘provision for peak capacity, then double it’.

The heart of the problem lies in the traditional relationship between the server and its dedicated storage, and between the location of stored data and users’ ability to get at it. For decades, only one model persisted – directly attached storage (DAS) – where individual storage units are tied to, or even in-built within, a specific server. Aside from the inherent wastage, that model has also created several other headaches: if either the server or the storage fails, both become useless; providing user access to the dispersed data has required cumbersome copying and data movement; and it makes the management of back-up complex.

Enter the SAN

The solution has been a long time coming. By linking all the storage resources over a network, not only is data more accessible, but the failure of one unit has only a moderate impact, utilisation rates can be much higher, and back-up and restore can be carried out in a more centralised fashion.

Over the past few years, many IT organisations have been building such storage area networks (SANs), pooling their storage arrays and pumping data around the network using specialist switches and data servers or controllers.

Early evidence suggests SANs are dramatically improving storage utilisation rates and allowing companies to consolidate previously dispersed data on larger systems. Additionally, the new networks are reducing storage staff levels as each administrator can now handle much larger amounts of storage.

“Take the scenario where a business only has internal storage [drives within each server],” says Josh Krischer, a storage market analyst at technology research house Gartner. “If the customer needs more capacity, it means physical installation of disks and all the back up that would precede that. Two or three people will have to work on the installation and during that time the application is off-line. With a SAN, you can do that with a mouse click or even dynamically.”

Savings on systems management costs are crucial to the case for storage networks as adopting a SAN architecture involves a large upfront outlay. But businesses, especially larger businesses, appear to be convinced the outlay is justified.

“The choice of a SAN is a foregone conclusion for many of our mid-sized and large customers,” says Mark Lewis, chief technology officer of high-end storage systems vendor EMC. “They are not buying direct attached storage [any more] because it is too hard to manage and because of its low efficiencies. The main reason companies are moving to SANs is in order to meet their main business requirements for lowest cost.”

SANs have other advantages too. As data is shared between a number of disk units, hardware failures are easier to fix. A SAN is configured so that there is more than one copy of the information, so even if one drive or a drive array fails, there is a standby copy of the information.

Administrators should also find it easier to back up a SAN, as data is first copied to disk as a ‘snapshot’ – even as the applications are still running – and then the information is copied to lower-speed, lower-cost disk systems, to tape or it is replicated across the network to a second site.

And, given that there is no sign that data growth rates are going to ease, it is important that SANs should be easier to upgrade. Adding additional storage arrays to a SAN does not mean shutting down the whole network, and new capacity can be distributed between multiple applications.

Pick and mix

Although many of the technologies that support SANs have been around for over half a decade, implementing a SAN is still complex, and, aside from the upfront cost justification, there are still plenty of obstacles that can prevent organisations obtaining all the promised business benefits.

“Storage area networks are now very mature,” believes Paul Hammond, solutions consulting director at CNT, a specialist storage networking company. “But one of the problems we are often asked to solve is how can a business avoid being locked into

a single vendor’s SAN set-up? Organisations don’t want to risk having to pay through the nose for future upgrades,” says Hammond, “but products are now coming onto the market that can help prevent that.”

The prospect of building a SAN from different suppliers’ hardware and software has been pioneered by industry bodies such as the Storage Network Industry Association (SNIA), the main source of the necessary interoperability standards. SNIA’s Bluefin standard sets out to enable organisations to mix and match compatible SAN hardware, as well as to make SANs easier to manage.

Most storage hardware and software companies back the standard, although products that comply to the standard are not yet widely available. Alongside their support for Bluefin, some vendors are already shipping their own interoperability tools, notably EMC with its WideSky middleware that aims to keep in step with Bluefin.

Even where businesses source their storage network from a single supplier, though, there can be problems. Setting up and configuring a storage area network requires a good deal of skill, even where the project is on a green-field site and does not incorporate existing storage modules.

Putting together a SAN means ensuring compatibility between storage controllers, data switches, storage arrays and storage resource management software. As that might suggest, even so-called single vendor SANs will use some elements sourced from third parties.

Then there is the matter of building the physical network. To ensure data access and movement is adequately rapid, most SANs today are deployed using fibre channel networking. So companies need to graft on new skills in this area or seek specialist support from vendors or systems integrators.

“We recommend that if you don’t know how to put together a SAN you do it with a vendor or integrator,” advises Josh Krischer at Gartner Group. “If you do want to do it yourself, at least use tested and certified components. There are enough out there.”

Making sure the components all work together is a major concern and has given rise over the past year to compatibility testing. Several of the larger vendors have invested heavily in compatibility labs, where customers can test specific combinations of their hardware and software before deploying a SAN.

Sun Microsystems, for example, with three storage testing labs (in Scotland, the US and Asia), does not charge customers who want to test their set ups, even in cases where they plan to use a mix of Sun and non-Sun storage equipment.

“As much as we love this technology, we can overestimate the sophistication of the customer,” cautions Tom Georgens, president of LSI Logic Storage Systems, a storage networking specialist. Customers have also been adopting SANs in a phased manner, proving the concept in certain parts of their business before they go for more universal implementation. “The number of true SAN applications is still a subset of all storage systems and heterogeneous SANs are just a percentage of that,” says Georgens. He suggests that rather than try to match hardware and software across the board, businesses should use “fixed, certified installations” to ensure compatibility.

Cost points

Above all, though, cost remains a hurdle – especially for mid-sized and smaller companies, which may not already use the high-end storage arrays that form the backbone of most SANs. With fewer servers and storage arrays to consolidate, they often lack the economies of scale necessary for the investment in a SAN architecture, especially as their integration costs might not be dramatically lower than those experienced by larger companies.

And, although the cost per megabyte of disk storage has fallen dramatically over recent years, the cost of SAN equipment has not fallen as far or as fast. Major cost savings through the re-use of existing storage capacity has also proved elusive. On paper, it should be possible to re-deploy older storage arrays in a SAN, especially for applications that do not depend on high performance. But incompatibility and integration costs have often stood in the way.

“You need to be pragmatic about reuse,” suggests EMC’s Mark Lewis. “Some infrastructure reuse will be possible but using older arrays and networking hardware is a difficult trade off. It can make sense, but in most cases it does not, mainly because of the sheer amount of technical change.”

For new data centre deployments in larger businesses, though, there is little doubt that SANs represent an opportunity. Industry figures show that networked storage now accounts for more than half of the new capacity being installed. And as technology initiatives such as Bluefin become more widely adopted, SAN technology will become the default architecture for large organisations and it will progressively find its way into the mid-market too.

See also: SANs are now beginning to fill one of their biggest promises

The benefits – of greater capacity utilisation, easier and more cost-effective management and greater resilience – outweigh the obstacles that still stand in the way of the storage area network.