It was the threat of Hurricane Charley that convinced the Charlotte County Florida Government to move to a virtualized data center.
In 2004, just before the storm hit, the county's IT department moved 10 servers and associated storage from its data center in Port Charlotte, Fla., to a hardened backup facility located inside a nearby jail.
Fortunately, Mark Ramsey, manager of IT operations, anticipated the storm's seriousness early enough to accomplish the shift in time. Nevertheless, the county's employees suffered downtime: three hours after the UPS battery died, plus "about 12 hours spent breaking gear down, transporting it, putting things up."
On top of that, the jail's air conditioning system broke down, causing the servers to overheat before the UPS ran out of juice. Some data got corrupted, and the county's maintenance contract became invalid, Ramsey said. "We definitely didn't want this to happen again."
The county's solution: a "disaster avoidance" strategy based on VMware Inc. virtualized servers and LeftHand Networks Inc.'s iSCSI-based virtual networked storage platform.
As a midsized organization, Charlotte County is at the forefront of a trend that's already well established among enterprises -- three-quarters of which now use virtualization for production applications, according to a recent report from Boulder, Colo.-based Enterprise Management Associates (EMA) Inc.
Palo Alto, Calif.-based VMware's ESX software sets up multiple, software-defined virtual machines (VMs) on a single physical x86 computing device. Tasks and applications can then be shared and load balanced across VMs residing on one or more physical servers, including servers in different locations.
On the storage side, Ramsey replaced the county's 5-year-old Fibre Channel storage area network with LeftHand's Virtual Storage Appliance. Unlike Fibre Channel, IP-based iSCSI "doesn't require a special skill set," he said. "It's the same type of network traffic we've been moving all along." Furthermore, IP-based storage, voice and video traffic can run over the same fiber optic connection, whereas with Fibre Channel, the county had to pay for two dedicated strands of dark fiber.
Another plus, Ramsey said, was LeftHand's modular design, which enabled Charlotte County to upgrade capacity in small increments. "I used to have to estimate and pay for the storage and I/O capacity I'd need for the next three years, which is tough: Things change," he said.
Most importantly, LeftHand's storage management system is fully integrated with VMware, which is crucial to Charlotte County's disaster avoidance strategy.
By February 2007, the county had deployed a new, hardened data center 12 miles away from the main facility. The two centers are connected over a 10 Gbps Ethernet fiber optic link. During normal day-to-day operations, VMware servers at each center support a different set of users and applications. But if one data center goes down, VMware's High Availability and VMotion tools recognize the problem and automatically transfer VMs and the tasks they're working on to the other facility, with no interruption to users or applications, Ramsey said. When the downed data center is back online, VMware automatically shifts the load back.
On the storage side, LeftHand's two-way replication keeps the two centers in sync. The vendor's SAN/iQ software manages storage on both centers as a single volume so if one site goes down, the VMs that get booted up on the second site obtain immediate access to all relevant data for their tasks, Ramsey said.
Virtualization has saved Charlotte County big on hardware costs, Ramsey said. The county's original server farm consisted of 80 Intel machines, which averaged 3% to 5% utilization, he added. Now, 144 VMs running on 10 IBM LS41 blade servers do the same work. The air conditioning system was also downsized by about 50%, he said.
Virtualized storage is a crucial adjunct to virtualized computing, because it makes data and storage capacity available to applications and processes, independent of which server they happen to be running on, noted Michael Karp, a senior analyst at EMA. Since virtual systems are highly dynamic, with processes and VMs often moved from one physical location to another, backup systems also need to be location-independent, he added.
In the past year, major players like EMC Corp., Hewlett-Packard Co. and Brocade Communications Systems Inc. have all announced VMware ESX support for their networked storage virtualization offerings. Unfortunately, virtualization management tools and skills "are lagging well behind needs," Karp noted.
 |
|
|
|
|
It doesn't
have to be
a hurricane.
A UPS could go offline, or we could have a false fire alarm.
Mark Ramsey
manager of IT operations, Charlotte County Florida
|
|
|
|
|
|
|
|
|
|
In a 2008 EMA survey, IT professionals rated the gamut of virtualization management disciplines, including security, performance, problem, capacity and configuration management, as significantly harder than they did in 2006. The reason: Enterprises have been using virtualization long enough now to "realize the real difficulties of virtualization management," the report states.
A lack of tools makes it hard to pinpoint the cause when there is a sudden slowdown, or data is suddenly not available, Karp said.
Ramsey agreed: "The only area that's been frustrating [about the LeftHand storage setup] has been no built-in tools for managing and monitoring performance areas like input/output per second, bandwidth and throughput." His team depends on VMware performance tools to identify bottlenecks.
A product released by LeftHand today could address that issue. SAN/iQ 8 includes, according to the vendor, an integrated performance manager designed to identify bottlenecks associated with increased network and storage resource loads within virtualized environments by analyzing performance from the application server down to a specific volume or snapshot.
On the other hand, managing backup and recovery in a virtualized environment remains a tough gig, Ramsey acknowledged. "Without virtualization, you simply point a backup system at a server, and back it up. But VMs aren't necessarily tied to a physical server, which adds a layer of complexity to backup and recovery process."
That's why a high-availability disaster-avoidance strategy is so crucial in a virtualized data center setup like Charlotte County's. It minimizes the likelihood that data or processes will need to be recovered after a disaster hits.
Ramsey has a lot less to worry about, knowing that when a disaster takes out a server, or even an entire facility, critical applications will just keep on ticking. Because disasters do happen, he said. "It doesn't have to be a hurricane. A UPS could go offline, or we could have a false fire alarm." Recently, for example, dust got into the big air conditioning units at the brand-new hardened data center, abrading the belts. Under the old setup, Ramsey's team would have had to horse all the servers over to the backup facility -- again.
Not this time. "We moved all the VMs off the data center and shut it down, and ran everything off the second data center while we cleaned out the gear," Ramsey said. The relocation took only 30 to 45 minutes. Most importantly, "when we fired it up [after cleaning], everything was where it was supposed to be."
Elisabeth Horwitt is a contributing writer based in Waban, Mass. Write to her at editor@searchcio-midmarket.com.
