Tapping the cloud as a software testing service |
 |
By Christina Torode, Senior News Writer
12 May 2009 | SearchCIO-Midmarket.com |
|
|
Many midmarket companies have a backlog of software testing and quality assurance (QA) projects waiting in a queue for months, but cloud service providers are offering a tempting alternative: Put your application development and QA labs in our virtual IT labs.
And companies are jumping on board, according to one recent survey. The survey of 1,771 corporate software buyers listed application testing and development as one of the top five uses for public clouds, according to a study by research firms The 451 Group and ChangeWave. The Q4 2008 survey also found that 60% more respondents said they were going to adopt public cloud computing, compared with the same group asked four months earlier (see box, below).
The cloud holds a huge appeal for application development and QA projects in that instead of projects lining up in backlogs as a result of limited testing resources, IT can tear up and down testing labs and in turn get projects to market faster. Companies don't have to build out their testing infrastructure, which could cost millions since testing environments need to mirror the production environments in which the applications will actually run. Testing environments also sometimes sit unused as application refresh cycles peak and wane.
Mike Casullo, CIO of WildBlue Communications Inc., a satellite broadband service provider with more than 400,000 subscribers and 250 employees, compares the dilemma of limited IT testing resources to having to wait months for a parking spot.
|
| More cloud research |
| View a webcast of "The Sky's the Limit: How Cloud Computing is Changing the Rules," by 451 Group analyst William Fellows to learn more about the research firm's study on cloud computing. No registration is required. |
|
|
|
|
"I was tired of going into meetings and telling everyone that I couldn't get a project into QA this month because the parking lot was full," Casullo said. "I didn't want to build a bigger test environment -- it would cost millions -- so now I buy parking spots as I need them."
Those parking spots are preproduction IT labs for application development and QA testing that Casullo's IT team of 50 tears up and down as needed through a monthly contract with Seattle-based cloud service provider Skytap Inc. WildBlue's application developers self-provision the test environments using a Software as a Service-based virtual lab management application developed by Skytap. The test environment then resides on a shared virtual infrastructure, based on VMware Inc. technology, which Skytap built in its data center and a colocation facility.
The initial virtual lab WildBlue built with Skytap tested applications that are downloaded to customers' computers to enable the satellite broadband connection and allow WildBlue's IT staff to diagnose connectivity problems. Prior to using a cloud-based virtual testing environment, WildBlue's IT staff members had to simulate numerous custom configuration scenarios, including many different operating systems, on their own infrastructure of VMware virtual machines.
The project was in the design phase beginning in August 2007 and in beta by November. If WildBlue had tested the application in-house, Casullo estimated that his team would have still been in the beta phase in May 2008 -- taking six months longer.
Casullo wouldn't put a price tag on the project, but pricing for a virtual lab starts at about $2,500 to $3,000 a month, Skytap said.
|
|
|
|
|
The benefits of the cloud are much more far reaching than hard costs -- how much cost savings do you put on getting
a project out three months earlier?
Mike Casullo
CIO, WildBlue Communications Inc.
|
|
|
|
|
|
|
|
|
|
Next up were virtual test environments for most of WildBlue's custom applications including marketing, finance, customer care and order entry. Overall, Casullo estimated that working with Skytap has lowered his infrastructure costs by more than 60% and saved more than $1 million in upfront capital costs had WildBlue built more IT testing lab resources in-house.
Cost saving is icing on the cake when a CIO sees his time to market increase for every application development, QA and software refresh testing project, however, Casullo said.
"The benefits of the cloud are much more far reaching than hard costs --- how much cost savings do you put on getting a project out three months earlier?" Casullo said. "The business side doesn't care if it's in the cloud, or in a virtual environment. They just want their applications sooner, and they want them to work."
Skytap was the right fit for WildBlue on many levels, but other factors for that fit were that WildBlue's application production environment also resided on virtual machines and the hypervisor of choice, VMware, was used by both parties. Testing of the application as far as virtual machine specifications, network settings, loads and so on in the virtual lab environment reflected the true production environment.
Limitations of the cloud computing test environment
However, a test environment in the cloud is not for everyone. Some cloud providers limit the types of configurations, servers or storage they support, such that they won't mirror a customer's real-world production environment.
"The cloud [test environment] doesn't reflect production environments --applications run on specific [physical] servers, specific virtualization technology, specific networking and bandwidth, and that is hard to replicate," said James Staten, an analyst at Cambridge-Mass.-based Forrester Research Inc. "Then there's the issue of cloud providers not supporting the technology that you use at all, like a specific HP blade server."
On the virtualization side, some cloud providers offer only a vanilla virtualization test environment. "Even in a virtual test environment, you want it to look just like production to see how it behaves," Staten said. "So there will be cases when preproduction level testing still has to be done in-house."
Let us know what you think about the story; email: Christina Torode, Senior News Writer
|
|
|
|
