The potential for information technology to substantively add value and change the competitive landscape for thousands of companies has never been greater. New technologies are coming online, vendors are more willing than ever to negotiate, and investment money is starting to flow again.
Yet the ability of IT leaders to speak convincingly about measuring the business value of their contributions has never been more suspect. Technology executives need to get much better at showing and telling the value of IT. We need to "talk" the IT value "walk."
For this CIO Habitat report, we asked IT execs participating in the recent Las Vegas CIO Boot Camp -- plus CIOs attending the May Forum of UCLA's IS Associates -- to talk about the metrics they use to document IT value. As expected, the vast majority of metrics discussed were machine-based measures of device utilization and system availability. This is Metrics 101. To graduate to the next class of IT measurement, we must find a way to reflect the business value being created.
The Bible for Measuring IT Value Hasn't Been Written
When Adam Smith published An Inquiry into the Nature and Causes of the Wealth of Nations 229 years ago, he set forth a theory of value still used around the world. Its guiding principle was that objects and/or services are valuable because of scarcity or because of the amount of labor embodied in them.
Unfortunately, IT has no corresponding Adam Smith. While many have tried, the bible of IT value has not yet been written. Most organizations don't have the metrics, methods or mind-set to authoritatively dissect a proposed technology solution and expose the future value it will deliver. Very few CEOs today are involved in developing IT metrics (see "Changing the Conversation on Measuring IT Value"), and most companies (82%) don't use any outside providers to track such metrics.
Indeed, an environmental truth that eats away at the credibility of our profession is that many IT organizations aren't even all that competent at specifying the cost of IT projects, let alone the value they might generate. This is a long-lived problem that must be fixed.
Kelly Williams, executive vice president and CIO at San Jose, Calif.-based mortgage industry innovator First Franklin Financial Corp., knows the value of metrics. His industry (sub-prime mortgage financing) didn't really exist until Fair Isaac Corp. came up with a metric (the FICO score) to reliably measure the credit scores of potential borrowers. Credit scoring is a method of determining the likelihood that credit users will pay their bills, and a credit score condenses a borrower's credit history down to a single number.
Wouldn't It Be Great If We Had Such an IT Value Score?
"While there obviously are many tools and processes for measuring 'how' IT works, I have seen very few used in a way that effectively measures the value or contributions that IT makes to the business," Williams notes. "This is a source of great frustration."
In the absence of an internal process for monitoring value, many enterprises benchmark their costs. For example, RadioShack benchmarks costs as half the value equation, with return on key projects measured as the other half, says former CIO Evelyn Follit.
CIO Habitat research confirms that very few organizations have a formal way to measure IT value, although most (60%) do benchmark various IT performance measures. Why is this?
MIT Media Labs' Michael Schrage, who is co-director of the e-markets initiative there, says he's seen numerous efforts to measure IT value over the years, but "nothing really impressive."
"I'm pushing organizations to use 'process metrics' as a bridge between technical capacity and business capability," Schrage says. "The metrics I see that I take seriously are the Web-based ones that measure customer satisfaction with the Web site interaction."
Many CIOs, sensing political danger, avoid the value metric issue entirely. As one midmarket CIO at a manufacturing company puts it, "We use wet fingers in the air now to ensure we are going with the wind."
"It's not the IT metrics that matter. Tying business metrics with IT performance matters more," adds consultant Priscilla Emery of e-Nterprise Advisors in Altamonte Springs, Fla. "The other departments within the organization tend to take credit for increased sales through Web access, increased volume because of marketing and better products, etc. It's tough to measure IT's value unless it is part of the business equation at the beginning of a new product or service lifecycle."
It Takes a Village to Create a Viable Value Metric
Tom Mantz is no stranger to measurement. Upon graduating from Harvard Business School, he worked with business author and Harvard professor Dick Nolan, whose respected "stages theory" is perhaps the most widely used management framework for IT planning. He also worked with David Norton and Bob Kaplan in the early phases of their collaboration on what led to the creation of the "balanced scorecard" management approach. He took this "measurement thinking" into the real world as the (now-retired) CIO of $6.6 billion industrial gas company Praxair Inc. He observes that:
- In places where successful metrics exist, they are developed jointly with business
- Black and white numbers alone do not carry the day. There is always some accountant somewhere
who can disprove the numbers if given the opportunity.
- Internal auditors should be involved in the process. They are often the "scorekeepers of last
- If IT uses a tool in isolation from the business, the numbers generated -- no matter how rigorous -- will not be accepted outside of IT. Excel spreadsheets work just fine "if everyone is in the game together."
Williams at First Franklin Financial would agree. He believes that if there are metrics, they must be relevant to both the IT and business communities.
"I am firmly in the camp that IT should both share in the business success of an effective IT initiative, and also share in the lack of success for an ineffective initiative, as measured by financial (or other business-oriented) results," he says. "My experience is that there is generally little real support for that notion, from either the IT or the business sides of the house. The former tend to be focused (and compensated) on simple delivery: 'I did what you asked, on time and within budget; therefore, I am successful.'"
Yet in trying to measure IT's unique contribution, are we doing the right thing? Measuring value is a good thing. Overmeasuring microvalue may actually be wasteful.
In our increasingly interdependent world, where people, processes and technologies are viewed as the base elements of value-creating alchemy, teasing out IT's unique and separate value contribution may be overkill. Doing so may be like trying to figure out which of the two hydrogen atoms or the oxygen atom make the most important contribution to a sip of water. Measurement is addictive. We want to make sure we are measuring on the correct side of the decimal point.
Joe Puglisi, CIO at construction and engineering firm Emcor Group Inc. in Norwalk, Conn., oversees a federated group of midsized regional businesses. From his vantage point as a corporate CIO, he notes that "In many ways, IT is like senior management (CEO, CFO). It underlies the whole of the organization, and its impact, therefore, is pervasive and difficult to measure separately from the performance of the organization. As technology becomes more integral to each and every department, it becomes more difficult to evaluate separately."
Whatever metric you use, you must communicate it effectively to the business. Dennis Klinger, CIO at Florida Power & Light Co. in Miami, explains that finding "the best fit with the company culture" is also significant. "To the extent that I could, I try to enlist key senior management to help form and define metrics," Klinger says. "At FPL more than anywhere else I have been, the company itself is very metrics-focused. The fact that IT began managing with metrics and also tried to parallel existing company metrics helped form a common language and also added credibility."
Different Strokes for Different Units to Measure IT Value
The CIO at a major property and casualty insurance company explained his company's approach this way: "IT value metrics are developed internally and matched to the business unit. In other words, something done for the workers' compensation business unit would be totally different from [metrics] for personal automobile, which would be different from those for commercial building coverages. Most of these have been in play for many years, with modifications made as the business changes."
Gary King, CIO at fashion retailer Chico's, echoes this business-by-business IT value measure. "My conversation with the business unit changes based on the business unit. For some, it is about cost savings, for others the ability to scale or drive revenue," he says, noting that steering committees monitor delivery dates and project scope.
"Value measurement forces discipline. Many people hate this," says a services firm CIO. "The chief operating officer and divisional COOs love it, since it forces discipline and business rigor -- and solid business fundamentals."
How Metrics Change the Business Conversation Regarding IT Value
James Barry, vice president of application development at payroll firm ADP Inc. in Roseland, N.J., believes that metrics are what get him a spot at the strategy table. "Without any quantitative measure of my organization, I would not be at the table with the decision makers," he says. ADP tracks uptime percentages and total cost per client for building, supporting and maintaining systems. "No matter how many stats we give, they always want more," Barry says.
Providing believable, respected metrics also helps counter the "technology is too expensive" argument, adds the CIO of a global financial services firm. "What I now do is have a business conversation regarding whether the business activity is generating appropriate value for the shareholder."
Another CIO agrees. "The metrics had the benefit of making the conversation a business conversation, regardless of the topic," says the CIO of an East Coast insurer. "They tended to focus on improved profit, lowered costs, increased satisfaction, greater market share, etc." Systems projects that supported various business initiatives were blended into the overall project plan along with the other drivers. "These tended to increase the level to which IT was a business partner, with its own deliverables, as part of an overall plan -- which is always more valuable than having system projects in isolation," he adds.
"It is my experience that not all metrics are numeric. How the company feels about technology and IT is also a major factor at all levels," notes Reesa Abrams, a longtime educator and principal at Techno-Coach in Monterey, Calif.
The power of metrics comes not from the numerical values they portray but in how effectively they guide us and our businesses to the same page.
When you know where the goalposts are, it is much easier to score.
Thornton May is a respected futurist, adviser and educator whose insights on IT strategy have appeared in Harvard Business Review, The Wall Street Journal, BusinessWeek and numerous computer industry publications. To comment on this story, email email@example.com.
This was first published in June 2005