Home > Midmarket CIO News > PPM software vendor scorecard: One company's vendor evaluation process
Midmarket CIO News:
EMAIL THIS

PPM software vendor scorecard: One company's vendor evaluation process

By Kristen Caretta, Associate Editor
16 Jun 2009 | SearchCIO-Midmarket.com

Technology news and tips for midmarket CIOs
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google

Choosing a project and portfolio management solution, like any enterprise application, can be a challenge. But never more so than when a company is seeking one standard PPM software package to replace some 10 different packages residing in 10 different project management offices (PMOs) -- the result of merger and acquisition activity over the years. Such was the case at Dalton, Ga.-based Shaw Industries Inc., a manufacturer of flooring products, which had varied tools and processes surrounding project reporting, logging and managing in its PMOs companywide.

More PPM resources
Software vendor evaluation criteria: Sample scorecards for IT projects

For a successful project manager, look for qualities of a good leader


Revised project and portfolio management standards get critical review
"The groups were siloed in how they managed projects and communication," said Greg Livingston, director of IS planning and systems development. "As we grew, project management tools were created as needed amongst the groups, and we lacked consistency in how we handled tasks."

This lack of consistency posed problems in meetings. When different groups brought reports together, "the reports looked different, the prioritization processes looked different," Livingston said. "Business partners never knew which version they would be getting."

The application mix included Microsoft Excel, Lotus Notes, Evernote (a point solution to manage Scrum projects), a custom Java solution called ShawOnline Activity reporting, .NET work order systems and Jira, an open source tool. Although some were effective as point solutions for functions such as resource management, application portfolio management (APM), project and portfolio management (PPM) and basic logging and tracking, none was an enterprise-centric bundle.

For example, people found the open source solution, Jira, effective for logging and prioritizing requests, according to Livingston, but it wasn't robust enough for enterprise use. "It wasn't enterprise level for us. There were no tools for resource allocation, financial management or dashboards, and the software required multiple instances of data," he said.

So when Shaw Industries decided it was time to standardize on one kind of PPM software, its requirements included a focus on process improvement and effective resource allocation, as well as alignment with methodologies already in place like Lean IT and Scrum.

"We needed a solution that would meet our two main needs: consistency in information extracting and reporting in our project execution, and a better picture of our application portfolio," Livingston said. "You cannot have an effective Lean Six Sigma strategy without a measurement process and metrics. We needed a single solution to help us manage our business."

Developing a vendor scorecard for vendor evaluation

The next step was turning these requirements into a scorecard by developing a list of evaluation criteria and a scoring system. Livingston created a vendor scorecard that covered the six dimensions Shaw wanted in a product, from PPM to APM, financial management and more. Product evaluators would then score products on specific attributes in these categories, using a rating scale of one to 10. Categories were not weighted per se, but some had more features to rate than others.

For example, for PPM software, the evaluation criteria were:

  • Project prioritization and business and effort analysis.
  • Project planning and tracking from conception to final delivery.
  • Capacity planning, including resource allocation and assignment.
  • Time entry and tracking, with a focus on functionality and ease of use.
  • How well the product supports Scrum methodology.
  • Michael Hanford, a research director at Gartner Inc., said he is a big fan of using a vendor scorecard, as long as (like Shaw's) it's developed with one's organization in mind.

    "I've seen some problems where people borrow a scorecard, find one in a book or look at one that someone else is using and think, 'If someone else used it, it must be good,'" Hanford said. "The thing you have to remember about scorecards is that it's a distillation of your goals, your needs and your corporate culture. What works for someone else won't necessarily work for you."

    Hanford recommends being thorough but not going overboard. "Some monstrously complex thing won't serve you well either because you're so focused on the scorecard, you forget what it is you're supposed to be evaluating," he said. Most important are simplicity, ease of use and the knowledge that the scorecard is just one aspect of your vendor selection process, along with other research and customer references, he said.

    On to the vendor evaluation process

    With an evaluation framework established, the Shaw team turned to Gartner's Magic Quadrant to identify the PPM industry leaders. Using the profiles and general evaluations in Gartner's research, it narrowed the search to two vendors: CA Inc. and Compuware Corp. "This is where the in-depth evaluation started," Livingston said. "We had two valid products from two well-respected vendors."

    The thing you have to remember about scorecards is that it's a distillation of your goals, your needs and your corporate culture. What works for someone
    else won't necessarily work for you.

    Michael Hanford
    research director, Gartner Inc.
    Evaluation teams then used the vendor scorecard during product demos and vendor meetings. At the end, the scores were close, but Shaw Industries eventually chose Compuware's Changepoint solution over CA's Clarity.

    What put Compuware on top? The biggest factor was the value of its PPM software, according to Livingston. "What you get in relation to what you pay was a deal for us. The features that were important to us just came out to be a better price," he said.

    The partnership and relationship between the two businesses was also a big selling point. After all of the evaluations and formal meetings, "we just related better on a personal and business to business level with Compuware."

    And finally, Changepoint had a feature called the agile accelerator, developed for agile IT and Scrum methodologies. "CA offered to adapt the product to work with our Scrum methodology, but Changepoint was already designed for this," Livingston said.

    At this writing, Shaw is still in the implementation process, which will take place across all 10 PMOs, with the project governance processes remaining the same. Livingston is satisfied so far with his team's decision. "It's strategic. We can allocate staff more effectively and stay competitive," he said.

    Let us know what you think about the story; email: Kristen Caretta, Associate Editor



    Tags: Vendor selection and management for the midmarketIT project and portfolio management for the midmarketSystems management for the midmarketVIEW ALL TAGS

    Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google



    RELATED CONTENT
    Vendor selection and management for the midmarket
    Saving money on software vendor maintenance contracts: A CIO series
    Seven tips for cutting vendor maintenance on business applications
    Adopting a beta tool: Risks vs. rewards for a midsized enterprise
    How to decide if changing technology vendors is worth the time, risk
    Software vendor evaluation criteria: Sample scorecards for IT projects
    How to cut application maintenance fees without undue risk or hardship
    CIOs taking risk of cutting vendor maintenance contracts to save money
    Windows 7 review: A closer look at this operating system for business
    Microsoft among ERP vendors increasing built-in vertical functionality
    10 tips for renegotiating your virtualization licensing contracts

    IT project and portfolio management for the midmarket
    Change management strategies: Best in IT Leadership Series
    Free project management templates for IT managers
    CIO: Don't attempt BPM system without mapping process flows
    7 steps to a workable business process management strategy
    IT and business management: Service, process and project performance
    An IT governance model needs risk and communications components
    What Google Wave means for IT: Collaboration in IT management tools
    As ERP system implementation goes live, ERP benefits start accruing
    ERP case study: Implementing ERP to manage growth, fix legacy issues
    Multitasking in IT: Sought-after skill or burnout bedlam?

    Systems management for the midmarket
    What will net neutrality mean for SMBs?
    Midmarket data center management guides: Tips and best practices
    Windows 7 review: A closer look at this operating system for business
    Microsoft among ERP vendors increasing built-in vertical functionality
    How to create and measure success of a SharePoint governance program
    10 must-have steps for an effective SMB information security program
    FAQ: Business process management defined
    Management tools for virtualized servers: A look at the options
    Virtual server management vs. physical servers: What's the difference?
    ERP implementations: In search of ERP best practices

    RELATED RESOURCES
    2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
    Search Bitpipe.com for the latest white papers and business webcasts
    Whatis.com, the online computer dictionary



    Midmarket CIO Technology Advisor
    About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
    SEARCH 
    TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

    TechTarget Corporate Web Site  |  Media Kits  |  Site Map




    All Rights Reserved, Copyright 2007 - 2009, TechTarget | Read our Privacy Policy
      TechTarget - The IT Media ROI Experts