Business Mentor
Tracing Root Causes Back to IT

This month, I'm going to start with some trivia and then tell you how it relates to IT. (If you ever become a contestant on the show Who Wants to Be a Millionaire?, you may find this approach useful.) Physicians use the famous phrase "first do no harm" to acknowledge that human acts -- even those backed by best intentions -- may have unwanted consequences.

But where does this motto come from? (And here's your final answer.) Contrary to a commonly held belief, it's not the Hippocratic Oath. The closest approximation to "first do no harm" can be found in the Hippocratic Corpus: "to help, or at least to do no harm." That's what Wikipedia tells us.

So what does any of this have to do with being a CIO? I've been thinking about that phrase as we review our service-level performance. We are attempting to find problems' root causes. So, for example, we want to trace the source of a network outage at our remote sites or find out why some users get locked out of applications or learn why application response time has increased over the past three months.

After some investigation, we discovered that in cases of poor service-level performance, we are often the cause. In other words, the IT department has made a decision or a change that results in a failure or that threatens reliability and performance. But since we are the source of the harm, we are also able to eliminate the problems we introduce.

Doing Less Harm

Several years ago, I gathered with a group of IT leaders to discuss and define a set of change management best practices. The resulting list mapped well with both the IT Infrastructure Library (ITIL) standard and the Microsoft Operations Framework, or MOF. (I prefer MOF, which is available for free on the Microsoft website.) The group identified five important elements of effective change management:

1. Create and maintain a test environment that mimics the production environment. Also commit to testing every change.
2. Ensure that every proposed change is approved by a change review committee. The committee should include representatives from each IT function (for example, if the database administrator wants to apply a patch to the database, the application group should be involved in the review and approval of the change).
3. Create "rollback" rights. A change review committee has the right to roll back changes that don't pass tests or that cause "harm."
4. Communicate change clearly and concisely to affected users. They are often ignored until too late in the process.
5. Deploy changes on a predictable schedule (during the third weekend of the month, for example).

Emergency changes should follow the same process but not be postponed for a regularly scheduled meeting of the change review committee -- and they may have to be made right away. Otherwise, the process should be maintained.

I've discovered that we can resolve most of our service-level issues by making planned, tested and communicated changes to our production systems. With this set of best practices, we find we do less harm more often.

Digg This!     StumbleUpon     Del.icio.us