Inchoate Curmudgeon

A web search of forced ranking yields hundreds of links discussing the practice of ranking employees by human resource organizations for various purposes. This is not what I mean by the term here.

Forced ranking is the reasoned selection of candidates from competing projects, tasks, initiatives, scenarios and features. Components of the exercise involve quantifying and ranking qualitative attributes, casting initiatives and alternatives against a group or organization’s strategic objectives and ranking the intersections between problems and solutions for elaboration and selection. In short, forced ranking assists in deciding what to do when, with a documented underpinning in the organization’s objectives, charter and vision.

Forced ranking, or at least the techniques associated with forced ranking, are more fully documented for management disciplines, primarily project and program management, as well as product quality engineering methodologies, such as Six Sigma.

My first experience with forced ranking was developing a PowerBuilder application for a program manager at the Waste Isolation Pilot Project in Carlsbad, New Mexico some 15 or 20 years ago. At the time he was managing some rather elaborate spreadsheets for developing what he called “above the line and below the line” exercises. Many of the criteria were meaningless to me but deadly serious to him. The one I’ll never forget is color of money, but I digress.

For IT Architecture, forced ranking provide techniques for bridging the competing objectives of various business and technical organizations when selecting and defining IT projects and initiatives. Forced ranking can be used for varied problems from rationalizing IT architecture roadmaps or program plans, to selecting between architectural alternatives equivalently satisfying business objectives, to selecting candidate scenarios for large scale infrastructure projects like server consolidation.

Most people are familiar with the concept of cost-benefit analysis. The exercise relies on selecting two axes along which to rank alternatives. This is probably the most frequent simple ranking exercise, where one axis is financial cost related, the other most related to benefit.

Ranking items with these scales in a graph results in a scatter diagram where one of the four quadrants shows the greatest benefit alternatives yielded for the lowest cost. This will typically be the upper-left quadrant if cost is on the x axis, benefit on the y axis, and both axes are increasing values from the origin.

A ranking exercise becomes more interesting under several circumstances:

• accurate financial costs are difficult to derive or not meaningful for the alternatives,
• exercises looking at lost opportunities or potential revenue loss versus costs,
• exercises where the meaning of benefit is under contention, or
• exercises where factors impeding or confounding initiatives are under analysis.

Architecture initiatives frequently involve these exercises since they are often focused on qualitative characteristics of alternatives, and the circumstances for decision making involve negotiating competing stakeholder objectives.

The point is to take the emotion out of deriving the best problem targets and solution alternatives that meet competing stakeholder objectives. The point is to translate fuzzy qualitative characteristics into something measurable.

This brings up a critical point I’d like to make with regard to the process of forced ranking: the exercise itself is at least as important as the result. The process of decomposing and working though competing objectives with stakeholders, who don’t really understand each other’s perspectives, is invaluable in casting initiatives, solutions and problems in a way everyone at least knows where they stand, even if they don’t agree.

Six Sigma House of Quality or Quality Function Deployment (QFD) methodology is the best formal example I know of for forced ranking exercises. House of Quality is a very rigorous and comprehensive approach that is way beyond the scope of this discussion.

The more simplistic approach I like to take to forced ranking involves the following steps, and may be applied iteratively or in any order until next steps become apparent and actionable:

• list and appropriately elaborate the problems under consideration for resolution,
• elaborate problems by derivation of root causes for the immediately perceived problems,
• list and appropriately elaborate the solution alternatives,
• derive objective measures for qualitative characteristics of the alternatives,
• list and rank any constraints to solution alternatives, and
• rank the problems and solutions based on the scales.

Examples of some of these points may help to illustrate. The following particular examples are common in classic server consolidation exercises. The same principles used to derive these would apply to any architectural optimization exercise.

• An organization may want to reduce space and power consumption in the data center. These items my be confounded or exacerbated by the prevalent deployment of services on a single service instance per server basis. These are examples of problems under consideration and help to define the problem space. The deployment method, one-to-one server and service, is an example of a root cause problem driving the space and power issue.
• Solution alternatives to these problems include deploying multiple services on a single server, appropriately sizing servers to maximize utilization and using physically smaller and less power hungry servers. These solutions and many others make up the solution space.
• Objective measures for analysis of scenarios might include total space and power cost reductions. Less apparent discriminating factors might include the number of organizations that would be required to participate in a project or task to consolidate a particular set of servers, or the estimated time required to perform a particular scenario, or the high-availability characteristics of the applications, or the network security zone the servers reside in. These other measures are examples of attempts to derive and resolve the relative difficulty, risk or complexity of a particular set of problem or solution alternatives.
• Examples of constraints for the example might include caps on operations versus capital expenditures, preference for a particular type of server hardware due to qualitative characteristics or fit-for-service classification, and restrictions on the population of servers or services under consideration.
• Ranking involves determining which candidates in the problem space are solved by the candidates in the solution space as well as scoring the ranked intersections.

Simplifying and grouping alternatives into bands can assist greatly in achieving the focus required to proceed, particularly when the measuring method has a great deal of ambiguity. Categorization and measurement should conform to the principle of not measuring a mile with a micrometer.

For example, measures of project or initiative length could be easily and meaningfully divided into a month, less than three months, three to six months and more than six months. Measuring any more accurately than this is not worth the effort, depending on the stage of the exercise, namely ranking alternatives as opposed to defining a project scope for approval.

Trackback URI |