Why Benchmarking Doesn’t Work

When most consultants evaluate a client’s IT operations, they rely on benchmarks to provide a cost and performance baseline, set goals, and measure progress. But there’s a problem with this approach: it simply doesn’t work. Cost benchmarks force client data into a generic model that isn’t able to capture the unique differences in client service strategies, and can’t account for service quality, performance levels, or consumption issues. These limitations ultimately lead to assessments based on invalid data that don’t help the client company meet its objectives.

Benchmarking is broken

Benchmarking suffers from several critical limitations that make it an inadequate tool for assessing a company’s IT services and measuring progress.

• Generic models – Perhaps the greatest issue with benchmarking is that it relies on standardized models that aren’t fit to the unique characteristics of each organization. This means that they can’t account for differing client goals and strategies. For example, they may not accurately reflect the equally viable strategies of focusing on lowest-cost services versus managing IT as a strategic investment. Forcing data into a standardized service and cost model doesn’t align with how most companies view their IT services, which means that the benchmark results don’t mean very much.
• No accounting for financial parameters – Companies have a wide range of financial options when building out their IT applications, infrastructure and services. They must decide whether to lease versus buy, capitalize versus expense, time the acquisition and manage the volume of purchases. Unfortunately, many benchmarks don’t take these parameters into account.
• Out of date data – If a benchmark relies on data that is more than six months old, it may have limited validity today. In order to be effective, benchmarks must be based on fresh data that accounts for recent changes.
• Not fit to client profile – Every client will have different requirements and amounts of leverage. Most benchmarks don’t provide a practical assessment of what is available within the industry as it relates to the client’s industry position and financial considerations.

A better approach

At Wavestone US, we believe that traditional benchmarks are problematic and have no place in IT consultations. Our approach differs in that we use data from our engagement experiences in combination with more conventional benchmark information to create a comparison between the client’s cost structure and those of our other clients. The relative subjectivity of comparing client services with those of other organizations requires us to provide as many details as possible to compare and contrast the IT services including the scope of services performed, service delivery models, service level attainment, and, if applicable, contractual terms. This provides a more comprehensive, up to date assessment that’s fit to the client’s unique needs.

A complete and uniform understanding of the client’s current performance, limitations, and challenges provides a better foundation for future planning. This allows for a more natural progression to sourcing strategy development, scope and timing of RFPs and other critical strategic decisions. With a full understanding of current capabilities, risks, constraints, and goals, it is possible to create a better roadmap for IT service development that reduces costs and delivers better results.

To learn more about Wavestone US’ services, visit http://wavestone.us/capabilities.