How Do We Implement A 10-Point Rating Scale?

Dear Workforce: How Do We Implement a 10-Point Rating Scale?
Try this: Do one cycle with the 10-point scale, but reduce the number of points on the scale the next year if data show that no one is using the lowest points on the scale.

 A business unit in our company is introducing a 10-point rating scale. This process has been approved and will be used in our performance management system. However, I have yet to find any documentation to support the rationale for a 10-point rating scale. How high is too high? How should we implement this?

—Going for Three Points, software/services, Melbourne, Australia

Dear Going for Three Points:

Several questions are embedded here.

First, and possibly most important, is this: How much influence do you have over this decision and how much political capital are you willing to burn to push back against a 10-point scale and advocate for a three-point scale? It sounds as if the 10-point scale has momentum with the group's decision-makers, and possibly some broader cultural acceptance in this business unit.

One diplomatic approach might be to strike an agreement to perform one cycle with the 10-point scale, but reduce the number of points on the scale the next year if data show that no one is using the lowest points on the scale. It's a virtual certainty that you'll be able to compress this scale after one cycle. It's hard to make 10 discrete, robust, meaningful performance distinctions—if you really try to figure out what behaviors or outcomes distinguish a 2 from a 3, for example.

Based on much field and academic research, a 10-point scale is likely to yield ratings concentrated between scores of 7 and 9, with little of the lower half of the scale ever being used by raters. Similarly, a three-point scale is likely to produce ratings mostly in the top two categories. Although the lowest 30 to 40 percent of a 10-point scale may get little use, its greater number of choices allows evaluators a few more points on the continuum and may help get at underlying performance differences. A three-point scale may be prone to exaggerating the central-tendency problem, tempting evaluators to promote low performers up to the second category since the lowest category is likely to be seen as a harsh failing grade.

There is no known research to suggest either a 10-point or three-point scale is some sort of optimum measure for data reliability. Though 10 may be too fine a cut, and three a bit too compressed, research repeatedly shows that almost all scalar rating methods are fraught with some common weaknesses.

When performers are evaluated one at a time on a scale, rather than in simultaneous ranked comparison with other performers, two types of reporting errors are common: leniency (the tendency to avoid giving low ratings, even when the rater has observed low performance) and central tendency (clustering performers' ratings close together, rather than reporting wider performance differences, even when they are observed). However, comparison rating (forced ranking) is a big and challenging innovation and is sometimes unpopular. It doesn't sound as if your group is ready for that.

Regarding implementation: Whatever scale you choose, add behavioral anchors. These are short descriptions of behaviors and results that typify a given part of the scale. Print or display them right on the scale, each beside its associated number, to be easily seen while rating. This will increase raters' confidence that they know what they are reporting when choosing a particular number. In addition, some research suggests that describing the frequency of a behavior ("almost always meets important deadlines") rather than its degree ("excels under deadlines") improves data reliability.

SOURCE: Harold Fethe, organizational consultant, St. Augustine, Florida

Thanks to Crain Communications Inc. / WorkForce Management Online
http://www.workforce.com/section/newsletters/feature/dear-workforce-how-do-we-implement-10-point-rating-scale/index_printer.html