Skip to content

Beware of numbers for subjective measures

Updated: January 11, 2013 - 3:49 pm

Posted: January 11, 2013

In a world gone mad, we see a lot of people abusing numbers. For me, that's troubling because I like numbers. I acquired the ability to count them in the first grade and completed all required stats courses in graduate school. But decades of textbooks have empowered us to misuse numbers in real-world situations, and this impacts the workplace.

Plus "stats people," like lawyers, speak a complicated language of their own. Information is presented in "stats-speak" designed to defy common sense translation.

So despite how dreary a topic, managers must know when their numbers are abused as well as the consequences for trying to make rational decisions based on them.

Now before your stats people rise defiantly, note I'm not talking necessarily about numbers used to assess concrete things, like production costs or marketing expense. When those numbers are combined with, say, numbers for product price or customer return rate, they produce correlation coefficients that work just fine.

My concern is when numbers are applied to subjective things involving people. When this happens, numbers might not make sense.

 

 

Based on what was learned in elementary school, we assume inerrancy in math: Addition, subtraction, multiplication and division work the same way with the same results each time they are employed.

Without such precision, statistics could not determine which factors cause what workplace results. But remember, it is impossible to add any set of numbers when the exact distance between each integer is unknown and therefore unequal.

Wow, does that mean 2+2 ? 4? Correct, if the distance between 1 and 2 is not identical to the distance between 2 and 3. This doesn't happen in measuring "age," for instance, where there are 365 equal days separating each year. (Let's not talk about Leap Year.)

But distances between numbers become blurred when we use social science techniques, such as scaling - "please rate on a scale of 1-5" - to assess perceptual indicators of things like "employee performance" or "client/customer satisfaction." Why? My perceived distance between "1" (very satisfied) and "2" (just satisfied) is not the same as yours. My "4" (dissatisfied) may actually be your "5" (very dissatisfied).

If we cannot add numbers, we cannot perform mathematical manipulations. We may not have accurate information needed to make rational decisions.

 

 

Social science stats people side-step this fatal flaw. Some create "dummy" variables out of each value within the scale, while others develop dichotomized ("yes-no") values in lieu of asking complex questions about reasons "you like buying my product." Still others prefer to add - remember, we can't do that - the scaled values to create an index that ranges from 1-100 - I guess the assumption being that having more numbers will numb our fear about not knowing the distance between each individual number.

While textbooks permit such mathematical tap-dancing, it causes important consequences for the workplace. Validity - measuring what we think we're measuring - becomes questionable and reliability - the ability to repeat an assessment with identical results - becomes probabilistic. Workplace decisions live vicariously with statistical tests that alert you as to just how wrong your information might be.

What to do? Managers work in a quandary, needing accurate information about subjective issues yet understanding the limitations of social science math. Move with caution. High-level stats are but one instrument in your tool box. Use correct tools for each project. Especially when assessing subjective dimensions about people, consider multiple tools and simpler statistics like median or mode and qualitative methods like customer/client focus groups or in-depth employee roundtables. A cardiology nurse once told me as I lay tied to tubes and wires: Rely on your own knowledge of your body and look less to the monitors. Your "body" is your workplace.

Given your experience, know what makes sense in your particular field or setting before you embrace a generalized assessment tool that may well be based on the abuse of your own numbers.

 

James D. Slack Ph.D. is director of the Master of Public Administration Program in the Robertson School of Government at Regent University. He can be reached at jslack@regent.edu.

Post-event coverage

To view a handouts from the speaker, click this link:
Useful Websites Resource Guide.