Author: Managing Partner – Roger Pearman
The popularity of the use of self-report tools in the pre-employment selection and placement decisions has had exponential growth in the last decade. Publishers of these tools put forth propositions about the ability of the tool they support to provide the basis of good selection decisions. And such claims require discerning professionals to take a closer look at the propositions offered or they get stuck in a “sticky wicket.”
The Society of Industrial and Organizational Psychologists (SIOP) Standard
Some years ago, the Society of Industrial and Organizational Psychologists (SIOP) took up the issue of using self-report tools in pre-employment selection and developed a systematic and comprehensive process for determining the veracity of such tools in a critical process for hiring talent. The standards they create are exacting and complex and designed to help reduce false negatives and false positives—rejecting those who are qualified or accepting those who are not. At the end of the day, no matter how statistically exacting, SIOP warns of lurking problems in the process, and of the essential importance of structured interviews to verify capabilities and reduce as much bias as possible.
Setting aside the evidence of bias in self-reports…for a moment.
We can set aside for the time being the evidence that there is bias in self-report; individuals simply describe themselves as they think they are, and some of the time their report is accurate and often it is not. So, to address the various issues in self-report tool use in employee selection, SIOP noted that correlational studies between the selected assessment and performance factors for a given job are not sufficient. Further, contrast testing between high performing individuals and low or moderately performing individuals are essential but incomplete. Consider that an assessment manual reports that there is a p<.0001 level of significance in correlating a particular scale with a given job standard and the correlation is .65. Seems extraordinary, right? In fact, such a correlation means that only 42% of what makes up one contributes to the other, leaving 58% subject to other contributing factors. Is that good enough?
An Example from Personality Tools
Consider that most personality tools today have a measure of extraversion or social engagement. Just because individuals report themselves as extraverted doesn’t mean that is true. Further, just because there may be correlations between extraverted scores and certain job requirements doesn’t mean that is true for the candidate applying for the job. In short, the statistics do not equate to reality; they give hints and hypotheses which a structured interview can help verify.
The “Behavioral Periodic Table”
Imagine that for any given job, the “behavioral periodic table” (metaphorically speaking) requires certain elements (i.e., behaviors) be present. The tool being used to examine and determine if a given candidate has those elements must be exceedingly precise and comprehensive and appropriate to the task. The elements vary in weight (or by parallel, vary in importance to the job), the elements need to combine to make specific compounds (positions), and all the necessary elements need to be included. No such self-report tool can achieve that level of specificity for behaviors associated with a given job.
The Ideal Fit Process
While normalized assessment tools can provide some hints and estimations, they are not definitive. The ideal fit process would involve an organization creating a profile with prioritized behaviors for what is essential for success, an individual self-rating on those behaviors, and a structured interview process for verifying the reported information. In such a process there is a greater chance at precision as it relates to the needs of a specific job and the capabilities of a specific candidate. And even in this ideal, there is still room for measurement error, which is why the structured interview is essential. In no case should a survey, assessment, or pre-employment self-report tool be the defining datapoint for hiring or promoting a candidate. To do so increases legal risk and ensures wasted resources.
We took all of these factors, and more, into consideration during the development of our pre-employment Fit Appraisal.