By Phil Drazewski
Among the many personnel measures which can be used to predict job performance, one measure has historically been used far-and-above all others: the job interview (Campion, Palmer, & Campion, 1997). In most cases, these interviews are conducted with little-to-no structure other than a few handshakes and a run through the resume; afterwards, managers tend to rely on “their gut” to make hiring decisions. But how useful are these unstructured interviews compared to other hiring methods?
In 1998, Frank L. Schmidt and John E. Hunter published a landmark study comparing 19 common selection procedures used for predicting job performance (see Table 1 below). Surprisingly, despite their widespread application, the authors found that unstructured interviews provided less validity than a number of other methods including measures of general mental ability (GMA), work sample tests, integrity tests, structured interviews, job knowledge tests, job tryout procedures, and peer ratings.
Table 1
Personnel Measures’ Ability to Predict Overall Job Performance (adapted from Hunter & Schmidt, 1998)
Personnel Measure | Validity (r) |
Work sample tests | .54 |
GMA tests | .51 |
Employment interviews (structured) | .51 |
Peer ratings | .49 |
Job knowledge tests | .48 |
Integrity tests | .41 |
Employment interviews (unstructured) | .38 |
Conscientiousness tests | .31 |
Schmidt and Hunter’s findings begged one question: why are we so bad at predicting workplace success using unstructured job interviews? Industrial/organizational (I/O) psychologists and social psychologists have since identified a number of unconscious biases we all have that affect our decision making:
1. Inconsistency in questioning
Perhaps the most common unconscious bias occurs when managers asks different candidates different questions. This likely stems from the fact that candidates come to the interview with vastly different life experiences, and though the manager may find these differing experiences interesting, it can result in trying to compare apples to oranges when it comes time for decision-making. Fortunately, this bias is also fairly easy to fix; as Table 1 demonstrates, by simply adding structure to the interview (i.e., asking candidates a standardized list of questions), managers can increase the validity of their hiring process by more than 10%.
2. Halo/horns effect
The halo effect is a cognitive bias initially identified by renowned psychologist Edward Thorndike. It involves managers’ allowing their overall impression of a person to influence their feelings and thoughts about that person’s character or properties (Forgas & Laham, 2007). If the manager likes one aspect of a person, they likely will have a positive predisposition toward everything about them; if the observer dislikes one aspect of a person, they will have a negative predisposition toward everything about them (i.e., the horns effect).
3. Implicit & explicit stereotyping
One group of biases that are often hard for individuals to acknowledge are those attributed to stereotyping based on demographic and/or physical traits (e.g., race/ethnicity, gender, socioeconomic status, religion, physical appearance, and smell) (Baron, 1983; Dipboye & Colella, 2005). However, a number of research studies have shown even individuals who do not have explicit biases likely still have implicit biases that are often harder to control and/or are deeply engrained in one’s culture. Fortunately, simply being aware of the fact that we may hold implicit biases can help decrease one’s likeliness to stereotype based on a multitude of demographic and physical traits (Lepore & Brown, 2002).
4. Serial position effect
Serial position effect is a term coined by another famous psychologist, Hermann Ebbinghaus, and refers to the phenomenon of recalling more information about an item/person based on that item’s or person’s position within a list. Namely, when individuals are asked to recall a list of items/people in any order, people tend to be better at recalling items/people at the beginning (i.e., primacy effect) and end (i.e., recency effect) of that list (Deese & Kaufman, 1957). As such, managers should make a conscious effort to give interviewers in the middle of the process as much attention as they’ll likely give those interviewing first and last.
5. The interviewer effect
Often, it isn’t possible for one manager to conduct all the interviews for a position. However, having different managers rate different candidates can lead to variability/bias between managers/raters. Often, this is referred to as the interviewer effect, interview variance, or interviewer error (Groves & Magilavy, 1986). Each manager may value different traits in a candidate, and thus a candidate that does poorly with one interviewer might have done fairly well with another. To minimize inter-rater bias, managers should utilize identical (or near-identical) structured interview questions. They should also make an effort to minimize their own contributions to the content of the interview, as this can result in conversational tangents that are likely not to match up with the other manager’s interview content.
Check out Phil Drazewski’s other recent blog post, The rise of I/O Psychology.
References
Baron, R. A. (1983). Sweet smell of success? The impact of pleasant artificial scents on evaluation of job applicants. Journal of Applied Psychology, 68, 709-713.
Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50, 655-702.
Deese, J., & Kaufman, R. A. (1957). Serial effects in recall of unorganized and sequentially organized verbal material. Journal of Experimental Psychology, 54(3), 180-187.
Dipboye, R. L., & Collela, A. (2005). Discrimination at work: The psychological and organizational bases. Mahwah, NJ: Erlbaum.
Forgas, J., & Laham, S. (2007). Halo effect. In R. Baumeister, & K. Vohs (Eds.), Encyclopedia of social psychology (410-411). Thousand Oaks, CA: SAGE.
Groves, R. M., & Magilavy, L. J. (1986). Measuring and explaining interviewer effects in centralized telephone surveys. Public Opinion Quarterly, 50(2), p. 251.
Lepore, L., & Brown, R. (2002). The role of awareness: Divergent automatic stereotype activation and implicit judgment correction. Social Cognition, 20(4), 321-351.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274.