Why the most popular personality assessments may be working against you

Forced-choice personality assessments are trending, but the cognitive load they create can undermine your data. Here’s why free choice works better.


Personality assessments are more popular in hiring than ever, with over three-quarters of employers using personality and skills tests as a standard part of their vetting processes. A flood of AI-polished resumes has made it harder to evaluate candidates on paper, and more organizations are turning to assessments to get beyond the surface.

That instinct is right. But the formats gaining the most traction right now carry a design flaw that undermines the very thing they’re supposed to measure — and most hiring teams don’t realize they’re collecting noisier data because of it.

The issue comes down to how personality assessments are structured. Many of the most widely adopted tools on the market today use a forced-choice format, which requires candidates to compare a set of statements and identify which one is most like them and which is least. However, when candidates are spending several minutes parsing subtle semantic differences between options, they’re often problem-solving instead of self-reporting. To be clear, this is a question about personality measurement specifically. Cognitive ability tests work differently — they have right and wrong answers, so comparing options is the point.

Assessment design is a science decision, but it’s also a people decision. The format you choose shapes the quality of data you collect, the experience candidates have, and what your hiring process signals about your culture. There’s a better approach, and the reliability data backs it up.

Forced choice has become the dominant format in personality assessment because it feels rigorous. Asking candidates to weigh four statements against each other and identify which is most and least like them seems like it would produce more deliberate, considered responses. McKinsey uses a version of this format. So do many of the assessment providers that have seen the sharpest growth in adoption over the past few years.

But the cognitive load that format creates is a major problem. A candidate working through a forced-choice item isn’t simply describing themselves. They’re simultaneously running a comparison exercise, often between statements that are similar enough that the differences are genuinely subtle. That takes time and mental effort, and it compounds across an assessment.

By the end, you’re measuring how well they perform under cognitive strain rather than their self-concept. There’s nothing wrong with measuring cognitive ability — it’s a legitimate and useful predictor of job performance. But it belongs in a cognitive assessment, where the questions are designed for it and scored against right answers. When a behavioral assessment measures it by accident, you don’t get a bonus data point. You get a muddier read on the thing you were actually trying to measure.

That’s a design problem. A well-constructed assessment should reduce friction between a candidate and their honest self-perception. The data quality you get from someone who is fatigued or second-guessing their way through an item set is meaningfully lower than what you get from someone moving through a natural, intuitive experience. That gap will show up in your hiring decisions.

Free choice captures authentic self-perception

Our Behavioral Assessment takes a fundamentally different approach. Rather than forcing comparisons between statements, candidates simply select the adjectives that apply to them — as many or as few that feel accurate. When candidates feel a sense of autonomy in how they’re being asked to describe themselves, they engage honestly. Face validity — the degree to which an assessment feels relevant and fair to the person taking it — directly influences the quality of responses you get. A candidate who trusts the process is a candidate who gives you usable data.

That also has practical implications for who completes the assessment at all. Candidates who encounter a cumbersome, time-consuming format early in the hiring process are more likely to abandon it or disengage quietly and rush through. Either outcome costs you information. A format that respects how people naturally think about themselves produces better results more often than forced choice.

The free-choice format also removes the pressure to reconcile ambiguous options. When there’s no forced comparison, there’s no overthinking. What you’re left with is a cleaner read on how someone genuinely perceives their own behavioral tendencies, exactly what a personality assessment is supposed to deliver.

Longer doesn’t mean more reliable

There’s a reasonable assumption that a longer, more complex assessment produces more thorough results. More questions, more data, better picture. In practice, that logic doesn’t hold up.

Our Behavioral Assessment takes about six minutes to complete. Every five years, we put it through EFPA recertification, a process where independent auditors with PhDs in I/O psychology review our methodology and actively look for weaknesses in it.

As part of that process, we conduct test-retest reliability studies that track the same individuals across multiple time points, up to four years apart. Test-retest reliability measures how consistently a person gets the same result over time; the closer the score is to 1, the more stable the assessment. The values we see range from .2 to .5, which is considered high for multi-year measurement, where some natural drift in self-perception is expected. In some cases, these values even surpass those reported for the Big Five, one of the most extensively studied personality frameworks in the field.

That kind of stability over time is what makes an assessment defensible to use in hiring. A longer, more complex format doesn’t produce it. Thoughtful design does.

Assessment design is a people decision

Most organizations choose an assessment the same way they choose any HR vendor: by evaluating features, checking validation claims and moving on. What often gets skipped is understanding what the experience communicates to the people taking it. Every candidate who completes your assessment is forming an impression of your organization before they’ve spoken to a single person on your team.

How candidates experience an assessment shapes how honestly they engage with it. That has downstream effects on employer brand, offer acceptance and the quality of the talent pool you attract over time. Candidates talk. A hiring process that feels fair and respectful of their time reflects well on your organization. One that feels burdensome does the opposite.

The best assessment programs are built around a clear understanding of what you’re trying to learn about candidates and what experience you want them to have while you’re learning it. Format, length and design aren’t secondary considerations — they determine whether the data you collect actually reflects the people behind it. An assessment that candidates trust produces results you can trust. Want to see how our Behavioral Assessment compares? Explore our assessments.


Leadership tips and HR content worth sharing.

Join 150K+ business and HR leaders. One email a month.

THE LATEST FROM OUR BLOG


Copy link