When it comes to behavioral science, validity is everything. If a tool is not validated for workplace use, then what good is it? You might as well get your behavioral insights from a random person on the street (or from AI).
Content validity refers to the extent to which a tool used to measure something, such as a test, survey, or interview questions, adequately represents all aspects of the construct it’s designed to measure.
Put more simply, does the content of the assessment fully cover all the relevant aspects of what you’re trying to assess?
For example, take a spelling test for kids in grade four. If the test covers spelling patterns and rules appropriate for their grade level that they’ve just spent weeks learning in class, the test has strong content validity. If, however, the test includes words and patterns far exceeding their learning level that they haven’t learned in class, the test would have low content validity.
By the same token, take behavioral interview questions designed to evaluate specific skills that are outlined in the job description. If “communication skills” are critical to the role but the interview questions only focus on work ethic without touching on active listening or clear articulation, the questions have low content validity.
At The Predictive Index, we offer crucial insight that’s driven by 60 years of behavioral science. Our workforce assessments give you the people data you need to hire the right candidates the first time, build exceptional teams, and achieve milestones both large and small.
How to establish content validity
So, how do you establish content validity?
When it comes to recruitment tools like interview questions and personality assessments, establishing content validity requires a thorough job analysis to identify the essential components of the job; this allows you to define the content domain, which refers to everything the assessment needs to cover. The job analysis could include surveys, interviews, or focus groups with supervisors and employees formerly in the role.
It’s also essential to align the content with the specific job requirements. Does the assessment cover the critical tasks, responsibilities, duties, skills, abilities, and experience needed to excel in the position?
Next, involve Subject Matter Experts (SMEs)—people with deep knowledge and experience in the job role, skills, or domain of knowledge being evaluated. This could also include employees currently in the role or those who have recently been promoted, as well as supervisors, trainers, or consultants.
SMEs help determine if the content of the assessment is clear, representative, relevant, and effectively covers all of the key aspects it’s designed to include.
Integrate PI with your existing workflow
With 97+ integrations, you can keep everything in sync to align your talent strategy with business goals.
Methods to measure content validity
But how can SMEs measure content validity?
Content validity ratio (CVR)
Content validity ratio (CVR), developed by Charles H. Lawshe, is a quantitative method to evaluate how essential individual items are to the overall assessment. SMEs classify each item as “essential,” “useful but not essential,” or “not necessary.”
The is the formula to calculate CVR. CVR values range from -1 to +1. Higher positive values show greater agreement on how essential the item is. Critical values are used to determine if the CVR is statistically significant based on the number of experts.
Generally speaking, the more experts you have, the lower the CVR value you need to reach statistical significance since there’s more collective judgment. With fewer SMEs, you need a higher CVR to show agreement beyond chance.
If the CVR is equal to or greater than the critical value, the item has acceptable content validity, which means the agreement among experts is statistically significant. If the CVR is below the critical value, the item likely lacks acceptable content validity and may need to be revised or removed.
Content validity index (CVI)
Content validity index (CVI) provides an overall index of content validity for the entire assessment.
There are a few ways to calculate CVI:
- Item-level CVI (I-CVI) refers to the proportion of experts who rate an item as relevant, such as a rating of 3 or 4 on a 4-point scale.
- Scale-level CVI (S-CVI/Ave) refers to the average of the I-CVIs for all items in the assessment.
- Scale-level CVI (S-CVI/UA – Universal Agreement) refers to the proportion of items that all experts agree are relevant. This is a stricter measure.
Higher CVI values typically suggest stronger content validity, but recommended thresholds vary depending on the number of experts.
Measuring content validity requires the systematic collection and analysis of the judgment of SMEs. Do the experts feel the assessment content is relevant and representative of the domain of knowledge? CVR and CVI offer numerical scores that strengthen these expert-based judgments.
Curious how we handle validity at The Predictive Index? Here is the reliability and validity evidence for the PI Cognitive Assessment.
Why it’s important for HR leaders & managers
When it comes to talent optimization, content validity helps HR leaders and managers ensure that any tools or assessments used during hiring, development, or performance management directly measure the specific skills, abilities, talent, behaviors, and experience needed to thrive in a particular role.
Applications in talent acquisition and management
So, how can assessments with high content validity be used in talent acquisition and employee management?
Assessments with high content validity can:
- Predict job performance.
- Identify qualified candidates.
- Improve candidate engagement.
- Ensure fair and legally defensible hiring.
- Identify training and development needs.
- Help develop targeted training programs.
- Measure the effectiveness of training.
- Streamline performance management.
- Facilitate internal mobility, succession, and career development.
Assessments with high content validity truly measure what matters for the job, helping to connect what people actually need to do in their role with how to evaluate them. This way, when HR leaders and managers make decisions about hiring or promotions, they can focus on the things that really matter for the work, which ends up being better for everyone, both the company and its current and prospective employees.
Best practices for HR leaders & managers
Content validity is essential to ensuring the relevance and representativeness of any assessment as it relates to the defined construct or knowledge domain being evaluated. However, the validity of the assessment doesn’t stop there.
Our dedicated Science Team at The Predictive Index observes, builds, and improves the defensibility of our assessments by administering ongoing research and validity studies. Our assessments, which are also reviewed by third parties, are continuously reevaluated to ensure fairness, reliability, validity, and unassailable psychometrics.
With solutions like the PI Behavioral Assessment and the PI Cognitive Assessment, your organization can gain a deeper understanding of the unique behaviors, capabilities, and potential of current and future employees. Our tools are designed to provide you with actionable insights that drive better hiring decisions, foster team collaboration, and nurture leadership development.
Want to learn about other forms of validity? Check out our guides on construct validity and face validity.
