Managing multiple Behavioral Assessments

Millions of people complete the PI Behavioral Assessment (“BA”) every year, some of whom may be asked to complete the assessment additional times. Whether it’s as a part of a hiring process, or as a request from a manager, colleague, or friend, multiple assessments may be more common than you may think. 

When someone has multiple sets of behavioral assessment results, it’s important to consider which results should be displayed within the PI software. In this article, you’ll learn best practices when setting a new active assessment.

Reasons to select a new active assessment

Several factors should be considered before selecting an active assessment when someone has multiple Behavioral Assessment results. These factors include when each assessment was completed, under what circumstances the assessment was taken, and to what extent the respondent agrees with their results. 

1. When was each assessment completed? 

At a high level, research generally supports the notion that personality characteristics are relatively stable throughout adulthood (Costa & McCrae, 1988); however, they are not completely static, and personality can change slowly over time. While different disciplines of psychology disagree on the exact steps of personality development, most theories focus on a combination of nature (e.g., traits) and maturation (e.g., developmental stages and experience) that becomes increasingly more stable as people enter adulthood (Costa & McCrae, 1997), reaching the highest stability somewhere in the 50- to 70-year-old range (Roberts & Del Vecchio, 2000).

PI has conducted numerous test-retest studies using the PI Behavioral Assessment which show that assessment results tend to remain stable for up to six years. In practice, this means that when someone retakes the PI Behavioral Assessment within a few years of the initial administration, their scores will often be very similar on the two administrations. For that reason, when combined with other sources of information, BA results are suitable for decisions that may be in effect for as long as eight years, such as hiring. 

Because of the stability of the BA, the PI software will set the default BA as the respondent’s first BA. However, if the original set of results is several years old, you may want to consider updating the active assessment to more recent results. Because individuals age and can change slowly over time or as a result of unique or life-changing experiences, they may have newer results that are more reflective of their personality and behaviors. 

2. Why did the respondent complete each BA?

In addition to considering the age of each assessment administration, it may be important to consider the purpose for which each assessment was completed. Was the second administration completed as a part of a hiring process? Was the second assessment necessary due to technical difficulties in the first administration? Was the second assessment completed because the respondent no longer felt aligned with their previous results?

If an individual has assessment results that are several years old, and they are asked to complete the BA as a part of a hiring process for a new role, then we recommend that they retest. For example, if a person takes the Behavioral Assessment when applying for a job, is hired, then is considered for a promotion five years later, it would be advisable to allow that person to retake the assessment. This is especially important for fairness considerations if other employees or outside candidates are being considered who have had the opportunity to take the BA more recently. 

If the second administration was completed as a part of a hiring process, then it would be advisable to use that administration as the active assessment. However, if someone has taken the Behavioral Assessment twice in recent years, and the second administration was not initiated for a high-stakes decision, the ”Self” behavioral factors from the first administration should be sufficient for most low-stakes applications of the BA, such as coaching. Nevertheless, the test user should use his or her own judgment about the conditions, such as understanding why the assessment was administered twice—did the participant have a technical problem with the first administration and need to be retested? If so, then the first result may not be valid. 

In the absence of any extenuating circumstances or additional considerations, the first Self behavioral factors should be the default scores to use. There are two reasons for this: 

Exposure and/or feedback may cause people to respond differently in the second administration. 

The first time someone takes a personality assessment, they are usually a clean slate. As they complete the assessment for the first time, their responses are not biased by experience with or exposure to the test format, the items, or the assessment output. For example, people who have been assessed with the PI Behavioral Assessment usually receive some type of report, feedback, or coaching around their results. 

The experience of taking the Behavioral Assessment and receiving information about the meaning of the scores may raise people’s awareness about the assessment and lead them to answer differently in a repeated administration. Sometimes, this occurs due to a desire to stay aligned with the first administration (e.g., I want to look the same); other times, this can occur because of attempts to appear different (e.g., I want to show the coaching worked). Regardless, the exposure to both the test format and the results can lead a participant to consciously or unconsciously alter their approach to taking the BA. It is best to avoid these situations when possible. 

Results from multiple administrations may appear graphically different, but are expected to be quite similar from a statistical standpoint. 

Scores from two administrations may appear different when in fact they really aren’t, at least not from a statistical or interpretation standpoint. It is important to understand that all assessments have some degree of error behind their scores. The formal term, “standard error of measurement,” refers to the idea that it is impossible to perfectly measure someone’s personality because, unlike physical characteristics (e.g., height), personality cannot be observed directly. 

For the PI Behavioral Assessment, the standard error is between 0.3 to 0.5 sigma for each behavioral factor; this means that if someone retests, their scores may appear different, when in reality their scores are just fluctuating within the expected error range. The person’s personality has not meaningfully changed, but the output may look slightly different. Unfortunately, few people are aware of psychometric realities like measurement error, which is why we believe it is a better idea to simply rely on the first set of results in most cases. 

3. Does the respondent still agree with their results?

As previously discussed, personality is a stable construct but may change slowly over time or as a result of unique or life-changing experiences. There are times when respondents have BA results that are several years old and no longer feel like they accurately describe them. 

Although participants should not be asked to complete the Behavioral Assessment more than once per year, and participants should only be surveyed when there is a need to collect behavioral scores, there may be unique circumstances in which a respondent opts to complete a second BA so that their results will describe their personality and behaviors more accurately. 

If a respondent requests updating their active BA to a second and/or more aligned administration, it would be advisable to do so. This will ensure that the respondent (as well as their team and manager) receives recommendations, insights, and guidance that pertains to a Behavioral Pattern that describes them well. 

Conclusion

By default, the PI software sets each respondent’s first Behavioral Assessment as the active assessment. However, administrators have the ability to designate a different BA as the active assessment. Considerations they should make to inform this decision include:

  • How old each assessment is
  • The circumstances under which the second assessment was completed
  • Whether the respondent has newer results that they feel describe them better 

For most applications, it is best practice to only assess someone once and rely on the respondent’s first set of results. When combined with other sources of information, the Behavioral Assessment can inform decisions for as long as eight years. However, if substantial time has passed, if a new high-stakes decision needs to or has been made, or if an employee no longer feels their results describe them well, it may be advisable to offer a re-administration of the assessment and/or to designate more up-to-date results as the active assessment in the software.

To summarize: If someone takes the BA more than once, it is a best practice to use the Self behavioral factor scores of the first administration unless the respondent completed the second assessment as a part of a hiring process, as a result of extenuating circumstances, or because they no longer feel aligned with the initial results. 

References

Costa, P. T., Jr., & McCrae, R. R. (1997). Longitudinal stability of adult personality. Academic Press. 

Costa, P. T., Jr., & McCrae, R. R. (1988). Personality in adulthood: A six-year longitudinal study of self-reports and spouse ratings on the NEO Personality Inventory. Journal of Personality and Social Psychology, 54, 853–863. 

Roberts, B. W., & DelVecchio, W. F. (2000). The rank-order consistency of personality traits from childhood to old age: A quantitative review of longitudinal studies. Psychological Bulletin, 126, 3–25.

To inform us of a typo or other error, click here. To request a new feature, click here.