Predictive validity VS criteria validity

Many years ago, somewhere in 2008, when I took my first steps in developing a system for diagnosing candidates via the Internet, I asked Dr. Mirav Hami, then a young psychologist who had recently graduated and was recruited by me, to develop a skills test system called Logi (logic) Pass (Passes):

1. What is validity

2. What does it mean 

3. Why aren't HR managers excited and keep asking about its validity?

To be honest, I didn't understand. Up until that moment, my thought process was such:

Let's say we have a math test. If it turns out that those who succeed in it are indeed better than the others in math - hallelujah, we have validity!

But no, this is called criteria validity and it created quite a bit of confusion for me.

What those managers were asking me was about Predictive validity, However, I still needed to understand that predictive validity is a type of criteria validity. The truth is, even some of those managers didn't know exactly what they were asking for. They wanted a precise number - 70% 80%?

It seemed to me that at least some of them wanted to get some "bench-mark" in order to know if we were good

To my understanding, at the time, we offered them a great tool with which they could precisely tell which of the candidates is better with numbers, which is good with words, who is more intelligent, and who has a better memory.

We indeed checked, and it was undoubtedly true. For example, the candidates who scored highest in math were, in fact, better in it than those who did poorly on the test. So our tests had excellent "validity".

But in retrospect, I understand it was inaccurate, so how do psycho-technical tests predict success in a specific profession?

Later we also developed personality and integrity tests, and today I already know very well what validity they were looking for☺

Nowadays, we have thousands of customers, hundreds of thousands of test takers, and still, a good portion of our new customers ask us, even today, for only one type of test. Usually, it goes something like this:

1. Skills tests –"I'm looking for talented people. Everything else is uninteresting and hard to predict..."

2. Personality tests – "Skills are nice, but I'm interested in the employees' personalities. We don't build rockets here..."

3. Reliability test – "This is the only thing that interests me. I need honest and reliable employees, and I will check the rest myself"

The article aims to examine the consequences of such a choice on the predictive validity of psycho-technical tests

In order to achieve high predictive validity rates, and this is what those managers asked of us at the time, even if you are interested in only or mainly one thing, you must broaden your view... Look around. Haven't you met great medical professionals or teachers who were very different from each other In their character and skills? Lawyers? Mechanics? Officers?

I assume so. Let's examine this using the following example:

Let's say we are looking for a nurse for the emergency department at the hospital. What personality should we look for? What are the skills we determine as essential?
It might look something like this:

Now that you see the rather partial lists I have presented to you that fit the profile of a nurse, is it reasonable to forego one or another set of tests?

Will we give up the skills? General intelligence? (a weighted score of some of the skills tests), see here: Whether and how thinking ability tests can help assess a candidate's personality.

Will we give up on personality traits? Self-responsibility? probably not.

So why, when looking for couriers, mechanics, service personnel, or managers, should we settle for only part of the information such a system can generate for you?

Moreover, a computerized system has significant technological advantages, such as the ability to monitor the examinee's behavior during the skills tests and learn from it about his personality characteristics, for example: 

1. Impulsivity (quick response without thinking)

2. Fatigue (the beginning of the test is significantly better than the end)

3. Excitement (hesitant at the beginning and confidant later on)

And many more personal traits that can be measured during the skills test..

The same goes for modern reliability questionnaires. See, for example, a partial and limited list of indicators tested in a reliability test:

Watch the complete list here.

I am confident you understand that these characteristics overlap and benefit our personal traits assessment.Reliability tests in the internet age are a broad topic with many implications in the modern working world.

Back to my first steps in developing the system, I realized that in order to have suitable and sufficient predictive validity, as those managers asked me, and rightly so, we must formulate complementary tools to the skills tests, and that's what we did, so why use only one type of test?

Conclusion:

Just as you wouldn't dare to choose a basketball player just because he is tall (proper use of a criterion - height test), choosing your candidates using only one questionnaire will be wrong, no matter how good it is.

Another reason is that even though it is right for you to focus on, for example, personality, there are areas that overlap with skills and credibility that the system "sees" and reflects in the final reports. So there is no reason for you to give them up.

Only an intelligent combination, which is based on statistics and considerable experience that an integrated system has and will be reflected during the compilation of the profile with the help of our professional guidance, including the desired length of the test, can predict a precise level a person's success in a position and give you predictive validity..

It is essential that the costs of the tests required by the profile (the type and amount of tests) do not affect the overall cost of the evaluation. In order to provide the client with the highest predictive validity possible (by administering multiple tests), it's crucial to detach the element of cost.

Gabi Adam, CEO, Logipass.