Unstructured Interviews and the Lottery

Is your company using an unstructured interview process to hire?

You may as well flip a coin or hold a lottery.

The pre-employment interview continues to be one of the most commonly-used tools in the hiring process to identify and select a quality hire – someone who will perform beyond expectations, fit the culture, and stay. We (as nerdy Industrial-Organizational Psychology practitioners) often talk about interviews as structured or unstructured. One has tremendous predictive value (i.e., is strongly related to job performance), and one… does not. If you know even a little bit about the science of effective interviewing, you’ll know that the former is the structured interview.

A structured interview is a standardized technique of evaluating and comparing job candidates using pre-set questions created through data-driven methods (e.g., competency modeling – understanding the job). These questions focus on the knowledge, skills, and attributes required to perform, live your culture, and grow with you long-term. All candidates applying for a job are asked the same job-relevant questions so that meaningful and accurate comparisons can be made about each candidate’s likelihood to perform successfully.

We’re not done yet. To receive the gold star for a structured interview, you must also have a pre-set system for recording and evaluating candidates’ responses. This is essential, as managers can differ widely on how they evaluate candidates and the standard to which they hold them – standards that can be job irrelevant. Structured interviews get everyone on the same page, accelerating new hire success and avoiding bias and potential legal trouble.

As I-O Psychologists and Practitioners, building processes and tools that fairly evaluate people for promotion, hiring, or any employment decisions is our expertise and passion. We leverage data and science-driven processes to build solutions that are seamless, easy to use, and drive performance and results. When we partner with a company to build an interview process, here’s what we deliver:

The Unstructured Interview

All of this is great, but if you’re seeking to get your team onboard with structured interviewing, let’s discuss the alternative – the unstructured interview. Just as we want to be able to accurately and objectively compare candidates, it makes sense to compare the options we have of carrying out an interview.

Unlike a structured interview, an unstructured interview has no – or very few – pre-set questions. Interviewers are free to ask questions that make sense in the moment based on the candidate’s performance in the interview and the materials available. For most of your important deliverables at work, you probably have a process, right? Why would you also not have a standard process for hiring? Interviews without structure are a recipe for disaster. The majority of questions asked during unstructured interviews are job-irrelevant and differ widely between candidates, making it almost impossible to evaluate whether one candidate truly is better suited for the position than another. However, this is one of the main reasons why companies still use unstructured interviews – and why so many hiring managers prefer them: they are flexible and allow for a smoother-flowing conversation to take place.

Because there are different questions and a lack of consistency in evaluating responses, how can you understand candidate potential? A common concern about adding structure is that the interview will be dry, cold, and robotic. But this is a misconception. Structured interviews don’t have to be rigid and uncomfortable – it’s still appropriate to open the interview with an icebreaker, such as asking the candidate about their interests and hobbies, so long as this information is not used in considering the candidate for the position. Even communicating the structure and expectations of the interview before beginning can help reduce anxiety, tension, and even awkwardness. Here’s an example:

In this interview, I will be asking you several questions that have to do with situations you are likely to come across if you worked at our company, and that are relevant to succeeding in this role. I ask these questions of every candidate. I may be taking a few notes, so please forgive me if I’m not making as much eye contact as I would like. I’m more than happy to repeat any question or provide clarification where appropriate. Please take all the time you need in answering, there is no rush. After I’ve finished, you’ll have an opportunity to ask me any questions that you might have. Please let me know if you have any questions before we get started. Otherwise, let me know when you’d like to begin.

See? It doesn’t have to be painful at all. So, I already shared one main reason as to why hiring managers continue to use unstructured interviews. But there’s one other key reason – people are overly confident in their ability to make a good hiring decision, simply by asking a few of their “trademark questions.” Understandably, you’re expert at what you do and your culture – it makes sense that you would believe you can use your judgment to select quality hires. However, a landmark study going back four decades shows that our intuition and gut feeling is ineffective in identifying quality hires – and how completely useless unstructured interviews are as a selection tool.

Should you conduct an unstructured interview or hold a lottery to hire? Either option will give you the same result.

In the late 1980’s, Texas was short on physicians. Legislature required the University of Texas Medical School to increase the class size of incoming students from 150 to 200 – this was after the admissions committee had already selected its preferred 150 students. The pool of remaining students was made up of candidates who had received low rankings from the committee.

Researchers at the University of Texas examined whether this initial ranking mattered, and whether it was predictive of the students’ performance both during and after medical school. The performance of the initially accepted and initially rejected students turned out to be almost identical. Almost 75% of the difference in ratings between the initially accepted and rejected students was based on the committee’s perceptions of the candidates from unstructured interviews they had conducted – and yet the subsequent performance of both groups was practically the same.

What does this mean? Our beliefs and intuition about performance is flawed – and the committee may as well have held a lottery or flipped a coin to randomly select the original 150 admitted students. A lottery would have saved them – and candidates – time from conducting those ineffective, unstructured interviews.

Now, I’m not saying that we should replace interviewing in favor of holding lotteries for hiring! But to build remarkable teams and places to work, you should use a structured process. Science and data provided unparalleled support for structured interview methods. Compared to unstructured interviews, structured interviews:

Are twice as predictive of job performance.

Provide almost four times as much accurate and job-relevant information.

The next time someone argues against your company having a structured method for interviewing – suggest they try flipping a coin and see if they have better results.

Are you ready to better understand how structured interviews can build remarkable teams and places to work, contact us today to start a conversation.