Are interviews for hiring no more accurate than running a lottery?

What does your interview process look like?

If it's like most you may as hold a lottery instead.

It's the most frequently talked about part of the hiring process - and the most hotly debated: the interview. For many it's the primary way to identify and select quality hires – contributors who will perform, live your culture, and stay long-term.

We (as nerdy Industrial-Organizational Psychologists and practitioners) often talk about interviews as structured or unstructured, but in reality, most interviews fall somewhere in the middle. At one of end of the continuum how we interview brings has tremendous predictive value (i.e., is strongly related to job performance), and on the other end… we fall short. If you know even a little about the science of effective interviewing, you’ll know that structure brings accuracy and validity: our ability to consistently accurately forecast candidate potential.  What does structure mean when it comes to interviewing? Here's what an intentional process can deliver and how to go about building it.

A structured interview will -

• Come from an established framework of competencies and understanding of job requirements. Begin with the end in mind: what is job performance for this role? This is the starting point for delivering a standardized process. Candidate potential is understood using pre-set questions created through evidence-based methods (e.g., competency modeling – understanding the job).
• Follow pre-determined steps. With your competencies, or articulated understanding of performance in place, think through the overall workflow, or architecture, for the entire interview process.
   
  • Would a phone or pre-screen be helpful? Who will conduct the phone screens? 
  • Will video interviews be used?
  • How many interviews total?
  • Timeframe. Will interviews be held on the same day and how long should it take for the interview process to be completed from start to finish?
  • Who are the stakeholders that will be using each step of the process? Recruiters, for example, may take on the pre-phone screen and initial interviews and hand off that information to hiring managers for the primary interviews. Information on what requirements and what aspects of the candidate's background would be most useful to these final stakeholders? 
  • Also, consider the nature of your hiring process and size of the group using the solution. We've built structured hiring and interview processes for centralized businesses as well as geographically dispersed operations like retail and restaurants, distribution centers, and manufacturing plants. Consider how you will earn commitment and train each stakeholder groups and what it will take to earn their buy-in.  Building a structured interview process can look very different if everyone is in one location compared to managers and stakeholders dispersed globally.
• Ask candidates established questions focusing on the knowledge, skills, and attributes required to perform, live your culture, and grow with you long-term. By beginning with a clear view of what we need in the role, starting with competencies and culture, developing a question bank that's informative, consistent, and candidate-engaging will be easier. Because all candidates applying for a job are asked similar job-relevant questions meaningful and accurate comparisons can be made about each candidate’s likelihood to perform successfully. In the first step we talked about identifying the stakeholders who will be running the interview process. Consider using this group to support building your question bank and interview process. Subject matter experts, oftentimes individual contributors or other leaders who aren't as involved in hiring, can provide knowledge and critical incidents to build out questions.
• Triangulate information gathered through other steps of the hiring process. This is a huge opportunity to increase efficiency and effectiveness of the interview, but is often ignored as much as a gym membership at the end of January. Let's share what we mean by this. Is a certain skill set or scheduling availability key? Ask questions to affirm that what candidates say matches what they shared on the application. It might not come as a surprise that we're going to say this, but consider leveraging any hiring assessment information you have available in the interview. Our hiring assessment tools and reports provide questions focused on key areas of opportunity that are critical to competencies required on the job. 
  
  
• But wait - there's more! To receive the gold star for interviewing, you must also have a pre-established system for recording and evaluating candidate responses. This is essential, as recruiters and hiring managers can differ widely on how they evaluate candidates and the standard to which they hold them – standards that can be job irrelevant. Consistent training, and ongoing refresher training, keeps hiring managers and recruiters focused on shared standards. These trainings also provide an opportunity to hear what your hiring team is thinking about the process and spot pitfalls before they morph into full-blown buy-in busters for your carefully crafted structured interview process.

Unsplash+ In collaboration with Getty Images

With this approach to design, implementation, and execution, structured interviews bring everyone on your hiring squad, and by extension your organization, together. Consistently using your structured interview process accelerates new hire success while dodging bias and building legal defensibility. Ideally, the structured interview provides useful information for onboarding, builds stronger bonds with new hires, and sets the stage for success.

As I-O Psychologists and Practitioners, building processes and tools that fairly evaluate people for promotion, hiring, or any employment decisions is our expertise and passion. We leverage data and science-driven processes to build solutions that are seamless, easy to use, and drive performance and results. When we partner with a company to build an interview process, here’s what we deliver:

The Unstructured Interview

All of this is fantastic, but if you're trying to rally your team around structured interviewing, let's chat about its wild cousin—the unstructured interview. Just like we want to compare candidates with the precision of a Swiss watch, it makes sense to weigh our interview options with the same finesse.

Unlike its buttoned-up sibling, the unstructured interview is a free spirit with few, if any, pre-set questions. Interviewers can ask whatever tickles their fancy based on the candidate's performance and the materials at hand. For most of your crucial work tasks, you probably have a plan, right?

Why would you also not have a standard process for hiring? Interviews without structure are a recipe for disaster.

When left to our own natural tendencies it's easy to veer away from job competencies and questions centered on critical incidents related to our culture. We end up with interviews with irrelevant questions that differ widely across candidates, making it almost impossible to evaluate whether one candidate truly is better suited for the position than another. However, this is one of the main reasons why companies still use unstructured interviews – and why so many hiring managers prefer them: they are flexible and allow for a smoother-flowing conversation to take place.

Because there are different questions and a lack of consistency in evaluating responses, how can you understand candidate potential? A common concern about adding structure is that the interview will be dry, cold, and robotic. But this is a misconception. Structured interviews don’t have to be rigid and uncomfortable – it’s still appropriate to open the interview with an icebreaker, such as asking the candidate about their interests and hobbies, so long as this information is not used in considering the candidate for the position. Even communicating the structure and expectations of the interview before beginning can help reduce anxiety, tension, and even awkwardness. Here’s an example:

In this interview, I will be asking you several questions that have to do with situations you are likely to come across if you worked at our company, and that are relevant to succeeding in this role. I ask these questions of every candidate. I may be taking a few notes, so please forgive me if I’m not making as much eye contact as I would like. I’m more than happy to repeat any question or provide clarification where appropriate. Please take all the time you need in answering, there is no rush. After I’ve finished, you’ll have an opportunity to ask me any questions that you might have. Please let me know if you have any questions before we get started. Otherwise, let me know when you’d like to begin.

Unsplash+ In collaboration with Toa Heftiba

See? It doesn’t have to be painful at all. So, we already shared one main reason as to why hiring managers continue to use unstructured interviews. But there’s one other key reason – people are overly confident in their ability to make a good hiring decision, simply by asking a few of their “trademark questions.” Understandably, you’re expert at what you do and your culture – it makes sense that you would believe you can use your judgment to select quality hires. However, a landmark study going back four decades shows that our intuition and gut feeling is ineffective in identifying quality hires – and how completely useless unstructured interviews are as a selection tool.

Should you conduct an unstructured interview or hold a lottery to hire? Either option will give you the same result.

Here's a case study that supports our point on structure and interviewing. In the late 1980’s, Texas was short on physicians. Legislature required the University of Texas Medical School to increase the class size of incoming students from 150 to 200 – this was after the admissions committee had already selected its preferred 150 students. The pool of remaining students was made up of candidates who had received low rankings from the committee.

Unsplash+ In collaboration with Alex Shuper

Researchers at the University of Texas examined whether this initial ranking mattered, and whether it was predictive of the students’ performance both during and after medical school. The performance of the initially accepted and initially rejected students turned out to be almost identical. Almost 75% of the difference in ratings between the initially accepted and rejected students was based on the committee’s perceptions of the candidates from unstructured interviews they had conducted – and yet the subsequent performance of both groups was practically the same.

What does this mean? Our gut instincts about performance are about as reliable as a weather forecast from a fortune cookie. The committee might as well have played bingo or tossed a coin to pick the original 150 students. A lottery would have saved everyone the hassle of those pointless, unstructured interviews.

Now, I’m not saying that we should replace interviewing in favor of holding lotteries for hiring! But to build remarkable teams and places to work, you should use a structured process. Science and data provided unparalleled support for structured interview methods.

Compared to unstructured interviews, structured interviews:

• Are 2x as predictive of job performance. 

• Provide almost 4x as much accurate and job-relevant information by gathering the right information and focusing our decision-making on those factors..

The next time someone argues against your company having a structured method for interviewing – suggest they try flipping a coin and see if they have better results.

Featured image photo by Alejandro Garay on Unsplash