People who study personnel psychology have long understood this. In 1979, for example, the Texas Legislature required the University of Texas Medical School at Houston to increase its incoming class size by 50 students late in the season. The additional 50 students that the school admitted had reached the interview phase of the application process but initially, following their interviews, were rejected. A team of researchers later found that these students did just as well as their other classmates in terms of attrition, academic performance, clinical performance (which involves rapport with patients and supervisors) and honors earned. The judgment of the interviewers, in other words, added nothing of relevance to the admissions process.
Research that my colleagues and I have conducted shows that the problem with interviews is worse than irrelevance: They can be harmful, undercutting the impact of other, more valuable information about interviewees.
In one experiment, we had student subjects interview other students and then predict their grade point averages for the following semester. The prediction was to be based on the interview, the student’s course schedule and his or her past G.P.A. (We explained that past G.P.A. was historically the best predictor of future grades at their school.) In addition to predicting the G.P.A. of the interviewee, our subjects also predicted the performance of a student they did not meet, based only on that student’s course schedule and past G.P.A.
In the end, our subjects’ G.P.A. predictions were significantly more accurate for the students they did not meet. The interviews had been counterproductive. ...
What can be done? One option is to structure interviews so that all candidates receive the same questions, a procedure that has been shown to make interviews more reliable and modestly more predictive of job success. Alternatively, you can use interviews to test job-related skills, rather than idly chatting or asking personal questions.
--Jason Dana, NYT, on the illusion of getting to know somebody