Reducing Cognitive Bias in the Tech Industry

I read an article last year in USA Today about comments that Jesse Jackson made concerning the tech industry's lack of diversity:

"There's no talent shortage. There's an opportunity shortage," he said, calling Silicon Valley "far worse" than many others such as car makers that have been pressured by unions. He said tech behemoths have largely escaped scrutiny by a public dazzled with their cutting-edge gadgets. - USA Today

My immediate reaction to Jesse Jackson's comments were to simply refute them. If you look at it from strictly a numbers standpoint, the percentages of minorities getting a technology related degree are no where near the percentages that these minorities are present in the overall population. So of course there is going to be a lack of diversity in the technology industry. I was so tempted to write a blog post about how wrong this perspective was, but I didn't. Then I read something that changed my mind...

I am slowly finishing up a Masters Degree in Systems Engineering (I say slowly only because doing so while working "full+" time is rather challenging). The class I took last semester, Decision and Risk Analysis, part of our required reading is the book Blink by Malcolm Gladwell. Blink is a book that, in a way, is the opposite of decision and risk analysis. Blink looks at how quality decisions can be made literally in the blink of an eye and the author covers this from the perspective that our brain can quickly process information (even going as far as showing how the human brain is more powerful than today's super computers at some tasks). One of the topics covered in the book is that of "cognitive bias." Rather than go into a lengthy definition of cognitive bias I will give you a lengthy example of how how one industry found a way to eliminate this bias.

In Blink, Malcolm Gladwell tells the story of Abbie Conant, a trombone player who was auditioning for the Munich Philharmonic Orchestra in the summer of 1980. The auditions were being held behind a screen so the judges could not tell who was auditioning. This was done because it was known that the son of one of the existing members of the orchestra was auditioning. In order to ensure impartiality, the concept of a "blind" audition where the person would play their instrument from behind a screen was established. Abbie made a crucial error in her audition and as soon as the audition was over she was already leaving...knowing that there was no way after that mistake that she would be selected. The judges were "blown away" by the power and skill behind her audition. They heard the mistake, but they also heard everything else that went on during that audition. Luckily they called upon her to step in front of the judges before she left the building. When she went out to meet the judges they were shocked that she was female. The name that was used for audition (Herr Abbie Conant) accidentally masked her gender and at that time it was extremely rare to have female musicians in the orchestra much less one that could play the trombone with so much power. Needless to say, she got the job. However, she ended up having to fight legal battles over being hired then fight again later on for her place within the orchestra and what she was paid. Not long after this happened, musicians began organizing themselves and fighting for equality and fair auditions. This eventually led to the common use of screened or "blind" auditions and it forever changed the landscape of orchestras.

The key aspect of cognitive bias is that it takes places within our brain without us being aware. Instead of paying attention to the sound, power and emotion behind the music being played, our brain's cognitive bias has already passed a certain amount of judgement on that musician based on a split second of initial input. In the case of Abbie Conant, the cognitive bias would have pushed the judges to notice her as female and instantly corrupted the judgement of the audio portion of the audition (had it not been performed behind a screen). This initial snap judgement would have made it more difficult for the judges to have noticed the power behind her performance and would have highlighted the mistake she made during the performance. In fact, Abbie had enough experience with traditional auditions that she was 100% sure that the mistake she made ruined any chance she had of being selected. However, the screened audition eliminated the visual cognitive bias and enabled the judges to focus purely on the audio portion of the audition. That crucial mistake made during the audition turned out not to be so crucial after all when taken properly into perspective with the rest of her audio performance.

So what does all of this have to do with the tech industry? The USA today article said this about Jackson's stance on minorities in the tech industry:

"The government has a role to play" in ensuring that women and minorities are fairly represented in the tech workforce - USA Today

I couldn't disagree more with this idea that the government has a "role to play" in this. Did the government step in to enforce a "screened audition" policy on orchestras? No, they didn't have to. The musicians banded together once they realized there was a way to fight against this cognitive bias that benefited both the quality of the orchestra and musicians fighting for a more unbiased way to have their abilities evaluated. I have a hard time believing that anyone wants to be on a losing team. The tech industry isn't actively skewing their hiring practices to exclude minorities (they wouldn't want to because much of their customer base is made up of minorities), but they may be doing so without realizing it.

I think the reason we see such a discrepancy in the number of minorities in tech compared to the general population is two-fold:

  1. The number of minorities getting a formal education in tech related fields is also disproportionate to the number of minorities in the overall population
  2. Cognitive bias is playing a substantial role in the hiring and promotion of minorities in the tech industry

Item 1 above is easily proven when you look at university enrollment data, so I totally disagree with Jessie Jackson's statement that "there's no talent shortage", but I agree that there is an "opportunity shortage". However, if you open up opportunities for minorities then I think the talent shortage will take care of itself. In other words, if you open up opportunities for minorities (or at least take down the barriers) then they will start coming over to the STEM (Science, Technology, Engineering, Math) areas of education. So how do we fix the opportunity shortage?

Just like in the story of the Munich Philharmonic Orchestra I believe there is an example that can be set within the tech industry that will forever change the landscape of minorities in tech going forward. There is one company in the tech industry that has already crossed technology with the liberal arts and infused that into their "corporate DNA"...Apple. What better company to take a lesson learned from the classical music world and apply it to technology? But how?

Cognitive biases are subtle yet extremely powerful. Aggressive measures must be used to eliminate or significantly reduce cognitive biases. In the case of the Munich Philharmonic Orchestra, all visual cognitive biases were eliminated by using a screen in front of the performer. In order to see a similar uptick in the hiring and promotion of minorities in the tech industry as to what was been seen with women being hired into orchestras, the tech industry must adopt an analog to the "screened auditions." I'm not suggesting that the entire interview/promotion process be "blind", but I certainly think aspects of it should be. You can't take all bias out of hiring and promotion, but you can at the very least give the decision maker an unbiased performance-based assessment of each candidate and ensure that the performance measurements weigh-in on the decision just as much as the interview process. One way to do this would be for a separate portion of the HR department to sanitize the resumes and applications for a given position before the evaluations begin. The initial screening would then be done without names, nationalities, genders, affiliations, schools attend, region in which you live, etc...anything that could potentially bias the person involved with the initial screening. Base that first round of evaluations purely on past performance and current capability. It can even be taken one step further. Once the field has been narrowed down to those candidates that are going to be interviewed, have the 1st interview be done virtually. In other words, have the candidates come into your facility for the interview but have the candidate and the person conducting the interview in separate rooms (they never see each other). The questions and the conversation will all be done via computer chat session. That way there is no way for cognitive bias to enter into the equation during that first interview. At some point you are going to have to sit in the same room face to face in order to make the final determination. However, if you remove as much of the cognitive bias as possible up front you are assured to be left with only the most qualified people to interview in the final round. You could even setup the entire process so all the previous steps have a score and a weight to the score such that the final face to face interview can only hold so much weight in the final decision. A similar approach should also be considered when selecting for promotions. This would require some kind of outside group to be used since most likely the HR department would know internal candidates that are applying for promotion and being cognitive bias along with that knowledge.

What I have suggeted above is no easy task. In order for something like this to work you must have a very well defined scoring and weighting system and you absolutely MUST know what qualities and capabilites you are looking for when going into the process, otherwise you risk hiring someone who is not well suited for the job. You can never completely eliminate bias from a decision when a human is involved. The main strength we bring to a decision like this (intuition) can also lead to our greatest weakness (bias). Acknoledging that bias and then doing your best to eliminate it is going to be a win for everyone involved. Employers will be hiring the best of the best (not just what they incorrectly percieved as the best) and minorities won't have to fight so hard against these biases. Even the extremem measures I suggest above won't change things overnight. Opportunities for minorities need to come before the qualified minorites will apply. Nobody wants to struggle through school for years only to find they don't have a shot at a job. But little by little as opportunities are shown to exist the minorities in turn will choose to enter into the educational areas with these opportunites.

Just like with the example of Abbie Conant and the Munich Philharmonic Orchestra, I think it is going to take a very high profile company like Apple to adopt some kind of aggresive means to minimize cognitive biases in the hiring and promotion of employees in order for this change to take hold. Apple is a company that is both bold enough to do this and is in a position to gain the most from it. Apple products are purchased by an extremely wide variety of people, which means Apple needs an equally wide variety of employees that understand that segment of Apple's customer base. I don't know enough about Apple's hiring and promotional tactics today. Maybe they are already doing some of this now. But wouldn't it be great if they could help lead a cognitive bias revolution in the tech industry similar to what occured in the music performance industry after what happened with Abbie Conant and the Munich Philharmonic Orchestra? Leading a change like that would leave a mark on the tech industry that would be more impactful and longer lasting than the iPod, iPhone and iPad combined.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.