Summary
A.I. algorithms are built using machine learning techniques applied to large datasets to derive patterns which can then be applied to new data. The algorithms themselves rarely have bias built into the models, but is often contaminated from bias within the datasets. Using TR Recruiter avoids the bias imbedded within other Algorithmic Hiring Tools (AHTs), as TR Recruiter :
- Does not use data scraping techniques to find patterns
- Does use a baseline (the Five Factor Model) recognised by psychologists as being objective
- Created its own proprietary dataset comprising 300,000 users and 5,000,000 images
- Avoids sampling bias as the demographic breakdown of the dataset broadly reflects those of the UK working population.
What Is Bias and How Does It Infect AI Models
The word bias (in the statistical sense) means distort, it exists in all statistical and data systems with one exception (Pi). Defining the type of bias can be tricky as although we are familiar with active bias when a decision is based either on a like or dislike of a characteristic (such as gender), passive bias is harder to define and therefore to detect and measure. By passive bias we mean when within the machine learning phase there is considerably more data on one type of group than on the other, but the AI algorithm is applied equally to both groups: an example is Google’s speech recognition which is 13% more accurate for men that for women.
AI algorithms evolve using machine learning techniques, at its simplest level machine learning is feeding large amounts of data into a system that then recognizes patterns, with the errors being corrected until the algorithm’s error rate is acceptable.
Bias can be built into the AI algorithms, in cases where for the sake of Equity (the ‘E’ in ED&I) the objective is to compensate for historic bias by applying a positive or ‘affirmative’ bias, however this is rare (and after the SCOTUS ruling in SFFA vs Harvard is likely to become more rare). Bias is inadvertently introduced into the algorithm via the datasets used to train the models*
Where TR Recruiter Fits in the Recruitment Process
The recruitment funnel suffers from a paradox.
- Soft skills make the difference between a good and a great employee
- The first opportunity to really assess soft skills is at the interview
- The recruiter has to discard the vast majority of applications to select the interview pool, based only on hard skill information.
Solving this paradox – providing soft skills analysis with hard skill analysis at the time the recruiter selects the interview pool can only greatly improve the quality of selections. TR Recruiter solves this paradox.
Types of Dataset Transmission of Bias and Their Effect on the Output**
Bias exists in every system, for recruitment to be free of any statistically significant bias, both the level of bias and the effect it has on the output needs to be understood, and compared to alternatives.
Creating proprietary data for a dataset is expensive, which is why machine learning models often use public data sets, these can be affected by:
- Dataset sizes. Bias is created when information on one group is either over or under represented. A dataset of employee information for a company of 100 employees is not going to be as representative as the dataset from a company with 50,000 employees. Stage 3 drug trials are usually conducted with a sample size of between 300 and 3,000 – 1% of the sample size created for TR Recruiter.
- Incomplete Data. When data on one group is more/less detailed than data on other groups.
- Historical Context. Amazon built a predictive AI model identifying suitable candidates for engineering positions. The dataset was from a period when women and minorities were under-represented, which is why Amazon discarded the system before it was put into use.
Recruitment Bias AHTs vs Humans***
Unconscious bias (eg recruitment bias created by humans) is widely acknowledged, but it is hard to quantify as it is unconscious, and, attempts to counter it will often result in affirmative bias, which is both a breach of the Equality Act, and limits the potential benefits to the employer of diversity and inclusion. The potential advantage gained by using an AHT in order to reduce recruitment bias is to be able to define, and empirically measure the bias, and use it where the bias within the AHT is lower than that within the current process.
- Ad Placement/Personalised Jobs Boards. Using AHTs that utilise ad placing algorithms to place job adverts on selected platforms (eg Facebook, Instagram etc) that target certain age, ethnic or gender requirements are responsible for restrictive bias. The data used to select the criteria often re-enforces stereo-types (in a 2019 Harvard Business Review research paper, ad placement algorithms for supermarket cashier positions were shown to an audience of 85% women). Personalised jobs boards data mine to find and repeat patterns as jobseekers and recruiters interact. These are vulnerable to small and incomplete datasets.
- CV Parsing. Often these use old solutions dressed up as new tech, recruiters have always asked ‘knockout questions’, these are now asked by chatbots or identified via CV screening tools. Some of these tools go further using machine learning to make predictions based upon past screening decisions. At first glance it may seem sensible for screening tools to model past hiring decisions, but they often reflect the patterns that many employers are actively trying to change through diversity and inclusion programs.
- Machine Learning to Predict Successful Employees. These tools recognize that soft skills are an important part of the mix, and attempt to use big data techniques to identify soft skill patterns from hard skill data. The data can be contaminated by the subjective measures used to identify high from low performers, these subjective measures often have elements of sexism, racism and other forms of structural bias. For example if an employer has never hired someone from a certain group, would the algorithm know how to evaluate such candidates effectively?
- Video Interviewing. Up to 500,000 data points can be collected in a 30 minutes interview asking 6 questions. Because datasets are expensive to build, a lot of this data is bought in which can result in sampling bias. An assessment of five commercial speech recognition tools – developed by Amazon, Apple, Google, IBM and Microsoft – found racial disparities in performance for African Americans as a result of insufficient audio data from this group when training the models. Classifications are not neutral and may be open to debate. Evidence suggests that it is not possible to reliably and accurately identify and label the variety of cross-cultural expressions of emotion and affect.
Why Using TR Recruiter Reduces Your Exposure to Bias
- TR Recruiter focuses on the assessment of soft skills. It uses patented proprietary technology (avoiding the inherent bias that often results from data scraping) to generate a soft skills assessment.
- By using an objective baseline – the Five Factor Model (Big 5), the TR Recruiter engine avoids bias contamination from subjective measures.
- Training the TR Recruiter involved creating a propriety dataset comprising of 300,000 individuals and over 5,000,000 facial images representing different ethnicities, cultures, ages and across gender.
- The TR Sample is truly representative of the UK working population. In the table below is a comparison between ONS data on UK working population and the training data used for TR Recruiter (TR).
| TR | ONS | TR | ONS | |||
|---|---|---|---|---|---|---|
| Aged between 16-19 | 8% | Ethnicity White | 38% | 81% | ||
| Aged between 20-29 | 14% | Ethnicity Asian | 23% | 10% | ||
| Aged between 30-39 | 22% | 24% | Ethnicity Black | 12% | 4% | |
| Aged between 40-49 | 19% | 22% | Ethnicity Mixed | 7% | 3% | |
| Aged between 50-59 | 20% | 22% | Ethnicity Other | 3% | 2% | |
| Aged between 60-69 | 11% | 9% | Male | 62% | 52% | |
| Aged between 70-75 | 6% | 1% | Female | 38% | 48% |
