The rise of artificial intelligence (AI) in recruitment processes offers the potential for significant improvements in efficiency, from sourcing candidates to screening CVs and assessing applicants. However, the use of AI in recruitment also raises important questions around fairness, privacy, and compliance with data protection laws.

In a recent audit of AI tools used in recruitment, the Information Commissioner’s Office (ICO) identified key areas where improvements are needed to ensure that AI systems are not only effective but also lawful and fair in their operation. The audit, which examined a range of AI providers and developers in the recruitment industry, found areas for improvement in the way personal data is processed, how candidates are informed about their data, and how AI tools mitigate the risks of bias.

Key Findings of the ICO’s Audit

The ICO’s audit uncovered several key issues that need addressing for AI recruitment tools to align with data protection laws:

• Personal data processing: Many AI systems were found to process personal information in ways that could be deemed excessive or unfair, with some tools collecting more data than necessary and retaining it indefinitely.
• Transparency: In many cases, AI tools were not sufficiently transparent about how candidates’ personal information was being used. Candidates were often unaware of how decisions were made or how they could challenge automated decisions.
• Bias and fairness: One of the most pressing concerns identified in the audit was the potential for AI tools to unintentionally introduce bias, either by screening out candidates based on protected characteristics (e.g., age, gender, or ethnicity) or by failing to accurately assess candidates in a fair manner.

Following the audit, the ICO has issued nearly 300 recommendations to AI providers and developers to help them improve their data protection practices and ensure compliance with the UK’s data protection laws, including the GDPR. These recommendations have been largely accepted or partially accepted by the organisations involved.

Ian Hulme, ICO Director of Assurance, commented:

“AI has the potential to make hiring processes more efficient and effective, but it also brings new risks that must be managed. If AI tools are not used in compliance with data protection laws, they could harm jobseekers by unfairly excluding them or compromising their privacy. Organisations considering the use of AI in recruitment must ensure they ask the right questions of providers and seek clear assurances about how personal data will be processed, stored, and used.”

Key Questions to Ask Before Procuring AI Tools for Recruitment

For organisations looking to integrate AI into their recruitment processes, the ICO recommends asking the following key questions before procuring any AI tools:

1. Have you completed a Data Protection Impact Assessment (DPIA)? A DPIA is a vital tool to assess the risks of using AI tools and ensure that any potential harm to individuals’ privacy is identified and mitigated. Recruiters should conduct a DPIA before deploying an AI tool, ideally as part of the procurement process, and update it as the tool evolves. This will help ensure compliance with the accountability requirements of data protection laws.
2. What is your lawful basis for processing personal data? Organisations must establish a valid lawful basis for processing personal data, whether through consent, legitimate interest, or another appropriate legal ground. Special care must be taken when processing sensitive data (such as race or health information) to ensure the processing is lawful and justified under specific legal conditions.
3. Have you documented responsibilities and set clear processing instructions? It’s crucial that both the organisation using the AI tool and the provider of the tool clearly define their respective roles in relation to data protection. Contracts should specify who is the data controller and who is the data processor. If the AI provider is a processor, organisations must set clear instructions regarding how data should be processed and monitored to ensure compliance.
4. Have you checked how the provider mitigates bias? The audit revealed instances where AI tools were unfairly filtering out candidates based on protected characteristics. Organisations must ensure that AI tools are designed to process personal data in a fair and unbiased way. They should ask the provider for evidence that bias mitigation measures are in place and ensure that fairness, accuracy, and bias are regularly monitored and addressed.
5. Is the AI tool being used transparently? Candidates must be informed about how their personal data will be processed by AI systems. Organisations should ensure that candidates are provided with clear, accessible privacy notices that explain how their data is being used and how AI systems make decisions or recommendations. Candidates should also be informed of their rights to challenge any automated decisions that may affect them.
6. How will you limit unnecessary data processing? The ICO audit found that some AI tools were collecting excessive personal information or retaining it for longer than necessary. Organisations should ensure that only the minimum amount of personal data needed for recruitment purposes is collected and that this data is not used for other unrelated purposes. This will help organisations comply with data minimisation principles under data protection law.

Moving Forward: Key Considerations for Ethical AI Use in Recruitment

The ICO’s audit findings underline the importance of transparency, fairness, and accountability when using AI in recruitment. As AI tools become more integrated into hiring practices, it’s crucial that organisations ensure their use complies with data protection laws and doesn’t inadvertently harm candidates or create discriminatory outcomes.

The ICO’s report provides a roadmap for organisations to navigate the complexities of using AI in recruitment, offering clear guidance on how to integrate these tools responsibly while protecting the rights of jobseekers.

Looking Ahead

The ICO will continue to monitor the use of AI in recruitment and other sectors, and has committed to providing further guidance as the technology evolves. As AI’s role in recruitment expands, organisations must stay vigilant about how these tools process personal data and ensure they are implemented in a way that fosters fairness and transparency.

Link to ICO article: Thinking of using AI to assist recruitment? Our key data protection considerations | ICO