Your CV could soon be evaluated by an AI before any human ever sees it. Many companies are already using Large Language Models like ChatGPT to automate the pre-screening of job applications. But a new study from Princeton University now shows: These systems develop severe biases when performing this task – and stereotype candidates more strongly than humans would.
Key Facts
- Researchers at Princeton University investigated how LLMs behave in job application screening
- LLMs not only adopt human biases from training data, but develop additional bias through their work experience
- Agentic AI systems (those that remember things and learn from them) particularly amplify this effect
- The systems stereotype applicants more strongly than human recruiters
How AI Creates Bias on Its Own
Scientists have long known that LLMs absorb human biases from their training data. However, the new study points to an additional problem: Models develop biases from the experiences they gain during daily work. This is particularly critical because agentic AI systems – those that remember things and attempt to learn from them – entrench these patterns and incorporate them into their decisions.
This means: The longer an LLM screens applications, the stronger its biases can develop and become embedded.
Why This Is Problematic for Companies
For German companies introducing AI-driven recruitment tools, this creates a dual risk. First, there are legal consequences: Employment law prohibits discrimination in hiring – and an automated system that systematically disadvantages certain groups violates this. Second, there is a reputational risk: If it becomes known that a company overlooks qualified candidates due to AI bias, it can cause significant damage.
Additionally: Companies that miss top talent because an algorithm filters them out lose in the competition for skilled workers.
What Does This Mean in Practice?
This doesn't mean AI in recruitment is fundamentally wrong. But it means companies cannot blindly trust automated pre-screening. What's needed:
- Regular audits of AI systems for bias
- Human oversight of system decisions
- Transparency toward applicants about AI use
- Documentation of decision criteria
The Princeton University study thus provides important evidence for what many HR professionals already suspect: AI can be a tool in recruitment, but only if properly monitored.
Implications for German companies: These findings should alert HR departments. Companies using AI systems for application screening bear responsibility for their fairness – not just morally, but legally. Automation without oversight is a risk no company can afford.
Sources
Editorially owned by Ideal Syka. Sources and method: Newsroom & method. Tips and corrections: ai@i6eal.de.
