A law professor at Vrije Universiteit Amsterdam spent two years investigating how an AI ban, unguided AI use, and structured training affect student performance—with surprising results: the group without AI access scored lowest both years. Researcher Thibault Schrepel admits: "I was wrong." He had assumed that AI without guidance would cause more harm than good.
Key Findings
- 66 students (2024) and 164 participants (2025) were randomly divided into three groups
- The no-AI group ranked last both years—despite deeper peer discussions
- Structured AI training delivered significantly better results in 2024; this advantage nearly disappeared by 2025
- Unguided AI use surprised positively: students learned to spot AI weaknesses through hands-on experience
The Experiment: Three Approaches, One Problem
Schrepel randomly assigned students in his "Law of AI" course to three groups. The task was identical: in teams of four to five, they had 20 minutes to improve a provision of the EU AI Act. Grading covered substance, clarity, proportionality, and innovation.
| Group | Condition | 2024 Result | 2025 Result |
|---|---|---|---|
| 1 | No ChatGPT | Weakest | Weakest |
| 2 | ChatGPT, no training | Middle | Middle |
| 3 | ChatGPT + training | Strongest | Similar to Group 2 |
The first group could not use ChatGPT. The second received AI-generated revision suggestions embedded directly in the text, without guidance. The third group received hands-on training in legal prompt engineering and learned to verify AI suggestions for consistency and accuracy. All students took the same exams: a multiple-choice test and a take-home exam.
Why the AI-Free Group Fell Behind
Without AI access, students made mostly minor rewording and clarifications. Only a few subgroups attempted substantive legal improvements. After ten to fifteen minutes, many groups ran dry—a phenomenon Schrepel calls "idea exhaustion." While the AI ban did foster deeper discussions, it wasn't enough to maintain work quality.
The second group, by contrast, often accepted AI suggestions uncritically. Students replaced terms like "shall" and "individual" with AI alternatives without understanding the legal implications. Every subgroup retained at least one misleading or legally irrelevant term from ChatGPT's output.
"I was wrong. He had assumed that AI without guidance would do more harm than good." – Thibault Schrepel on his original hypothesis
Only the trained third group engaged in genuine debate with the AI, testing different phrasings and questioning the substantive issues behind the assignment.
The Training Advantage Fades
In 2024, the trained group achieved significantly better results than the other two—especially on the more demanding take-home exam. By 2025, this advantage had nearly vanished. All three groups achieved similar performance levels.
Schrepel attributes this to growing familiarity with chatbots. Many students already use these tools in daily life, so formal training delivered less added value in 2025 than in 2024. However, ethical use and legal responsibility remain important teaching topics.
One finding held constant: the AI-free group scored lowest both years. Schrepel concludes that AI bans produce worse average outcomes than allowing AI access—though whether the advantage comes from structured training or mere hands-on experience remains an open question.
What This Means for Schools and Organizations
The study provides evidence in a debate intensely discussed in German schools and universities: should AI tools be banned or regulated? Schrepel's findings suggest that blanket bans are counterproductive. Instead, structured guidance—teaching how to use AI critically, recognize its limits, and verify results—may be key. For mid-market companies, this implies that employees using AI tools without training could produce quality problems. Targeted training could mitigate this risk.
Sources
Editorially owned by Ideal Syka. Sources and method: Newsroom & method. Tips and corrections: ai@i6eal.de.
