GDPR and AI: Data Protection for AI Systems

Where AI systems process personal data

For data protection, it helps to split an AI system into its phases. Each phase raises its own questions:

• Training and fine-tuning: Does the training data hold names, customer data, staff data or free text about people? Was the data first collected for another purpose?
• The model itself: There is no general answer to whether a trained model "contains" personal data. In its Opinion 28/2024, the European Data Protection Board (EDPB) said a model only counts as anonymous if it is very unlikely that anyone can pull personal training data out of it. This must be shown case by case.
• Use: Prompts, uploaded files and outputs may hold personal data. So may logs and chat histories.
• Outputs about people: Scores, forecasts, ratings or generated text about a person are personal data in their own right, even when they are wrong.

Key GDPR requirements

Legal basis and purpose limitation

Each phase needs a legal basis under Article 6 GDPR. Special categories such as health data also need one under Article 9. For training, firms often rely on legitimate interest (Article 6(1)(f)). This needs a written balancing test. If you want to train on customer or staff data you already hold, check whether the new purpose fits the purpose you first collected it for.

Transparency

Under Articles 13 and 14, you must tell data subjects about the processing. For automated decisions as defined in Article 22, this includes useful facts about the logic behind them and their meaning and likely effects. The right of access under Article 15 also covers data an AI system has created about a person.

Automated individual decisions (Article 22)

Some decisions rest only on automated processing and have legal or similarly serious effects. Examples are turning down a job application, a loan or a contract. They are only allowed under narrow exceptions.

In its SCHUFA ruling (C-634/21, December 2023), the Court of Justice of the EU made one point clear. Working out a score can itself fall under Article 22 if a third party relies on it to a decisive degree. A "human review" that just rubber-stamps the AI's result is not enough.

Accuracy and data subject rights

Generative AI can make false claims about real people. The accuracy principle (Article 5(1)(d)) and the rights to correct and delete data are hard to put into practice in trained models. So filters on the output and clear processes for data subject requests matter all the more.

Data protection by design and security

Article 25 requires data minimization from the design stage on. Article 32 requires suitable technical and organizational measures. These include:

• pseudonymizing training data
• limits on who can access what
• rules on how long to keep prompts and logs and when to delete them
• protection against attacks that rebuild training data from models

Practical approach

1. List your AI apps: Keep a register of all AI systems in use, including AI features in software you already have. Update your record of processing activities to match.
2. Clarify roles: Are you the controller or a joint controller? Or does a vendor also use your data for its own purposes? In that last case, it cannot be a pure processor relationship.
3. Review contracts: Check for a data processing agreement under Article 28. Check that the vendor may not use your inputs to train its model. Also check server locations and the basis for transfers to third countries.
4. Data protection impact assessment: You need a DPIA under Article 35 when new technology is likely to bring high risk. This is often the case for AI systems that rate people or affect staff.
5. Write a usage policy: State which data staff may enter into AI tools and which tools are approved.
6. Make human oversight real: The people who decide must have the power, the skills and the time to challenge AI outputs.
7. Bring in the data protection officer early: Not just before go-live, but when you choose the system.

Relationship to the EU AI Act

The EU AI Act adds to the GDPR but does not replace it. For high-risk AI systems it requires, among other things, data governance, logging and human oversight. Much of this paperwork can be aligned with the DPIA. In 2024, the German Data Protection Conference (DSK) published guidance on "Artificial Intelligence and Data Protection". It helps firms choose and use AI apps.

Conclusion

You can use AI in line with the GDPR if you do three things. Assess each phase on its own. Check vendor contracts closely. And pair automated decisions on purpose with real human oversight. SecTepe helps bring together data protection, information security and AI governance.

Frequently asked questions

May employees enter customer data into ChatGPT and similar services?

Only if three things are true: the company has approved the tool, a data processing agreement is in place, and the vendor may not use inputs for training. Private accounts are not suitable for this.

Is it enough to anonymize data before training?

Truly anonymous data falls outside the GDPR. But this is hard to achieve with free text. Often the result is only pseudonymized, and the GDPR still applies to that.