The deployment of artificial intelligence (AI) in real-world systems intensifies data-protection challenges. A combined reading of the EU General Data Protection Regulation (GDPR) and the AI Act (referred to here as the MI Rendelet) makes clear that data subjects must retain the rights and remedies guaranteed by EU law, including rights related to solely automated decision‑making and profiling. Ensuring those rights in practice raises legal, technical and communication issues once AI systems are live.
When does an AI system amount to GDPR "profiling"?
Under the GDPR, profiling is defined as any automated processing of personal data to evaluate certain personal aspects relating to a natural person (for example work performance, economic situation, health, preferences, behaviour or location). The GDPR preamble further requires that a data subject be informed about the logic of the automated processing and its possible consequences.
AI use will often meet the criteria of profiling because
- the processing is automated by definition;
- AI performs analytical or evaluative operations on personal data; and
- those operations serve to analyse or predict characteristics of an identifiable person.
The decisive element is the purpose of the analysis: if the AI is used to evaluate or predict personal attributes of an individual, the activity falls within the GDPR concept of profiling. Assessing this typically requires multidisciplinary cooperation between legal, IT and data‑science professionals.
Transparency obligations when AI performs profiling
Examples of AI uses that commonly constitute profiling include personalised marketing, credit scoring, fraud detection, recruitment screening, chatbots that monitor and adapt to user reactions, and workplace monitoring with AI.
Even before widespread AI adoption, explaining profiling to data subjects was difficult: controllers must make clear who monitors an individual’s online behaviour, which activities are observed and how those observations lead to personalised content. AI complicates this because
- controllers first need to explain, in an accessible way, what AI is — although the AI Act provides a definition, interpretation can be complex even for specialists, and end‑users’ technical literacy varies widely;
- controllers must then explain the logic by which AI‑based profiling operates and why the system reached a particular conclusion — here the so‑called black‑box problem becomes salient, as many machine‑learning models’ internal decision processes are not fully transparent even to developers.
These tensions have recent judicial resonance. In the context of C‑203/22 (CK, Dun & Bradstreet Austria GmbH, Magistrat der Stadt Wien), an Advocate General opinion delivered on 12 September 2024 stressed that transparency does not necessarily require disclosure of every technical detail of an algorithm. Instead, concise, accessible information about the main decision factors and their relative importance — and how they affected the outcome — can suffice. The opinion also recognises that protecting business secrets is a legitimate concern, but that claim cannot be used as a blanket excuse to avoid providing the required information.
That approach is pragmatic: it does not demand full technical disclosure that most data subjects could not meaningfully understand, while insisting that AI systems remain explainable in substance.
Automated decision‑making: right to human intervention and special categories of data
Where AI does not merely make a prediction or recommendation but automatically reaches a decision with legal effect or similarly significant impact on a person, specific GDPR rules apply. The GDPR generally prohibits solely automated decision‑making producing legal effects or similarly significantly affecting the data subject, except when
- EU or Member State law expressly permits it;
- it is necessary for entering into or performing a contract; or
- the data subject has given explicit consent.
In any case, controllers must inform data subjects about the existence of automated decision‑making, the logic involved and the likely consequences.
The GDPR also grants data subjects the right to obtain human intervention, express their point of view and contest the decision. In practice, implementing that right is challenging because
- to be meaningful, human review must be substantive and consider all relevant data; if the AI’s reasoning is opaque to the controller, a genuine substantive review may be impossible;
- as a result, a common practical solution is to remove matters where a person requests human intervention from the AI decision pipeline entirely and to disregard the AI output — which in turn raises proportionality questions about whether AI was necessary in the first place and creates organisational and logistical burdens.
Additionally, the use of special categories of (sensitive) data in automated decision‑making is heavily restricted: such processing is only permitted in narrowly defined circumstances — typically explicit consent or legal authorisation based on significant public interest. Failure to filter sensitive inputs or allowing data subjects to supply special data into the system can generate serious compliance risks and higher supervisory sanctions.
Practical steps for controllers
Two measures emerge as essential for controllers:
-
Use AI systems with adequate documentation that explains their operational principles sufficiently to enable discharge of transparency obligations. The AI Act currently requires such documentation primarily for high‑risk AI systems.
-
Carry out a thorough legal assessment (including prior to deployment) of the system’s purpose and the planned data‑scope to determine whether profiling or automated decision‑making as defined by the GDPR will occur. Compliance with the AI Act does not automatically ensure GDPR compliance.
The AI Act also mandates that information about the AI system be taken into account within the GDPR data‑protection impact assessment framework.
Conclusion and open issues
This article — the final part of a series — sketched selected data‑protection issues arising during two lifecycle stages of AI: training and production deployment. The discussion touched only some of the intersections between AI and data protection. Other important topics remain to be explored in depth, including the legal status of different actors in AI operations (controller, processor, joint controllers), contractual and liability implications, practical difficulties of data‑protection impact assessments for AI, incident handling, AI processing of children’s data, data portability in AI contexts, and cross‑border data transfers.
As a light closing note, the author reproduces a joke generated by the legal AI “Harvey” on 19 September 2024: “Why doesn’t the GDPR consultant go to the beach? Because they’re afraid of leaving too much data in the sand!”
