Remedies against discriminatory decisions taken or prepared by Artificial Intelligence Systems European Union, Council of Europe and the Venice Commission’s Approaches
Schnutz Rudolf Dürr · Revista de drept constituţional · 2025
The increasing integration of AI systems into decision-making processes across both private and public sectors, driven by efficiency and cost reduction, presents significant human rights challenges, particularly concerning discrimination.As Large Language Models advance, non-technical users can easily obtain useful results, but these must be thoroughly checked for errors and AI hallucinations.Many high-risk AI applications, as defined by the EU AI Act, can directly impact human rights in areas such as education, employment, healthcare, credit assessments and even the judiciary.While human oversight is often required, it may quickly devolve into superficial plausibility checks, leading to a high risk of perpetuating bias present in AI training data.On the basis of recommendations from the Venice Commission, this article highlights the need for individuals to have to access training data when appealing biased AI-based decisions.It suggests that specialised experts or civil society organisations, potentially using AI themselves, will likely be crucial in establishing such proof.The EU AI Act establishes a right to explanation for high-risk AI-based decisions and it mandates AI providers to maintain documentation on data type, origin, and curation methodologies of AI training.As individual decisions will typically refer to personal data, the General Data Protection Regulation (GDPR) provides additional complaints.For non-EU Member States ratifying the Council of Europe Framework Convention on Artificial Intelligence, adopting similar rules on training data documentation will be important to ensure effective complaint mechanisms.The effectiveness of appeals against biased AI-based decisions will depend on judicial interpretation and the work of supervisory authorities in ensuring the availability of training data information.Requiring AI providers to maintain records of training data should not be viewed as burdensome "red tape" but as a critical 1 Research for this article was partially done with the assistance of artificial intelligence.The statements made in this article are those of the author only and not of the Council of Europe Schnutz Rudolf DÜRR