Executive Summary
The government seems more focused on creating a public image of progress that reinforces global power dynamics, rather than developing sustainable long-term policies to harness AI where appropriate and for public benefit. Short-term gains and public perception are taking precedence over thoroughness and leadership. The reliance on foreign capital and systems designed and operated by private companies already dominant in the market, combined with economic and technological pressure from the state where many of these companies are based (the United States), makes the current push for AI adoption across the public sector concerning.
The main concerns are threefold:
- Who sets and adjusts the agenda for utilizing AI systems?
- Does the current approach of relying on private sector providers benefit UK society, and at what cost?
- How to safeguard human rights while harnessing AI adoption. Currently, these systems are not subject to meaningful scrutiny. The public is currently exposed to the risks of public sector AI.
Yet there exist regulatory mechanisms with the potential to align AI adoption with human rights, all of which should be considered as part of public deliberation. The public must have meaningful input into shaping law, regulation, and policy on AI. Doing so is part of securing and shaping reliable and trustworthy AI systems that produce tangible benefits for all members of UK society, not merely generating benefits for technology companies and their backers.
At the same time, it is crucial that the burden of recognizing and challenging problematic and potentially unlawful systems, and mitigating AI harms more generally, is not placed on individual members of the public. As such, the regulatory proposals outlined here assign responsibility to the UK state and industry with which it shares partnerships.
Recommendations
These proposals aim to address key aspects of ongoing AI development and usage practices, from systems design and testing to management and oversight.
- Pre-deployment impact assessmentsthat account for the public sector equality duty, data protection law, and the specific human rights applicable to the domain of a particular AI system’s proposed deployment.
- Sunset clausesOn AI systems approved for deployment that pose a high risk to human rights.
- Use-case authorization.Requiring AI systems to be limited to specified usage constraints, which must be reassessed and reauthorized should they be proposed for repurposing.
- Periodic algorithmic auditing.While AI systems are in use, which can include commissioning expert technical evidence to assist judicial review cases.
- Mandatory incident reporting.Of AI systems in line with human rights requirements to inform future research, discourse, and policy.
- Amend AI procurement practice to avoid vendor lock-in.To foster competition and transparency, ensuring state agencies have access to the best available technology, and clearly outlining the criteria for contract awards.
- Develop open-source alternatives to proprietary AI systems.Aiming to enhance the public sector's AI development capacity, focusing on adaptability and independence.
- Institutional reform.Establishing an independent, government- and industry-free public oversight body for AI regulation, with adequate resources, a clear mandate, enforcement powers, and accountability to Parliament.
The paper was written for Open Rights Group by Dr Richard Mackenzie-Gray Scott. Richard is an Associate Fellow of the Bonavero Institute of Human Rights at the University of Oxford, and works across human rights, digital technologies, constitutional studies and international law and relations, encompassing research, teaching, policy engagement and legal practice.