Executive Summary
Government currently appears more interested in creating a public image of progress that reinforces global power dynamics, than in creating sustainable long-term policies to harness AI where suitable and for the public benefit. Expediency and optics are taking precedence over diligence and leadership. The reliance on foreign capital and systems designed and operated by private companies already boasting market dominance, coupled with the economic and technological coercion by the state in which many of these companies are domiciled (the United States), makes the current desire for AI adoption across the public sector troubling.
The related concerns are threefold:
- Who sets and amends the agenda on utilising AI systems;
- Whether the current approach of being dependent on private sector providers benefits UK society, and at what costs;
- How to safeguard human rights while harnessing AI adoption. At present, these systems are not subject to meaningful scrutiny. The public is currently exposed to the risks of public sector AI.
Yet there exist regulatory mechanisms with the potential to align AI adoption with human rights, all of which should be considered as part of public deliberation. The public must have meaningful input into shaping law, regulation and policy on AI. Doing so forms part of securing and shaping reliable and trustworthy AI systems that produce tangible benefits to all members of UK society, not merely generating benefits for technology companies and their backers.
At the same time, it is crucial that the onus to recognise and challenge problematic and potentially unlawful systems, and to mitigate AI harms more generally, is not placed on individual members of the public. As such, the regulatory proposals outlined here assign responsibility to the UK state and industry with which it shares partnerships.
Recommendations
These proposals aim to address key aspects in the ongoing practice of AI development and usage, from systems design and testing, to management and oversight.
- Pre-deployment impact assessments that account for the public sector equality duty, data protection law, and the specific human rights applicable to the domain of a particular AI system’s proposed deployment.
- Sunset clauses on AI systems approved for deployment that pose a high risk to human rights.
- Use-case authorisation requiring AI systems to be limited to specified usage constraints, which must be reassessed and reauthorised should they be proposed for repurposing.
- Periodic algorithmic auditing while AI systems are in use, which can include commissioning expert technical evidence to assist judicial review cases.
- Mandatory incident reporting of AI systems in line with human rights requirements to inform future research, discourse and policy.
- Amend AI procurement practice to avoid vendor lock-in and promote competition and transparency, ensuring state organs have the best available technology at their disposal, where it is also clear upon what basis contracts are awarded.
- Develop open-source alternatives to proprietary AI systems in order to increase AI development capacity for the public sector that is adaptable and independent.
- Institutional reform that allows for a public oversight authority to regulate AI, which is independent from government and industry, appropriately resourced, provided a clear mandate with enforcement powers, and accountable to Parliament.
The paper was written for Open Rights Group by Dr Richard Mackenzie-Gray Scott. Richard is Associate Fellow of the Bonavero Institute of Human Rights at the University of Oxford, and works across human rights, digital technologies, constitutional studies and international law and relations, comprising research, teaching, policy engagement and legal practice.