3. Involvement of the public in a deliberation process to define acceptable risk and use of AI in government systems
3. Involvement of the public in a deliberation process to define acceptable risk and use of AI in government systems
We want the government to fund ongoing public discussions so that all New Zealanders can help decide how AI should be used in our country. If AI is going to radically transform the way that public services are delivered, we need to have a say in how that is done.
For example, if a government agency wants to use an AI system to review social housing applications, the public should weigh in on whether or not that is appropriate. We might collectively decide that it's okay for a machine to help sort the paperwork. We might also decide that a human official must always make the final decision to ensure that empathy and fairness aren't lost.
There are lots of areas where AI use requires some trade-offs. We can make the delivery of public services more efficient and therefore less costly for taxpayers, but that might increase the potential error rate and make it harder for people to correct those errors. Is that a trade-off we are willing to make as a country?
We might also want to collectively define some "unacceptable risk" areas that the government should just not go to. For example, we might collectively decide that we don't want our military to use lethal autonomous weapons. Or maybe we're okay with it! We don't really know, that's why we need to have the conversation.
Right now, decisions about which risks are acceptable and which benefits are worth the trade-offs are made behind closed doors, generally without public consultation or engagement. The government has even passed some AI-enabling legislation under urgency, removing the opportunity for the public to provide input through the Select Committee process. But AI is not just an IT upgrade; it is a technology that shifts how public power is exercised.
The impacts of AI systems will be felt by all of us, and all New Zealanders deserve the opportunity to have a say in how AI systems will be used. We need to stop making assumptions about what New Zealanders are okay with, and actually ask them to help draw the line between the public service they want and overreaching automation. People have a variety of concerns, ranging from immediate issues such as algorithmic bias and loss of privacy, to longer-term issues such as environmental damage and the future of work. These perspectives deserve to be heard.
We are calling for funded public deliberation processes - such as citizens assemblies and structured community workshops - to enable all of us to participate in a national conversation about government use of AI. What constitutes "high-risk" needs to be at least informed by the public's views and risk tolerances, and it needs to be grounded in our unique societal context. AI systems imported from overseas cannot automatically understand Te Tiriti o Waitangi obligations, or reflect the distinct values of our diverse communities. True public engagement gives people the time, space, and unbiased information to grapple with these complexities.
Seeking this input from the public and being transparent about the process and its outcomes is how we build genuine social licence. It moves us past broad anxieties about AI and allows us to be more specific in saying what is okay and what is not.
The Digital Council for Aotearoa New Zealand’s report on workshops around automated decision-making.
A citizens’ assembly in Switzerland was convened in 2025 to give guidance on national AI policy.
A report from the University of Edinburgh recommends the establishment of a ‘standing citizens’ assembly on AI and society’ (see e.g. p4).
The Ada Lovelace Institute in the UK ran a Citizens' Biometric Council to understand the public's view on biometric technologies like facial recognition.
Deliberative democracy processes such as Citizens Assemblies have been run successfully (on non-AI issues) in Auckland (Watercare), Tairāwhiti (Te Weu), Whanganui (District Council), and Wellington (City Council).
The Digital Council for Aotearoa New Zealand's report in 2021 was clear that everyday New Zealanders want to talk about automated decision making, but that they need structured, neutral spaces to do so.
Several submitters on the government's consultation on the Algorithm Charter in 2019/2020 pointed out that a top-down code would not work, and that creating trustworthy systems requires processes for the public, affected communities, and Māori to shape how these tools are developed and deployed.
The government's Responsible AI Guidance for the Public Services requires agencies to be completely transparent with citizens and maintain human supervision over AI outputs. However, this is guidance, and is also inconsistent with some approaches already taken by government agencies.
The Government Chief Digital Officer's Public Service AI Framework explicitly includes "social licence" as one of its core foundational pillars, asking agencies to understand how to build public trust and worker engagement in government use of AI.
Remember when we spent $22 million on the flag referendum and then ended up making no change? About $7 million of that was spent on public consultation and marketing as part of a "national conversation" on what the public wanted to have reflected in our flag. We're not saying that we have to spend that again here, but some funding should be allocated to enable a robust public deliberation process that tells us all what we want from AI use in government.
An example of this could be to hold an AI Citizens' Assembly, an independently run process where a randomly selected but demographically representative group of New Zealanders comes together to discuss government use of AI. Participants would be compensated for their time, with childcare/travel costs covered so that no one is priced out of participating. The Citizens' Assembly would seek to understand what issues people are worried about, and define the risk thresholds for when an AI system needs greater oversight and when it is sufficiently low risk. Experts can brief the participants on how current government AI tools work here and overseas, and where the risks lie.
This can be combined with other sources of public input, such as focus groups, surveys, and submission processes to enable all New Zealanders to make their voice heard. These various sources of input can be combined to identify acceptable use cases and allow government to understand the boundaries for what has public support and what is likely to lead to backlash.
If this works well, then it could be repeated every 1-2 years, to ensure that changes in public perception are detected and adjustments to the risk thresholds can be made. This could be legislatively mandated as a centrally funded and repeated activity to keep our understanding of the public's views up-to-date.
The lack of a clear public mandate is slowing things down already! Government agencies have to guess what the public and other stakeholders might feel about the deployment of AI, and in some cases are guessing incorrectly. This creates an environment of anxious hesitation. The cost of getting public feedback and building a map that defines the boundaries of acceptable practice is really not that high in comparison to the cost of failed AI deployment.
A lot of people currently have the "fear of the unknown". When people are given unbiased, plain-language information about how a technology works, the conversation naturally shifts away from broad sci-fi panic (or misinformation from overseas) towards constructive, pragmatic discussion. This doesn't slow down adoption - it ensures that when Ai is deployed, it is aligned to the social licence from the public.
The public aren't being asked to sit in a room and review code or audit algorithms - they are being asked to focus on the high-level principles and rules that will allow the government to navigate the high-risk AI systems safely, deciding which values must be protected, what safeguards are non-negotiable, and where human discretion must not be removed. This becomes the foundation that gives the government the social licence to innovate safely and confidently.