SCOTLAND — The Electoral Commission has called on ministers to introduce new legal controls over misinformation generated by AI chatbots ahead of Scotland's May Holyrood election. The call followed research by the think tank Demos showing that AI services gave voters misinformation in response to 34% of questions posed during a pre-election simulation.

Demos ran the simulation by putting 75 questions to five free AI tools, including ChatGPT, Google Gemini and Replika, about three real-life constituencies. The tests found that some tools invented fictitious scandals, gave the wrong date for the election, claimed incorrectly that voters in Scottish elections needed ID at polling stations, and placed candidates in the wrong contests.

"Voters want accurate information to help them engage with democracy and it is concerning that AI tools have made the spread of false or misleading information dramatically faster and more accessible than ever," Electoral Commission chief executive Vijay Rangarajan said. "The current legal framework should go further." He said ministers should introduce clearer duties on AI platforms to protect voters against misinformation and ensure algorithms did not mislead voters during critical election periods. Rangarajan also pointed to findings that half of voters in the 2024 general election had seen misleading information from AI.

Replika performed worst in the tests, with errors in 56% of its answers. It invented a candidate, made up accusations of nepotism by a candidate, and invented a date for a made-up expenses scandal. ChatGPT gave wrong information in 46% of its answers, fabricated an expenses scandal, returned inaccurate replies on voter eligibility rules, and got the date of the election wrong by two months. Google Gemini gave incorrect answers in 22% of cases, said a candidate had not taken a position on assisted dying when that person was a supporter, and wrongly claimed that a police investigation into a fraud case involving the Scottish National Party was continuing. Grok had an error rate of 9%, and its external links were frequently irrelevant or of poor quality.

A poll of 2,005 British adults commissioned by Demos found that 20% of voters had used AI chatbots or search tools for information about the parliamentary elections in Scotland and Wales and for English local councils, equivalent to 10 million people UK-wide. "This is a UK-wide, if not global, concern. The accessibility of these AI-tools – which are all developed and run by US corporations – is widespread in the UK, but we don't yet have the legislative framework to protect the public from misinformation, or our democracy from the knock-on impact of its circulation," Demos associate director Azzurra Moores said. She said ministers could quickly introduce legal requirements to make AI companies liable under UK defamation and electoral law, introduce mandatory safeguards on accuracy, and require AI firms to allow researchers to independently test their internal data and training sets.

A spokesperson for the Department for Science, Innovation and Technology said defending elections against AI misinformation threats was an absolute priority and that work was ongoing across government through its defending democracy taskforce, including closing loopholes in the Online Safety Act to ensure chatbots protected users from illegal content. "AI is critical to the UK's future prosperity and security. But if we want people to seize the benefits this technology promises, they need to be able to trust it," the spokesperson said.

A spokesperson for Replika said the chatbot was not designed for factchecking or search and that users were informed of that. "Replika is presented as a companion for reflection and self-expression, not as a source of factual or real-world information," the spokesperson said. OpenAI said that Demos's approach to testing ChatGPT was not how ChatGPT is typically used and appeared to use an out-of-date version of the product.