BEIJING — China released the third iteration of its AI safety governance framework this week. The updated framework includes risks posed by agents, cybersecurity threats from frontier models, and recursive self-improvement.

This release follows a series of regulatory steps taken by Beijing in recent years. In 2023, China issued its first national AI ethics guidelines. Prior to that, in 2022, China launched an algorithm registration system, requiring major platforms to register their algorithms with the Cyberspace Administration of China.

The regulatory push aligns with long-standing strategic goals. China’s latest Five Year Plan enshrines the goal of encouraging AI across the economy to promote growth. To support these objectives, local governments in China are offering funding and low rents to companies to help establish tech hubs.

Beijing has also moved to address specific consumer-facing applications. China issued national regulations this year aimed at curbing emotional dependency on companion AIs. Regarding the international dimension of AI governance, China’s foreign ministry stated that "fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance".

Dario Amodei, chief executive of Anthropic, offered a contrasting perspective on the geopolitical dynamics of AI development. Amodei stated that it is key "to keep democracies’ AI lead over autocracies as large as possible". He advocated for continued export controls to China on advanced chips and machinery. Despite his stance on maintaining a democratic lead, Amodei proposed a worldwide pacing of the frontier for AI development that would require cooperation with China.

Why It Matters

The release of the third iteration of China's AI safety governance framework marks an expansion of state oversight into emerging technical risks such as autonomous agents and recursive self-improvement. This move integrates AI safety into the broader economic strategy outlined in the Five Year Plan, which seeks to embed artificial intelligence across the economy to drive growth. The framework builds upon earlier regulatory mechanisms, including the 2022 algorithm registration system and the 2023 national AI ethics guidelines, creating a layered approach to governing AI deployment within China.

The divergence in approaches between Chinese authorities and Western industry leaders reflects the complex interplay between national security, economic competition, and global cooperation. While China emphasizes that confrontation disrupts global governance, figures like Dario Amodei argue for maintaining a technological lead through measures such as export controls. This tension exists alongside proposals for international cooperation on pacing AI development, reflecting the dual pressures of competitive advantage and shared safety concerns in the global AI landscape.

What's New

In 2023, China issued its first national AI ethics guidelines, which laid the foundation for subsequent regulatory frameworks, including the third iteration of the AI safety governance framework released this week. In 2021, China’s State Council issued the 'New Generation Artificial Intelligence Development Plan,' which outlined a roadmap for achieving global leadership in AI by 2030, aligning with the goals in the current Five Year Plan. In 2022, China launched an algorithm registration system, requiring major platforms to register their algorithms with the Cyberspace Administration of China, which is part of broader efforts to regulate AI deployment.

China's Five Year Plan has historically included specific technology development goals, such as the 13th Five Year Plan (2016–2020), which emphasized the growth of the high-tech industry and the development of artificial intelligence. The US government has previously imposed export controls on advanced semiconductor technology to China, most notably in 2020 when the Trump administration restricted exports of certain chips to Huawei. Dario Amodei, CEO of Anthropic, previously served as a senior researcher at the Machine Intelligence Research Institute (MIRI) and was involved in early discussions about AI safety and long-term risks. Dario Amodei is CEO and co-founder of Anthropic.

Timeline

2022: China launched an algorithm registration system. 2023: China issued its first national AI ethics guidelines. This week: China released the third iteration of its AI safety governance framework.