Innovation, governance

Alongside the surge in AI-driven opportunities, the rapid development of the technology is also bringing a new set of risks and governance challenges, making it increasingly urgent to strike a balance between innovation and safety — another key topic highlighted at the forum.

Jenny Shipley, former prime minister of New Zealand, repeatedly raised concerns over the social and ethical implications of AI during the event, warning that the fast-emerging technology could inevitably replace certain jobs, widen the digital divide, and introduce new risks related to privacy and ethics.

"We must ensure that robots serve humanity, rather than challenge social ethics and human values," Shipley said.

Although there is rising acceptance of AI tech globally, users have tended to underestimate the potential safety risks associated with these technologies, cautioned Yu of the CAICT.

The CAICT said mainstream large language models worldwide still have 15 to 30 percent of safety-related issues that need to be addressed, and AI developers "bear significant responsibility" for strengthening the safety and robustness of AI technologies, he said.

Against this backdrop, Jiang Xiaojuan, former deputy secretary-general of the State Council, China's Cabinet, said that over the past decade, governance efforts have largely remained at the conceptual level, while the role of social sciences in shaping governance frameworks has been insufficient.

"As a general-purpose technology, AI has significant potential to enhance efficiency, quality and safety across industries. The priority remains to promote its development, but this must be balanced with both economic rationality and social acceptability," Jiang said.

She further explained that "social acceptability" means AI development should align with public values and consensus, rather than being driven solely by scientists, requiring broader public participation and discussion.

With China's Government Work Report this year calling for improved AI governance, Zhang, the CAE academician, also outlined concrete steps to advance implementation. Zhang proposed three key measures: clear labeling of AI-generated content — an area where China has already taken legislative steps; ensuring all AI agents can be traced back to accountable entities; and prohibiting unrestricted self-replication of AI systems.

Zhang also warned of emerging risks tied to the proliferation of AI-generated content.

"Currently, about 65 percent of online content is generated by AI, and this is creating a negative feedback loop in model training," he said, describing it as one of the most pressing challenges in AI governance.

At the global level, some convergence is beginning to emerge. Sam Daws, senior advisor to the Oxford Martin AI Governance Initiative, said there is growing alignment among major economies regarding governing AI in the real economy.

"Legislation in the European Union, the United States and China shows a high degree of similarity in key areas such as system testing, safe deployment and performance monitoring," Daws said, adding that multilateral efforts — including ASEAN's AI safety network and United Nations-led mechanisms — are providing viable pathways for global cooperation.