Recap: Charting California’s Future in AI Governance

CCST Hosted First Public Dialogue on the Findings of the AI Frontiers Report with the Co-Leads, Announced Legislative Academy on AI

Overview

At the epicenter of global AI development, California’s leadership in governing frontier models will influence national and international standards. During this live conversation aboard the Historic Klamath in San Francisco, the co-leads of the recently released Joint California Policy Working Group on AI Frontier Models report discussed the report and answered questions about California’s future in AI governance. The report, commissioned by Governor Newsom, focuses on frontier AI technologies at the cutting edge of capability and scale.

Hosted by CCST, this event marked the first public dialogue around the report’s findings. The co-leads are Dr. Jennifer Tour Chayes of UC Berkeley College of Computing, Data Science, and Society, Dr. Mariano-Florentino (Tino) Cuéllar of the Carnegie Endowment for International Peace, and Dr. Fei-Fei Li of the Stanford Institute for Human-Centered AI. At the event, CCST and Omidyar Network also announced the launch of a Legislative Academy on AI with additional support from Kapor Foundation.

Watch

Key Quotes

Chayes emphasized the need to use education—particularly through California’s strong public higher education systems—to empower individuals to benefit from AI, while mitigating harm and inequality. She highlighted the important role of CSUs, community colleges, and the UCs.

“Over the past few decades, as new technologies have come in, certain sectors of our population in California and beyond have been disenfranchised and further disenfranchised. Never the intention, but nevertheless the result. Generative AI is such an incredibly powerful technology that I believe it should be possible—and we should be able to work together on this—to re-enfranchise a lot of people who have been disenfranchised.”

Cuéllar stressed the importance of balancing innovation and regulation. He outlined the report’s three pillars: considering both risks and benefits; grounding governance in rigorous, evidence-based methods; and building trust through transparency and verification.

“In thinking about how to govern frontier AI, we must consider the benefits as well as the risks. The risks are not small. The risks involve misalignment in general, scheming of models that begin to find ways of fooling the users (to think that something’s happened when it’s not, or something’s not happening when it is), and uplift in the development of weapons. We’ve got models that have been tested recently that show a 94% outperformance in virology knowledge relative to experts. However, the benefits stretch across science, education, finance, healthcare. So consider both the benefits and the risks.”

Li discussed the human-centered approach to AI, calling for curiosity-driven public sector research and access to compute resources in universities. She emphasized the underexplored power of spatial intelligence and robotics, urging policymakers to prepare for what’s just “around the corner.”

“AI is a term that stirs such extreme emotions on so many sides, and for me, to help good policymaking, it’s so important to keep the scientific core in our hearts. We try to ground those in that impossibly boring middle because we have to respect evidence, respect scientific method, respect data. Which brings me to your question. What if the North Star is the balance? If we embrace being boring a little bit? Not hyperbolic on either side. How do we balance, the safety as well as the innovation? It’s been many years now that as an AI scientist, I feel I keep saying this: there’s nothing artificial about artificial intelligence. At the end of the day, this is a technology that’s made by people supposed to work for people, and it will shape our future.”

High-Level Summary

• Foundational Approach: Chayes outlined the process behind the report, including consensus-building across sectors and a strong emphasis on inclusive public feedback. The group avoided prescribing specific laws in the report, instead offering actionable principles that could guide flexible policymaking.
• Main Themes: Cuéllar summarized the report’s framework: (1) balancing innovation and risk, (2) grounding regulation in evidence and simulations, and (3) enabling transparency—“trust but verify.” He cautioned against rigid, compute-only thresholds and encouraged adaptive governance.
• Innovation vs. Regulation: Li stressed the importance of not stifling innovation, calling for investment in education, public compute infrastructure, and a broader understanding of AI beyond large language models. She highlighted embodied (e.g., robotics) and spatial (e.g., XR) AI as areas poised for rapid growth.
• Out-of-Scope Priorities: While the report did not cover workforce impacts or education policy in depth, the panelists flagged both as urgent priorities. Chayes discussed ideas for a curriculum that combines AI literacy with tool-building skills to empower disenfranchised communities.
• Global Influence: The panelists acknowledged that California’s leadership is being closely watched internationally. Cuéllar and Chayes noted global interest in the report and the potential for California to shape governance norms.
• Next Steps: All three called on states and institutions to support public interest research, clarify governance mechanisms, and promote equitable access to AI’s benefits through thoughtful legislation and investment.
• Legislative Academy: To close the event, CCST announced the launch of a Legislative Academy on AI with support from Omidyar Network and additional support from Kapor Foundation.

Media Requests

Contact Keleigh Friedrich.

Photo Gallery

Photos by Mikel Shybut / CCST.

Panelists

Dr. Jennifer Tour Chayes
Dean
UC Berkeley College of Computing, Data Science, and Society

Dr. Mariano-Florentino Cuéllar
President
Carnegie Endowment for International Peace

Dr. Fei-Fei Li
Denning Co-Director
Stanford Institute for Human-Centered Artificial Intelligence

Jason Elliott
Moderator
CEO
Versus Solutions

Opening Remarks by
Dr. Julianne McCall
CEO
California Council on Science and Technology

###

About the AI Frontiers Report

Prepared at the request of Governor Gavin Newsom, the report leverages broad evidence to provide a framework for policymaking on frontier AI, taking into account the importance of innovation and establishing appropriate strategies to reduce material risks. It surveys AI research representing multiple disciplines to derive policy principles that can inform how California approaches the use, assessment, and governance of frontier AI.

Learn more and read the full report | Governor’s Press Release

About CCST

The California Council on Science and Technology is a nonpartisan, nonprofit organization that engages leading experts in science and technology to advise state policymakers — ensuring that California policy is strengthened and informed by scientific knowledge, research, and innovation. Established via a unanimous vote of the Legislature in 1988, CCST has been making California’s policies stronger with science and technology for nearly 40 years.