Introduction

Generative AI has become an incredibly attractive and widespread tool for people across the world. Alongside its rapid growth, AI tools present a host of ethical challenges relating to consent, security, and privacy, among others. As Generative AI has been spearheaded primarily by large technology companies, these ethical challenges — especially as viewed from the vantage point of ordinary people — risk being overlooked for the sake of market competition and profit. What is needed, therefore, is a deeper understanding of and attention to how ordinary people perceive AI, including its costs and benefits.

The Meta Community Forum Results Analysis, authored by Samuel Chang, James S. Fishkin, Ricky Hernandez Marquez, Ayushi Kadakia, Alice Siu, and Robert Taylor, aims to address some of these challenges. A partnership between CDDRL’s Deliberative Democracy Lab and Meta, the forum enables participants to learn about and collectively reflect on AI. The impulse behind deliberative democracy is straightforward: people affected by some policy or program should have the right to communicate about its contents and to understand the reasons for its adoption. As Generative AI and the companies that produce it become increasingly powerful, democratic input becomes even more essential to ensure their accountability. 

Motivation & Takeaways

In October 2024, the third Meta Community Forum took place. Its importance derives from the advancements in Generative AI since October 2023, when the last round of deliberations was held. One such advancement is the move beyond AI chatbots to AI agents, which can solve more complex tasks and adapt in real-time to improve responses. A second advancement is that AI has become multimodal, moving beyond the generation of text and into images, video, and audio. These advancements raise new questions and challenges. As such, the third forum provided participants with the opportunity to deliberate on a range of policy proposals, organized around two key themes: how AI agents should interact with users and how they should provide proactive and personalized experiences for them.

To summarize some of the forum’s core findings: the majority of participants value transparency and consent in their interactions with AI agents as well as the security and privacy of their data. In turn, they are less comfortable with agents autonomously completing tasks if this is not transparent to them. Participants have a positive outlook on AI agents but want to have control over their interactions. Regarding the deliberations themselves, participants rated the forum highly and felt that it exposed them to alternative perspectives. The deliberators wanted to learn more about AI for themselves, which was evidenced by their increased use of these tools after the deliberations. Future reports will explore the reasoning and arguments that they used while deliberating.
 

Image

^{The participants of this Community Forum were representative samples of the general population from five countries - Turkey, Saudi Arabia, India, Nigeria, and South Africa. Participants from each country deliberated separately in English, Hindi, Turkish, or Arabic.}

Methodology & Data

The deliberations involved around 900 participants from five countries: India, Nigeria, Saudi Arabia, South Africa, and Turkey. Participants varied in terms of age, gender, education, and urbanicity. Because the deliberative groups were recruited independently, the forum can be seen as five independent deliberations. Deliberations alternated between small group discussions and ‘plenary sessions,’ where experts answered questions drawn from the small groups. There were around 1000 participants in the control group, who did pre- and post-surveys, but without deliberating. The participant sample was representative with respect to gender, while the treatment and control groups were balanced on demography as well as on their attitudes toward AI. Before deliberating on the proposals, participants were presented with background materials as well as a list of costs and benefits to consider.

In terms of the survey data, large majorities of participants had previously used AI. There was a statistically significant increase in these proportions after the forum. For example, in Turkey, usage rates increased from nearly 70% to 84%. In several countries, there were large increases in participants’ sense of AI’s positive benefits after deliberating, as well as a statistically significant increase in their interest. The deliberations changed participants’ opinions about a host of claims; for example, “people will feel less lonely with AI” and “more proactive [agents] are intrusive” lost approval whereas “AI agents’ capability to increase efficiency…is saving many companies a lot of time and resources” and “AI agents are helping people become more creative” gained approval. After deliberating, participants demonstrated an improved understanding of some factual aspects of AI, although the more technical aspects of this remain challenging. One example here is AI hallucinations, or rather, the generation of false or nonsensical outputs, usually because of flawed training data.
 

Image

Proposals

Participants deliberated on nineteen policy proposals. To summarize these briefly: In terms of whether and how AI remembers users’ past behaviors and preferences, participants preferred proposals that allowed users to make active choices, as opposed to this being a default setting or only being asked once. They also preferred being reminded about the ability of AI agents to personalize their experience, as well as agents being transparent with users about the tasks they complete. Participants preferred that users be educated on AI before using it, as well as being informed when AI is picking up on certain emotional cues and responding in “human-like” ways. They also preferred proposals whereby AI would ask clarifying questions before generating output. Finally, when it comes to agents helping users with real-life relationships, this was seen as more permissible when the other person was informed. Across the proposals, gender was neither a significant nor consistent determinant of how they were rated. Ultimately, the Meta Community Forum offers a model for how informed, public communication can shape AI and the ethical challenges it raises.

Assigned into teams of three or four, students were given an area of strategic competition to focus on for the duration of the class, such as computing power, semiconductors, AI/machine learning, autonomy, space, and cyber security. In addition to readings, each group was required to conduct interviews with key stakeholders, with the end goal of producing a memo outlining specific policy-relevant insights about their area of focus.

But the final project was only part of the grade. The instructors also evaluated each group based on how they had used AI to form their analysis, organize information, and generate insights.

“This is not about replacing true expertise in policymaking, but it’s changing the nature of how you do it,” Volmar emphasized.
 

Expanding Students’ Capabilities

For the students, finding a balance between familiar habits and using a novel technology took some practice. 

“Before being in this class, I barely used ChatGPT. I was definitely someone who preferred writing in my own style,” said Helen Philips, an MA student in International Policy and course assistant for the class.

“This completely expanded my understanding of what AI is possible,” Philips continued. “It really opened up my mind to how beneficial AI can be for a broad spectrum of work products.”

After some initial coaching on how to develop effective prompts for the AI tools, students started iterating on their own. Using the models to summarize and synthesize large volumes of content was a first step. Then groups started getting creative. Some used AI to create maps of the many stakeholders involved in their project, then identify areas of overlap and connection between key players. Others used the tools to create simulated interviews with experts, then use the results to better prepare for actual interviews.
 

For Jackson Painter, the class provided valuable practice combining more traditional techniques for developing policy with new technology.

“I really came to see how irreplaceable the interviewing process is and the value of talking to actual people,” said Jackson. “People know the little nuances that the AI misses. But then when you can combine those nuances with all the information the AI can synthesize, that’s where it has its greatest value. It’s about augmenting, not replacing, your work.”

That kind of synthesis is what the course instructors hope students take away from the class. The aim, explained Volmar, is that they will put it into practice as future leaders facing complex challenges that touch multiple sectors of government, security, and society.

“This is a new type of policy work,” he said. “It's accelerated, and it increases the depth and breadth students can take in. They can move across many different areas and combine technical research with Senate and House Floor hearings. They can take something from Silicon Valley and combine it with something from Washington. It's an extraordinary thing.”

Real-time Innovation

For instructors Blank, Felter, and Volmar, classes like “Technology, Innovation, and Great Power Competition” — or sister classes like the highly popular “Hacking for Defense,” and the recently launched “Entrepreneurship Inside Government” — are an integral part of preparing students to navigate ever more complex technological and policy landscapes.

“We want America to continue to be a force for good in the world. And we're going to need to be competitive across all these domains to do that. And to be competitive, we have to bring our A-game and empower creative thinking as much as possible. If we don't take advantage of these technologies, we’re going to lose that advantage,” Felter stressed.

Applying real-time innovation to the challenges of national security and defense is the driving force behind the Gordian Knot Center. Founded in fall of 2021 by Joe Felter and Steve Blank with support from  principal investigators Michael McFaul and Riita Katila, the center brings together Stanford's cutting-edge resources, Silicon Valley's dynamic innovation ecosystem, and a network of national security experts to prepare the next generation of leaders.

To achieve that, Blank leveraged his background as a successful entrepreneur and creator of the lean startup movement, a methodology for launching companies that emphasizes experimentation, customer feedback, and iterative design over more traditional methods based on complex planning, intuition, and “big design up front” development.

“When I first taught at Stanford in 2011, I observed that the teaching being done about how to write a business plan in capstone entrepreneurship classes didn’t match the hands-on chaos of an actual startup. There were no entrepreneurship classes that combined experiential learning with methodology. But the goal was to teach both theory and practice.”
 

That goal of combining theory and practice is a throughline that continues in today’s Gordian Knot Center. After the success of Blank’s entrepreneurship classes, he — alongside Pete Newell of BMNT and Joe Felter, a veteran, former senior Department of Defense official, and the current center director of the GKC — turned the principles of entrepreneurship and iteration toward government.

“We realized that university students had little connection or exposure to the problems that government was trying to solve, or the larger issues civil society was grappling with,” says Blank. “But with the right framework, students could learn directly about the nation's threats and security challenges, while innovators inside the government could see how students can rapidly iterate and deliver timely solutions to defense challenges.”

That thought led directly to the development of the “Hacking for Defense” class, now in its tenth year, and eventually to the organization of the Gordian Knot Center and its affiliate programs like the Stanford DEFCON Student Network. Based at the Freeman Spogli Institute for International Studies, the center today is a growing hub of students, veterans, alumni, industry experts, and government officials from a multiplicity of backgrounds and areas of expertise working across campus and across government to solve real problems and enact change.

Prepared for Diverse Challenges

In the classroom, the feedback cycle between real policy issues and iterative entrepreneurship remains central to the student experience. And it’s an approach that resonates with students.  

“I love the fact that we’re addressing real issues in real time,” says Nuri Capanoglu, a masters student in Management Science and Engineering who took “Technology, Innovation, and Great Power Competition” in fall 2024.

He continues, “Being able to use ChatGPT in a class like this was like having a fifth teammate we could bounce ideas off, double check things, and assign to do complex literature reviews that wouldn't have been possible on our own. It's like we went from being a team of four to a team of fifty.”

Other students agree. Feedback on the class has praised the “fusion of practical hand-on learning and AI-enabled research” and deemed it a “must-take for anyone, regardless of background.”

Like many of his peers, Capanoglu is eager for more. “As I’ve been planning my future schedule, I’ve tried to find more classes like this,” he says.

For instructors like Felter and Volmar, they are equally ready to welcome more students into their courses.

“Policy is so complex now, and the stakes are so high,” acknowledged Felter. “But what we’re seeing in these classes is a passion for addressing real challenges from students who may not have otherwise thought they have a place at the table of national security or policy. That’s what we want. The best and brightest future policymakers are going to have diverse skill sets and understand how to leverage every possible tool and capability available to meet those challenges. So if you want to get involved and make a difference, come take a policy class.”