Prática de Shadowing: AI Strategy, Policy, and Governance | Allan Dafoe - Aprenda a falar inglês com vídeo

So I'm going to be talking about this cluster of words that we inquire about, strategy, policy, governance, sometimes we say cooperation.

But I want to first explain my preferred term, which is governance.

Some people, when they hear governance, they think regulation.

Governance is not just regulation.

In fact, regulation might be a very small part of governance.

Some people hear global governance, they think world government.

That is also not, does not follow.

Rather the term governance just refers to the whole mess of processes by which decisions are made.

And so that includes laws, regulations, policies, but also institutions and norms.

So we can begin provocatively with what we might call one theory of AI enabled governance, which is from Vladimir Putin.

leads an AI will rule the world." So this is, if you will, a theory of governance because it says how decisions will be made under an AI-enabled future.

Now, before proceeding, it's worth remarking that Putin was not staring ominously into the camera when he said it.

He was in fact encouraging Russian school children to do their science projects on robotics and other topics.

But this quote really did resonate around the world because I think get tapped into a fear that many people have,

that the sort of governance of all things of the world could be dramatically changed by AI, that power could shift, world order could change.

So that's why the question of the governance of AI, how decisions about AI will be made, are so important.

This leads to the normative definition, which is that we don't just want to think about what the processes are, we want good processes.

And by good, we mean something like effective, legitimate, inclusive, adaptive.

So I'm going to talk about a lot of things.

It will not feel like I've covered the whole space, and that's because I will have not done so.

And the reason that that is the case is because this problem is really hard.

The governance of AI will not be easy.

We can see this by thinking about the nature of AI as a general purpose technology, like electricity, the printing press, the combustion engine.

These general purpose technologies transform society, the economy, military in a deep fundamental way that's often hard to anticipate and very hard to govern.

So on the right hand side I list some of the properties

that are plausibly the case with AI and each of these makes it difficult to govern.

The fact that the benefits and the harms are

so diffuse makes it hard for political groups to mobilize together to address those harms and to realize those benefits.

The great uncertainty we have about what kinds of capabilities are coming

and what the implications of those capabilities will be makes it again very hard to build appropriate norms and regulations around it.

And so on with the rest of these properties.

To try to make sense of this at the Center for Governance of AI at the Future of Humanity Institute, we have been beginning work on a whole host of questions

and part of

that work has led to this research agenda where we try to articulate the main questions and tractable ways into the problem.

And this research agenda breaks the space up into four categories.

The technical landscape, politics, ideal governance, and then policy.

These four categories share a mapping with the conference organization.

So ideal governance is like the destination, right?

It's where we want to get to.

If we could all sit around the table and discuss calmly and rationally, what would we come up with?

Politics refers to the fact that it won't be a calm, rational, patient conversation around the table.

There will be interest groups.

There will be misunderstandings.

There will be coalitions.

There will be institutions with voting rules of different kinds, and that will shape what world we find ourselves in.

So we want to understand those political dynamics.

The technical landscape refers to what are the sort of technical constraints and possibilities made possible by AI and other technologies.

And then finally, policy refers to the lessons we draw from this for what we should be doing tomorrow.

What kinds of near-term steps can we take to steer us towards beneficial AI or AGI?

An analogy might help.

we are founding a city.

The technical landscape is the geographic landscape and perhaps the relative price of steel and aluminum and concrete.

The politics is again the interest groups, the values of different parties, the coalitions, the voting rules.

Ideal governance are the blueprints we come together and articulate for what the city could look like.

And then policy is what we're going to do tomorrow to make that happen.

I want to begin by distinguishing between two kinds of conservatism that I think scientists especially, it's helpful to distinguish for them.

And I'll illustrate that with a quote from Leo Zillard, the inventor of the neutron chain reaction.

From the very beginning, the line was drawn.

Fermi thought that the conservative thing was to play down his 10% possibility that a nuclear chain reaction may happen,

while Zillard thought the conservative thing was to assume that it would happen and take all the necessary precautions.

So as scientists, we don't want to make claims, statements that we can't support, right, with theory and evidence.

And that's a good virtue for scientific discourse.

But for policymakers, we want to take low probability, high impact possibilities seriously.

Because if we don't, some of them will be realized and we won't be prepared.

And so it's good to separate, while as scientists, we want to be calm and epistemically grounded when thinking about extreme possibilities from AI.

As policymakers, we want to allow ourselves the imagination to think what could come in five, ten, fifteen years so we can prepare ourselves for it.

So first I'll tell you a little bit about the technical landscape, but again I'm just going to dip into it, giving you some examples of research we've been doing.

So there's a lot of questions in this space.

We might ask, for example, what are the kinds of powerful AI systems that could emerge in the coming years?

What are their strategic properties?

Are they, say, offense biased or defense biased?

Do they lend themselves towards cooperation?

Or do they increase uncertainty and stochasticity and, say, power?

Where are the inputs and capabilities in the world, and how are they changing?

Inputs like compute, training data, talent.

Can we model AI progress so we can better anticipate what, in five or ten years, the landscape will look like?

Who will be the most prominent actors?

Can we forecast capabilities?

So some of the work that we might call mapping was done by Nick Bostrom and superintelligence, looking out into the future, trying to see what different advanced capabilities would look like and what the implications are.

Some other mapping work is done by Jeffrey Ding in his report,

Deciphering China's AI Dream, looking at what the current capabilities are in China for various kinds of AI

and trying to forecast how that could change into the future,

as well as what the constellation of actors in China produce in terms of goals for the country.

Some other work is done by Ben Garfinkel, looking at cryptographic systems, and we might simplify that and say blockchain,

could do for global coordination, global order, world order.

And I highly recommend this report as among the best analyses I've seen of what these really exciting technologies,

but also ultimately in many ways limited technologies, really do enable.

Some other work

that Ben has done is looking at how the offense-defense balance will change as the amount of resources spent by actors increases.

And this is relevant to AI because if AI scales up the resources available, say for autonomous weapons for cyber warfare, then it could change the character of conflict.

And this really interesting result we found was that for a number of different kinds of conflict domains,

as resources scale up, the offense-defense balance scales towards the defensive.

It doesn't go all the way necessarily to defense dominance, but it does become more defense biased.

And there's a number of other interesting questions, again, related to the strategic properties of advanced AI.

Some other work is surveying AI researchers at NeurIPS and ICML

and others to try to get some data on what the future might look like.

Now, of course, you shouldn't take these survey results as especially sort of, it's not from an oracle that knows the future, but it is one source of data,

and then we want to complement that with other kinds of data to try to get timelines.

I want to use this visual metaphor to clarify how we can think about AGI

100

and how we can think about our work in governance related to it.

101

So imagine the capability space or the task space is high dimensional, right?

102

10 to the 10 to the 100, Anthony suggested yesterday.

103

This, I can only represent two dimensions on this screen, so let's collapse it to two dimensions, say physics research and understanding humans.

104

AGI is not a point in this space, which some people might, you might think, because it's a word, it suggests it's a single thing.

105

AGI is sort of right here, when you get to human level on all capabilities.

106

Rather, AGI is a cloud, or a space, the northeast quadrant above this intersection.

107

And the reason I'm emphasizing that is that AGI is not one thing.

108

It's a huge space of possible kinds of technology.

109

AI systems right at that intersection look very different from AI systems up here or over here.

110

So that again is just a reminder that when we say AGI we're not describing a particular system, we have a clear vision in mind.

111

Now there's this other concept of transformative AI which has different definitions depending on who's using it.

112

I might use it to mean when an AI system that could radically change wealth, power or world order.

113

So, when would we see transformative AI?

114

We might graph it like this and everything, again, northeast of that.

115

And I think that's plausible because once we have actual AGI systems in this kind of simplest sense, that's very likely to be extremely transformative.

116

But probably the transformative implications will come sooner.

117

And this suggests that we might want to start from the present, that we understand relatively well,

118

and then try to extrapolate out under different pathways to the points where we see extreme transformative possibilities arising.

119

And that's what I'll describe now under the politics section.

120

So this is the big slide of some political challenges from near-term AI.

121

And again, I'm just focusing on near-term AI because they're easier to see.

122

But it's under this logic that if we can see them and understand them well, we can extrapolate out to when transformative possibilities start hitting and towards the implications of AGI.

123

And there's really each of these items we could talk about for an hour or a day or a whole conference.

124

So I'll just talk about a few.

125

One that we will be having a panel on is labor displacement and inequality.

126

That's clearly a very big issue that already many people in the world are concerned about.

127

And AI and automation could dramatically amplify and exacerbate those concerns.

128

There's concerns about influence.

129

If algorithms are ever more sophisticated at understanding our psychology and inferring our psychology from,

130

say, the digital traces we leave on Facebook and other online sites,

131

then what does that mean for the balance of power between marketers and consumers, or between governments and citizens?

132

Another political challenge that I want to emphasize, because most of these are sort of gloomy, or suggest AI is disruptive,

133

is that we might lose the innovation and the progress that we're seeing in AI if there's a fearful backlash.

134

So if trust in AI developers and in governments that are sponsoring that is lost, we could see a backlash and clumsy policy that follows.

135

And I'll share with you some survey results that are going public at the end of this conference that speak to this.

136

So this was a survey of Americans, and one of the questions we asked was to what extent they trust various actors to develop

137

or manage AI in the interest of the public.

138

And some takeaways are you're more likely to be trusted

139

if you are the US military or or if you're academic university researchers or scientists.

140

You're moderately trusted if you're a technology company, unless your name is Facebook.

141

And interestingly, this survey was actually done before the Cambridge Analytica scandal exploded.

142

So this, a loss of trust had been accumulating.

143

You're moderately trusted if you're an intelligence agency within the US government, but you are not trusted if you're the federal government or state government or the UN.

144

So this tells us something about at least Americans' perspectives,

145

and we'll be surveying Europeans and Chinese citizens going forward to understand how people think about the governance of AI.

146

Another interesting set of results we found relates to the demographics of support for developing AI and HLMI, which here you can think of it as almost AGI.

147

But before showing you the results, I wanna have you all sort of articulate your prior about the results.

148

So first, on gender, do you think men are more supportive of AI and HLMI, or women?

149

So everyone decide on your head, and then I'll have you raise your hands.

150

Okay, so if you think men are more supportive of developing AI and HLMI, raise your hands.

151

Okay, and if you think women are more supportive, raise your hands.

152

So, strongly leaning towards men, but some thinking women.

153

And how about education?

154

Do you think more educated people are more supportive of AI and HLMI or less educated people?

155

So raise your hands if you think more educated people are more supportive.

156

Okay, and less educated.

157

Great, so I'm very happy that the room didn't have consensus

158

because then I can say that there's value in the science that I'm about to present.

159

So we will learn something.

160

So the answer is the sort of prototypical person who is more opposed to AI is female,

161

less educated, poor, and without computer science background or experience.

162

And interestingly, this is for both AI and HLMI.

163

It correlates pretty strongly.

164

Some other less strong results related to religion and political partisanship, for example.

165

So this is one issue that hasn't broken yet on partisan grounds, though it might.

166

Speaking to a point Max made this morning, if we ask them about how sort of hopeful they are about HLMI,

167

in this case we ask them for the expected impact of HLMI, you'll see 12% chose extremely bad, possibly human extinction.

168

That's more, more than double, the proportion that said extremely good.

169

And then also unbalanced bad outweighs ever so slightly unbalanced good.

170

So Americans, at least, are not thinking that AGI is going to bring utopia, or they're not convinced of that.

171

And this is important for a few reasons.

172

One, these might be legitimate concerns that it would be good to address.

173

But two, this level of sort of negative expectation about what AGI means for them and for humanity

174

could again lead to that backlash, could lead to resistance to all the possibilities and the upside that we see.

175

So I think this is something we should be taking very seriously.

176

OK, so there's a number of other issues.

177

Again, I'm just going to jump in.

178

A report from early 2018 was the Malicious Use of AI report, which many of you may have seen, looking at the many ways that AI enables new malicious actions.

179

Again, it's not to say that AI is net negative, but it's worth being attentive to those possibilities.

180

Another new paper by Nick Bostrom is the Vulnerable World Hypothesis, which asks if technology becomes especially destructive.

181

So if one person could cause a lot of harm enabled new technology, what does that mean for world order and for the ideal governance?

182

Among these many issues, I think there's an underlying issue which I and others focus on, which is competition between firms,

183

countries, and especially great power security competition.

184

I see this as an amplifier and something that exacerbates many of these other issues.

185

And even issues like privacy, which you might think is an issue that a country can figure figure out for itself.

186

A well-governed country, say Sweden, should be able to formulate its own optimal policies with respect to privacy.

187

But competition puts constraints, or at least it forces on them trade-offs that they might not want to make.

188

For example, the EU might want to have stronger privacy legislation, but if they do that, they're worried that they will have difficulties cultivating new AI companies and AI champions.

189

So now they face this trade-off between economic prosperity and their sort of preferred values in how algorithms are deployed.

190

And this, I think, applies to almost all of these issues.

191

That competition between countries, and especially competition that touches on security, national security concerns, makes it much harder to solve these problems.

192

I wanna also complicate two metaphors that we often use for understanding risks from AI.

193

So we often talk about them as coming from accidents or coming from misuse.

194

Right, so an accident is something that arises

195

because the engineers weren't careful enough

196

or they didn't spend enough time looking at the system

197

or maybe the science was insufficiently developed or maybe the machinery is very complicated and so it's accident prone.

198

Whereas misuse is the kind of harm that comes from some malicious actor, usually a rogue actor, a terrorist,

199

a criminal, who gets their hands on the technology and then deploys it in a way that's against society's preferences.

200

These two perspectives are really useful, but they're incomplete.

201

And in fact, I think they miss a lot of the variation in where risk comes from, and also a lot of the opportunities for policy intervention.

202

So Remco and I suggest a structural perspective, which focuses on these various structural properties of society,

203

politics, the economy, that are more likely to generate risks.

204

And I have here John F.

205

Kennedy during the Cuban Missile Crisis as the sort of exemplary image, because I don't think we would call the Cuban Missile Crisis an accident.

206

It's not that the nuclear weapons systems were poorly built leading to the crisis, nor was it a case of misuse.

207

It wasn't a terrorist or a rogue actor outside the sort of legitimate political order that led to the crisis.

208

Rather, it was the legitimate leaders of two great powers who found themselves at the brink of nuclear war because of structural, strategic properties of the international system.

209

So I think a lot of our thinking has to be much more structural to really grasp where the risks come from.

210

We're looking at the levers of influence that the US government, for example, would have over AI companies in the industry

211

and vice versa to try and understand what the strategic game could look like in the future.

212

We're also looking at historical examples of attempts to control powerful technology,

213

like in this case depicted the conversations after World War II to control nuclear weapons.

214

So many of us weren't aware of this.

215

There was a serious conversation

216

that took place about the US giving up nuclear weapons to

217

the UN in order for there not to be an arms race in nuclear weapons.

218

This conversation did not succeed, but there's a lot of interesting lessons about how that took place,

219

including number one, that scientists can play an extremely important role politically and also enabling cooperation,

220

such as by finding technical possibilities that would otherwise be overlooked.

221

Okay, so moving on to ideal governance.

222

In many ways, this is the work that I think's been least done so far, and I applaud Max for both his book, but also at the two conferences,

223

Asilomar and here, really trying to have us articulate what are these positive visions that we can look towards.

224

Some work that we've done at FHI is this paper, Policy Desiterata for Super-Intelligent AI, where we try to articulate what are things that are,

225

what are sort of features that become more important in a world of advanced machine intelligence.

226

Some other work is again from the survey that is gonna be public in a few days, where we asked Americans what are the governance challenges that they regarded as most important and most likely.

227

And the takeaways here are one, they regard them all as pretty important.

228

So the bottom of this figure is 2.5.

229

That's somewhere between somewhat important and very important.

230

And the other takeaway is that there is variation in how Americans perceive these governance challenges.

231

Data privacy, cyber tax, and surveillance stand out as among the more salient to them.

232

And finally, it's good to do this work, this research, but it's even better if we can translate that into policy recommendations for the near term.

233

So I'll tell you about one.

234

This is called the Windfall Clause.

235

This was first articulated in Nick's book.

236

And the idea is, if some actor, some company, wins the AI windfall,

237

wins the AI lottery, they become this super corporation with the best AGI that's capturing all the markets,

238

it would be good if they redistributed a lot of that wealth to the world, to humanity.

239

And the idea behind this proposal is that it could be a legally committing obligation that they would voluntarily undertake now,

240

before when they're far away from this windfall, to give away the bounty in the event that they happen to win the lottery.

241

And what's really exciting is that, legally it seems to be permissible under Delaware law

242

to make this contractual commitment

243

so long as the expected costs at the time of the commitment are less than 10% of national annual taxable income.

244

So the basic idea is, as long as you're not giving away too much right away, and you're not, if the risk is sufficiently high that you're gonna win,

245

or the probability is sufficiently high or low that you're gonna win, then it's tolerable to share value,

246

and shareholders don't have a legal basis to sue against that.

247

So in summary, there's a lot of important questions work to be done.

248

And what's great is there's a lot of talent that's pouring into this field working on these projects.

249

So I look forward to, in the coming years, working with many of you to answer these questions.

250

Thanks.

O que é a Técnica de Shadowing?

Shadowing é uma técnica de aprendizado de idiomas com base científica, originalmente desenvolvida para o treinamento de intérpretes profissionais. O método é simples, mas poderoso: você ouve áudio em inglês nativo e repete imediatamente em voz alta — como uma sombra seguindo o falante com 1-2 segundos de atraso. Pesquisas mostram melhora significativa na precisão da pronúncia, entonação, ritmo, sons conectados, compreensão auditiva e fluência na fala.

Prática de Shadowing: AI Strategy, Policy, and Governance | Allan Dafoe - Aprenda a falar inglês com vídeo

O cenário: Governança da IA e conversas complexas

Trechos úteis e colocações para memorizar

Seu desafio de shadowing: Repita e imite!

O que é a Técnica de Shadowing?