Shadowing Practice: Reinforcement Learning: Machine Learning Meets Control Theory - Learn English Speaking with Video

Choose a practice mode

Each mode opens a full-screen practice room.

349 sentences
If sentences are too short or too long, click Edit to adjust them.
1
Welcome back.
2
So I'm really excited to do this lecture on reinforcement learning.
3
I've been wanting to do this for a long time.
4
Those of you who know me know that I love control theory and machine learning.
5
And reinforcement learning is kind of at this sweet spot between these two super important fields.
6
Okay, so reinforcement learning is essentially a branch of machine learning
7
that deals with how to learn control strategies to interact with a complex environment.
8
Okay, and so one of the ways I think about this, the way I'm going to define this is
9
that reinforcement learning is a framework for learning how to interact with the environment from experience.
10
This is a very biologically inspired idea.
11
This is what animals do.
12
So through trial and error, through experience, through positive and negative rewards and feedback, they learn how to interact with their environment.
13
Okay, good.
14
So, before I jump in, I want to show some motivating videos.
15
I really like this one where reinforcement learning is used to learn how to walk in this artificial environment.
16
And there's a lot of papers like this where people use
17
reinforcement learning as kind of an optimization framework to learn how to control a complex system, in this case, a bipedal walker,
18
often in a simulated environment.
19
And this just looks really cool.
20
And it's a difficult control problem.
21
This is a really hard non -linear control problem.
22
Now, the goal would be to take what you learn here
23
and start to port that over into the real world to make better robots
24
and better actual physical agents that can interact with the world alongside us, to learn how to learn like humans and animals do.
25
So another video I love, this is my dog Mordecai, and my wife has trained him, this is a treat on his nose,
26
to hold the treat on his nose until she says, okay, after which he can then grab the treat and eat it.
27
This is not an easy trick to learn.
28
And this, again, this goes to show you anytime you, anybody who's trained an animal,
29
a dog or any other animal has done some type of reinforcement learning or reinforcement training.
30
Okay.
31
And so that's actually where the word reinforcement comes from is that in animal systems and human systems,
32
you reinforce good behavior with rewards like treats.
33
Okay.
34
And so that's kind of the whole name of the game here is learning a good control strategy
35
or a good set of actions through positive reinforcements.
36
Good.
37
So that's what we're going to talk about today.
38
I'm going to walk you through the framework.
39
So I want to disentangle, there is a reinforcement learning framework, kind of the framework for how you learn to interact with the environment.
40
And then there is a hard optimization problem for how you actually optimize the agent's actions or policies given that framework.
41
And those are kind of two pieces that we're going to talk about today.
42
And then in a future video, I'm going to talk about kind of deep reinforcement learning or reinforcement learning with modern techniques and deep neural networks.
43
And some of the incredible applications and performance that you can get out of those systems.
44
Good.
45
So, also, I'll point out you can follow updates on these videos at eigensteve on Twitter.
46
Please like, please subscribe, hit the bell so you get notifications, and comment below.
47
Tell me what you want to see more of.
48
Tell me what you like or don't like.
49
Oftentimes, people in the comments provide a lot of really important, useful information that I might have left out of these videos,
50
so I think it's also a big service to other people watching these.
51
All right.
52
So we're going to jump in, and we're going to build this reinforcement learning framework from the ground up from scratch
53
and so at the heart of it you start with an agent
54
and an environment and the agent I actually like the name agent
55
because it implies some agency the agent gets to take actions to interact with the environment
56
so in the first example
57
and I'm gonna have a few examples we're gonna talk about a mouse in a maze
58
so the agent is a mouse the environment is a maze
59
the mouse gets to measure its current state in the environment so it measures its state S, notice that it doesn't measure the full state.
60
The mouse does not have a top -down view of the whole maze.
61
It just knows where it is right now and where it was in the past.
62
And then the mouse gets to take some action A.
63
It gets to make some decision about what to do next.
64
Okay, so it could turn left, it could turn right, or it could go forward in this case.
65
And only until the very end of the maze does the mouse actually get a reward R.
66
So these rewards are very sparse, few and far between.
67
In this case, if it goes to the very end of the maze, it might get a piece of cheese.
68
Actually, my wife tells me that when they do training experiments for rats,
69
they really like Froot Loops and it looks adorable because a Froot Loop is gigantic to a mouse or to a rat.
70
But the moral of the story here is that this agent gets to make some decisions.
71
It has control over its actions, so it has agency.
72
And the environment, it gets to measure where it is in the environment, and occasionally it gets rewards.
73
Very occasionally it gets rewards. And
74
so part of the goal of this system is to learn what actions actually caused it to get a reward or not.
75
Okay?
76
And this is, in some sense, in the machine learning lingo, this is called semi -supervised learning.
77
So if the mouse got a reward at every single stage of the maze, if at every correct turn it got a piece of cheese,
78
that would just be regular supervised learning, and those rewards would be called labels.
79
They would tell you, yes, you did the right thing, or no, you did not do the right thing.
80
But because the reward here is time -delayed, it comes at the very end of the game or very sporadically,
81
and it's not linked to every single individual action, we call that time -delayed label a reward, and this becomes a semi -supervised learning problem.
82
So it's still supervised in the sense that there is supervisory feedback telling the agent what worked and what didn't,
83
but it's not nearly as much information as in classical supervised learning.
84
And that's one of the major challenges of reinforcement learning is
85
that these labels are extremely rare and it's very hard to tell what actions gave rise to actually getting that reward.
86
So this is a much harder optimization problem and oftentimes requires much more data and much more trial and error.
87
And I'm going to talk about that.
88
Good.
89
I also like to think about the game of chess or checkers or tic -tac -toe, basically games in general, where the agent,
90
basically there are some rules of the game
91
and you get to make a finite set of actions to interact with that environment.
92
Now, in the case of chess, it's interesting because the environment is not just the rules of the game.
93
There's also an adversarial opponent trying to beat you there.
94
So you're trying to beat the opponent
95
You're trying to checkmate the other to the other side and they are trying to beat you
96
And so that's really interesting is that the rules of this game are trying to you know, there's an active
97
Player on the other side in this environment
98
Good you also might be a terminator trying to rule the world or try to learn how to walk
99
I actually think this one's kind of funny that in the matrix
100
Neo is actually the agent, from a reinforcement learning standpoint, trying to learn the rules of the matrix, which is the environment.
101
Okay, so let's go back to the chess example, because I think the chess example really exemplifies a lot of the issues with reinforcement learning.
102
So we're going to use this as kind of our exemplar problem.
103
At the end of the day, the big challenge in reinforcement learning is to design a policy
104
of what actions to take given a state S to maximize my chance of getting a future reward.
105
That's all that this agent can do is decide on a policy.
106
Now, this is called a policy and not a control law for a lot of reasons, partly because the environment is not deterministic,
107
it's probabilistic, and so this policy is also gonna be probabilistic.
108
Okay, so my policy pi, given a state and an action,
109
basically, it tells me what is my probability of taking action a given that I'm currently in state s.
110
And again, this is probabilistic because I might decide on playing a mixed strategy.
111
A normal control system like swinging up a pendulum on a cart, the rules never change.
112
The system is always given by f equals ma, and so my control law also is deterministic and never changes.
113
But in the game of chess, maybe my opponent's kind of random, or maybe I'm just learning how to play.
114
So what I'm going to do with my policy is maybe 80 % of the time
115
I'm going to move my pawn this way,
116
but 20 % of the time I'm going to try this other move just in case my environment changes
117
or just in case something different happens that time.
118
So you're going to use a probabilistic policy to explore and optimize the rewards coming from your environment.
119
Good.
120
And you get to take actions A.
121
That's the whole point, is that once
122
Once you have this policy and you know what is the probability of taking an action given a state.
123
And then you just run that policy and you see how much rewards you get.
124
Good.
125
And this all happens in time.
126
So you take actions at time step one, time step two, and so on and so forth.
127
You measure the state at time one, time two, time three, all the way up to ST.
128
And there are rewards that you could be getting at each of these actions and each of these measurements.
129
Now most of the time these are going to be null or empty.
130
You're not going to get any rewards until maybe the very end of the game of chess
131
But in principle you could get rewards at some points along the way
132
And again the game of chess is a really good example of how hard this is
133
Because you might create a policy of what you think is the right thing to do in chess to beat your opponent
134
But you only get one reward at the very end of the game
135
Maybe I played a great game of chess, and I made one mistake, and I lose the game.
136
Do I throw away that whole sequence of actions?
137
How do you figure out what actions were good and what actions were bad?
138
That's a very, very hard optimization problem, and that's at the absolute heart of reinforcement learning.
139
Okay, so part of helping design a good policy is understanding what is the value of being in a certain state, s, given that policy pi.
140
So once I choose a policy, I can start to learn what is the value of each state of the system,
141
of each board position in chess, for example, based on what is the expected reward I will get in the future
142
if I start at that state and I enact that policy.
143
I'm going to say that again.
144
That's a mouthful.
145
So the value of a state s given a policy pi
146
is my expectation of how much reward I'll get in the future
147
if I start in that state and I enact that policy.
148
And there's this gamma to the t, which is a discount rate.
149
And so what this is saying is that I am slightly discounting my future rewards compared to my immediate rewards.
150
So, gamma is a constant between zero and one
151
that basically tells you how much you favor getting a reward right now versus far in the future.
152
And this is intimately related to economic theory,
153
psychology, that generally people are more eager to get a reward now than wait for a delayed reward much later.
154
But the basic idea is that you can start to understand this policy
155
and what policies are good or bad based on what are good board positions, what are good value functions.
156
And this kind of is how a human would play,
157
is that you might, so the set of all states of a chessboard is combinatorially large.
158
There's too many to count.
159
You could never hold them all in your mind.
160
But we start creating rules of thumb of what are good board positions.
161
So for example, if I take my opponent's queen, but I still have a queen,
162
I probably have a better expected chance of winning and getting a reward.
163
And so you might just count the points on the board, and that would give you some proxy for the value of a given state.
164
That's one very rudimentary value function that you could use.
165
And over time, as you play and gain mastery, you might refine your value function and get a better idea of what matters in the game.
166
Okay, and that's also then going to help you refine your policy to get to those good states.
167
Good.
168
So in this large framework, again, this is the reinforcement learning framework.
169
The goal is then to optimize your policy to maximize your future rewards.
170
So at the end of the day, it's an optimization problem to solve for pi.
171
So usually we think of our environment as not being fully deterministic like we do in classical mechanics
172
and classical control systems often and instead we think of our environment as being somehow there's a random or a stochastic component.
173
So these are called Markov Decision Processes, MDPs.
174
And what that means is that if we are in a state S now and I take an action A now,
175
there is some probability of me going to a new state S at the next time step.
176
And I could go to multiple different states
177
and it's kind of you roll a dice and you go to that next state.
178
So I actually think about backgammon.
179
I think that's a great example of a game that it has rules, it has a deterministic element,
180
but at every turn you're rolling die and that gives you this kind of random Markov decision process.
181
So there's a probability of going from my current state and action to the next state S.
182
And that again, that makes it hard to optimize these policies
183
and that's why these policies have to be probabilistic in nature because your environment is probabilistic in nature.
184
So the credit assignment problem I've mentioned before, it's this idea that because your rewards are often very sparse and infrequent,
185
it's very hard to tell what action sequence was actually responsible for getting that reward.
186
This issue was recognized as early as the 1960s by Minsky and it's been one of the central,
187
it's the central challenge in reinforcement learning and it has been for six decades.
188
This is the problem that people are still working on today is how to beat the credit assignment problem.
189
And so a couple of keywords I think are important are dense versus sparse rewards.
190
So again, the game of chess has very sparse rewards.
191
You only know if you win when you checkmate or
192
when you are checkmated and you don't necessarily get concrete rewards at intermediate intervals.
193
If you had denser rewards, if you were playing with a more knowledgeable, like master, and they were telling you move after move,
194
oh, I wouldn't move that because then I'll do this.
195
Or, oh, that's a really good move because that makes this structure that's a really strong, you know, position.
196
They would be giving you extra dense rewards and they would be helping you learn faster.
197
But in general, if you have sparse rewards, then reinforcement learning is very sample inefficient, to use machine learning terminology.
198
Meaning, if I only got sparse rewards, I would have to play many, many, many, many, many times.
199
I'd have to have tons of examples to learn a good optimal policy given those sparse rewards.
200
So, sparse rewards and the credit assignment problem make it very hard to learn through optimization what the right policy is.
201
And that's related to sample efficiency.
202
So, in general, what we do in a lot of systems is called reward shaping where even if you get an infrequent reward,
203
an expert human might build a proxy reward so
204
that you get more dense intermediate rewards on the way to this final reward.
205
And so an expert human would basically guide the learning process by giving more dense rewards intermediately.
206
That's called reward shaping.
207
Okay, good.
208
So now we're going to talk about how, again, the ultimate goal is to optimize this policy.
209
So, there's lots and lots of strategies for this optimization problem.
210
And remember, I'm going to go back and say all of machine learning and all of control theory, almost, are optimization problems.
211
You can pose these as hard, non -linear, non -convex optimization problems, and then in the case of machine learning, you solve them with data, okay?
212
In the case of control, you solve them subject to the constraints of the dynamics.
213
This is no different.
214
This is at the intersection of machine learning and control theory, and reinforcement learning is again a big optimization problem within this framework.
215
And so to optimize this policy S and A, given measurements of your rewards, given this sporadic feedback, there are lots of strategies.
216
So there's differential programming.
217
So reinforcement learning and game theory kind of grew up together and differential programming is one of the optimization techniques.
218
Monte Carlo is an old strategy for optimizing these policies.
219
Just try a bunch of stuff kind of randomly.
220
Temporal difference is like an optimal balance between differential programming and Monte Carlo.
221
So it kind of finds the sweet spot of both of these.
222
And it's model -free, so it doesn't require you have any model of the system.
223
And it's related to the Bellman optimization.
224
So Bellman was one of the pioneers of optimal control theory
225
and also laid a lot of the foundations that are used in reinforcement learning today.
226
Now, in the reinforcement learning problem, and this is true, again, for most of machine learning, is that there's this balance and control theory,
227
for that matter, between exploration and exploitation.
228
So, this policy, pi, usually we're going to parametrize this.
229
We're going to have some parameters.
230
We're going to try to optimize those parameters to win the game, to get the reward.
231
Now, how much effort, how much do I put into optimizing a strategy or exploiting a strategy?
232
And how much effort do I go to try new things that might not work, but might also give me better rewards,
233
things I've never tried before?
234
How much effort am I going to use to explore good policies versus to exploit a policy
235
I think is the best one?
236
And this is always a challenge.
237
I'm not going to talk too much about this.
238
I talk about this a lot in other videos.
239
But this is a fundamental challenge in machine learning and control theory is this exploration -exploitation balance.
240
And it's a big problem in reinforcement learning also.
241
Policy iteration is, so basically you set up a dynamical system where based on your rewards,
242
you iteratively update the policy to make it better and better over time based on new information, based on better information from new rewards.
243
That's policy iteration.
244
And there are lots of strategies to do this.
245
So, I'm just going to name a bunch of them.
246
So, you can use simulated annealing, evolutionary optimization, gradient ascent, and you can use all of the modern tools in neural networks and machine learning,
247
stochastic gradient ascent, atom optimization.
248
So a lot of really interesting new work is happening just in the last 10, 15 years using deep learning to optimize these policies.
249
Okay, good.
250
Okay.
251
So I'll just give you some cool examples.
252
So this is one of my favorite examples of learning how to catch a ball in a cup.
253
This is a fun kids game.
254
And so an expert human first off gives the robot like one example to show that it's possible.
255
And then after imitation learning, the robot gets through trial and error.
256
Notice that there's a white screen and the cup is blue and the ball is red.
257
So it's using visual information from a camera.
258
And after a few iterations, it's actually getting pretty close.
259
I think this isn't so different from how a child would learn.
260
A kid's not going to learn this in two trials or three trials.
261
It might take a few dozen times before it actually gets close
262
and then learns how to catch the ball in the cup.
263
After 45 trials, it's getting very close.
264
It bounced right off.
265
I don't know if it'll get it at 60.
266
I think it barely misses.
267
But it's getting very, very close after 60 to catching it in the cup.
268
And finally, after 100 trials, this system has actually learned the rules of the game, the physics, of how to get that ball in the cup.
269
Very simple robotic example, but it's also pretty interesting.
270
I mean, this video is not that recent, but very interesting to show that it is possible to learn a real physical system.
271
This is another example I love.
272
This is called the Pilko Learner.
273
I encourage you to go read all about Pilko.
274
So in this case, they're learning kind of how to swing up and stabilize a pendulum on a cart.
275
And again, they are using some combination of trial and error and a physical model to,
276
you know, learn how to do this very efficiently with very few samples.
277
So you couldn't learn how to do this without learning a model or without having a model of the physics
278
Newton's laws, F equals MA.
279
If you just, actually, I tried this.
280
I downloaded some code in MATLAB and played around with this just to learn if you could swing up a pendulum.
281
It took like eight hours on my laptop and thousands and thousands and thousands of trials.
282
Very sample inefficient to learn like the random control signal
283
that gets you near the upright position where you can start to stabilize it.
284
So a lot of trial and error if you don't have a model.
285
So the Pilko learner in some senses model is leveraging the fact that there is physics.
286
We do know physics.
287
We do have models to learn this much, much faster, much more efficiently, many fewer samples.
288
And I forget which trial we're on, but after trial 5 or 6 or 7,
289
it actually does learn how to get this thing up and stabilize.
290
I think trial 6 is going to get really, really close.
291
Let's see.
292
All right, so, and notice that a human does have to get this thing back down to zero.
293
It's guiding, the human's guiding this process all along.
294
All right, so in trial six, it's going to get really close and almost do it.
295
And I think in trial seven, it's actually going to figure it out.
296
You're going to have to go watch the video to see what happens.
297
Okay, so again, this is our framework for learning.
298
We're trying to optimize this policy.
299
One last thing I'll tell you about is QLearning.
300
So instead of just learning the policy and the value function separately, in QLearning, you can kind of learn them both at the same time.
301
So there's this Q function that's not just a function of the state S, it's a function of the state and the action.
302
And it tells you what is the quality of being in that state and taking that action.
303
So it kind of combines the value and the policy.
304
You can almost think of it as like a value function of the state and the action,
305
assuming I do the smartest thing in the future that I can, the best thing in the future.
306
And so I'll walk you through what this could look like.
307
So the way that you update this quality function is you take your old quality function
308
and then when you get a reward you basically update this alpha is a learning rate, gamma again is the discount rate,
309
and what you're doing this max A of Q in the future basically says I'm assuming
310
that I'm always doing the best thing I possibly can in the future
311
and if I do the best thing in the future, kind of what is the quality of my current state and action.
312
So I'm gonna say this again, this is a little, it almost seems a little circular,
313
but it's a really nice way of combining the policy and the value into one function that you can learn.
314
And again, you can learn this with a deep neural network nowadays.
315
It says, given a state S and an action A, and assuming I do the best thing I can in the future,
316
what is the quality of being in that state and taking that action?
317
And this is really nice, because if you actually know the quality function, then once I find myself in a state S,
318
I just have to look across all of the actions A and pick the one with the best quality.
319
So it's a really nice way of choosing an action given this quality function.
320
When I find myself in state S, I just pick the action that gives me the best quality and I enact that action.
321
And if I do that in the future, I will maximize my value and that gives me a policy.
322
So that's really cool.
323
Okay.
324
I guess another thing I think is super interesting is hindsight replay.
325
So again, when we talk about the credit assignment problem and the sparse rewards problem, in that inverted pendulum example that I ran in MATLAB,
326
it took a really, really long time before this thing actually got near the upright position where it could start getting rewards.
327
And so what you do in hindsight replay is instead of throwing out all of the data
328
that doesn't actually get you a reward, what you do is you say,
329
maybe my system, my set of actions would be good for a different reward.
330
So maybe in that ball in a cup example, if I didn't get the ball in the cup, maybe the ball went over here.
331
What I do is I look back, I replay that event, and I say,
332
well, maybe someday I'll actually want to do this other thing that I just did.
333
So I better remember that, and I would encode that that was a good action sequence for a different reward structure.
334
Not the reward I want of getting the ball in the cup, but this other reward of getting the ball over here.
335
And then by learning how to get into these different states, you get a lot more reinforcement, a lot more kind of artificial rewards,
336
and you learn more about the physics and the dynamics of the system, about this kind of enhanced value.
337
And so hindsight replay has been an absolutely critical advance in making these more data efficient
338
and learning harder tasks that involve a more complex state space.
339
And it's much more what a human would do, right?
340
Maybe I'm playing tennis, and I mess up some aspect of hitting the ball, and it goes in a different direction than I thought.
341
But maybe in the future, that's exactly what I'm going to want to do.
342
And so I'm going to catalog that back, and when I need to do that in the future, I'm going to have used that information.
343
So it's much more memory efficient.
344
Sorry, reward efficient, sample efficient.
345
Okay, good.
346
So in this video, I've talked about reinforcement learning, which is a framework for learning from experience how to interact with the environment.
347
So in the next lecture,
348
I'm going to talk about how to do this with neural networks and some of the really exciting advances in the field.
349
All right, thank you.

Download App

Everything you need to speak fluently

AI PronunciationScore every sentence
IPA PracticeMaster every sound
VocabularyBuild your word bank
Vocab GameLearn while playing

Same Channel

FOR YOU

Recommend Video

Vocabulary and speaking notes for this lesson

This speaking lesson is built on the video “Reinforcement Learning: Machine Learning Meets Control Theory”. The speaker keeps coming back to these words: reward, learning, action, policy, learn. This video has 349 sentences and 4836 words to shadow. The speech runs for 25:54. The speaker talks fast, about 187 words per minute, so expect linked and reduced sounds. 83% of the words are among the 3,000 most common in English; the rest is worth studying before you start.

Key vocabulary in this video

15 words from the video worth learning, with pronunciation and meaning:

WordPronunciationMeaning
reward noun/ɹɪˈwɔɹd/Something of value given in return for an act.
optimization noun/ˌɑptəmaɪˈzeɪʃən/The design and operation of a system or process to make it as good as possible in some defined sense.
framework noun/ˈfɹeɪm.wɜːk/A support structure comprising joined parts or conglomerated particles and intervening open spaces of similar or larger size.
chess noun/t͡ʃɛs/A board game for two players, each beginning with sixteen chess pieces moving according to fixed rules across a chessboard with the objective to checkmate the…
mouse noun/maʊs/Any mammal of the many rodent families (especially Muridae) that have a small body and a long tail.
interact verb/ɪn.təˈɹækt/To engage in communication and other shared activities (with someone).
maze noun/ˈmeɪz/A labyrinth; a puzzle consisting of a complicated network of paths or passages, the aim of which is to find one's way through.
physics noun/ˈfɪz.ɪks/The branch of science concerned with the study of the properties and interactions of space, time, matter and energy.
dense adjective/ˈdɛns/Having relatively high density.
efficient adjective/ɪˈfɪʃənt/Making good, thorough, or careful use of resources; not consuming extra. Especially, making good use of time or energy.
assignment noun/əˈsaɪn.mənt/The act of assigning; the allocation of a job or a set of tasks.
probability noun/ˌpɹɑ.bəˈbɪl.ə.ti/The state of being probable.
replay noun/ˈɹiːpleɪ/An act or instance of replaying (of playing something, such as a game, again); a replaying of (something).
neural adjective/ˈnʊɹəl/Of, or relating to the nerves, neurons or the nervous system.
label noun/ˈleɪ.bəl/A small ticket or sign giving information about something to which it is attached or intended to be attached.

Phrasal verbs you will hear

WordPronunciationMeaning
figure out verbTo come to understand; to discover or find a solution; to deduce.
get out of verbTo leave, exit or escape from; to become free of.
get through verb/ɡɛt ˈθɹu/To overcome; to endure.
go back verbTo return to a place or state after having been there at a previous time.
go back to verbTo stem from; to originate in.
go over verbTo look at carefully; to scrutinize; to analyze.
grow up verbTo mature and become an adult.
look back verbTo think of (past events) in retrospect (in hindsight).

Sentences worth repeating

Short, complete lines from the video that you can reuse in everyday conversation:

  • I'm going to walk you through the framework.
  • Tell me what you like or don't like.
  • Do I throw away that whole sequence of actions?
  • I'm going to say that again.
  • I think is the best one?

Grammar in this video

The structures the speaker uses most, with the exact words from the video:

StructureIn the video
Passive voice be + past participle — the focus is on what happens, not who does itis called · would be called · is always given
Present perfect have/has + past participle — a past action that still matters nowhas trained · has done · I've mentioned
Relative clauses who / which + clause — extra information about a person or thingmatrix, which is · learning, which is

Pronunciation to watch

The speaker uses 72 contractions and reduced forms, such as I'm, we're, I'll. Say them the short way, as you hear them.

  • The “th” sounds: thumb /ˈθʌm/, mouthful /ˈmaʊθfʊl/
  • The “sh” and “zh” sounds: optimization /ˌɑptəmaɪˈzeɪʃən/, efficient /ɪˈfɪʃənt/, differential [dɪfəˈɹənʃəɫ], occasionally /əˈkeɪʒənəli/, artificial /ˌɑː.tɪˈfɪʃ.əl/
  • Long words — get the stress right: optimization /ˌɑptəmaɪˈzeɪʃən/, probability /ˌpɹɑ.bəˈbɪl.ə.ti/, differential [dɪfəˈɹənʃəɫ], occasionally /əˈkeɪʒənəli/, artificial /ˌɑː.tɪˈfɪʃ.əl/

How to practise with this video

  1. Listen to the whole video once without speaking and note the words you do not know.
  2. Start at 0.75× speed, shadow it sentence by sentence, then go back to normal speed once it feels easy.
  3. Record yourself and compare with the original, paying attention to words like reward, optimization, framework.

What is the Shadowing Technique?

Shadowing is a science-backed language learning technique originally developed for professional interpreter training and popularized by polyglot Dr. Alexander Arguelles. The method is simple but powerful: you listen to native English audio and immediately repeat it out loud — like a shadow following the speaker with just a 1–2 second delay. Unlike passive listening or grammar drills, shadowing forces your brain and mouth muscles to simultaneously process and reproduce real speech patterns. Research shows it significantly improves pronunciation accuracy, intonation, rhythm, connected speech, listening comprehension, and speaking fluency — making it one of the most effective methods for IELTS Speaking preparation and real-world English communication.

Shadowing technique: read the full step-by-step guide →