Shadowing Practice: Give me 31 minutes, I will make PyTorch click forever - Learn English Speaking with Video
Choose a practice mode
Each mode opens a full-screen practice room.
498 sentences
If sentences are too short or too long, click Edit to adjust them.
1
Give me 31 minutes.
2
I will make PyTorch click forever.
3
If PyTorch has ever felt intimidating, don't worry, you're not alone.
4
Every training script has these five lines.
5
Predict, measure the loss, compute the gradients, update the weights, and clear the gradients.
6
So what does backward really compute?
7
What does step change?
8
And why clear anything at all? But it's actually simple.
9
We'll walk through one tiny real example,
10
predicting what a New York taxi ride costs from 36 real trips first by hand then with these five lines
11
and watch it get close to the taxi meters own rate by itself by the end
12
you'll have mastered all five lines in 31 minutes we'll understand this whole board one stop at a time tensors
13
and what we can do with them how a model predicts
14
and measures its loss gradient descent by hand autograd and finally the professional loop let's get started
15
The first stop is Tensors, and to build one, we first need some data.
16
We'll use one real data set the whole way.
17
New York yellow taxi rides from the city's public trip records.
18
On three days in January 2024, a Monday, a Wednesday, and a Saturday, we took the first ordinary ride in every two -hour slot.
19
For each one, we wrote down what the meter charged.
20
Look at the boxed row.
21
It's Monday's noon ride, 1 .38 miles for $11 .40, and we'll follow it all the way through.
22
How do we hold these fares?
23
We use a tensor, which is just a grid of numbers.
24
It can have as many axes as we need, one for a row, two for a table like ours.
25
Look at the code.
26
We give Torch .Tensor our list of rows, one row per day.
27
We get back a grid with a shape, 3x12.
28
But which of those two numbers, 3 and 12, is which?
29
Each direction of the grid is called an axis or a dimension, DIM for short.
30
PyTorch counts them from zero.
31
DIM zero runs down the days and there are three.
32
DIM one runs across the two hour slots and there are twelve.
33
The shape gives the size of each DIM.
34
And when there's no data yet, how do we make a tensor of some shape?
35
Look at the code.
36
Torch .zeros, 2, 3, gives a grid full of zeros.
37
The 2 and the 3 are its shape.
38
2 along dim 0, so 2 rows, and 3 along dim 1, so 3 columns.
39
Torch .ones, 2, 3 makes a grid of that shape full of ones.
40
What if we want a shape full of some other value?
41
Torch .full takes the shape as one tuple, and then the value.
42
Here's 7.
43
And torch .i makes a 3x3 grid with 1s on the diagonal and 0s everywhere else.
44
Let's make random numbers next, the kind a model starts from.
45
Torch .rand draws each one between 0 and 1.
46
Torch .rand draws them around 0 from a bell curve.
47
And what about counting?
48
Torch .a range of 12 counts from 0 up to 11, just like Python's range.
49
The end, 12, is left out.
50
The main point.
51
A tensor is a grid of numbers and its shape says how many there are along each dim.
52
That's the first stop done.
53
Our fares sit in a tensor and we can build new ones from a shape.
54
On to the second stop, tensor operations.
55
What can we do with our table of fares?
56
Here are 12 of them, running on our table at once.
57
Each one answers a question about the rides.
58
Don't try to read them all now.
59
We'll go through the main ones together, one at a time.
60
How do we get one day's rides?
61
We pick by index, like in a Python list.
62
Look at the code.
63
Fares of 0 is row 0, which is Monday's 12 fares.
64
Picking one day drops the day axis, so the shape is just 12.
65
What about one time slot on every day?
66
Inside the brackets, a comma separates the axes.
67
First dim 0, the days, then dim 1, the slots.
68
A colon on its own means all of that axis.
69
So fares of colon 6 is every day, slot 6.
70
Slots start at midnight and last two hours each, so slot 6 is the noon ride.
71
Next, let's take the first few slots.
72
A slice takes a range and keeps the axis.
73
So fares of colon, colon three, is every day's first three slots from midnight to six in the morning.
74
That's a three by three grid.
75
Which rides cost more than $30?
76
Look at the code.
77
Fares greater than 30 gives a grid of true and false, shaped like fares.
78
It's called a Boolean mask, and we'll call it pricey.
79
Index with it, and we can pull out just the true cells.
80
That's five rides over $30.
81
What if every ride were capped at $30?
82
Torch .where of pricey, 30, and fares checks the mask sell by sell.
83
It puts 30 where the mask is true.
84
Everywhere else it keeps the original fare.
85
What's the average fare each day?
86
The days run down dim zero, so let's try fares .mean with dim equals zero first.
87
Pause and guess.
88
Will we get three numbers or 12?
89
Watch what happens.
90
It's 12, one average per time slot, and we wanted one per day.
91
Why 12?
92
This is important.
93
The DIM you pass is the axis that gets aggregated away.
94
Aggregating means combining many numbers into one, like taking their average.
95
So DIM equals zero average the days away, and the slot stayed.
96
We want the slots gone, so we pass DIM one.
97
Look at the code.
98
Fares .mean with dim equals 1 averages each day's 12 fares into one number.
99
That's about $19 .63 on Monday, $14 .32 on Wednesday, and $16 .24 on Saturday.
100
What was the priciest ride each day?
101
Max also takes dim equals 1, but it gives back two tensors, the highest fare and the index of its slot.
102
On Monday, it's $45 .70 at 10 in the morning.
103
On Wednesday, it's $44 .30 at 4 in the morning.
104
And on Saturday, it's $33 .80 at 6 in the morning.
105
What about the three priciest rides each day?
106
Top K of 3 gives the three largest in order and their slots.
107
On Monday, that's $45 .70, $43 .60, and $38 in slots 5, 2, and 1.
108
Here, Dim 1 isn't aggregated away.
109
It shrinks from 12 to 3.
110
Pop quiz.
111
Fares has shape 3 by 12.
112
What shape does fares .max with dim equals 0 dot values have?
113
Is it A, 3, 1 per day?
114
B, 12, 1 per slot?
115
C, 3 by 12, the whole grid?
116
Or D, just one number?
117
Pause the video here and pick one.
118
The answer is B.
119
The DIM you pass is the axis that gets aggregated away.
120
DIM 0 runs down the days, so max aggregates the days away and keeps the highest fare in each of the 12 slots.
121
Choice A is what DIM equals 1 gives, one number per day.
122
Choice C keeps both axes, but max with a DIM always aggregates one away.
123
Choice D is fares dot max with no DIM at all, the single highest fare.
124
How do we get one row per time slot instead of one per day?
125
Dot T for transpose swaps the two axes, so the 12 slots become 12 rows with one column per day.
126
Let's see each day in quarters.
127
Reshape 3, 4, 3 keeps the 36 numbers but gives them three axes.
128
That's three days along dim 0, four quarters of the day along dim 1, and three slots in each quarter along Dim 2.
129
Now a mean over Dim 2 averages the slots away, leaving each quarter's average.
130
On Monday, the night rides averaged about $32 and the evening ones were under $10.
131
And Flatten goes the other way.
132
It puts all 36 fares in one long row.
133
Monday first, then Wednesday, then Saturday.
134
We'll need it in a minute.
135
You'll meet plenty more in real code, so here are the everyday ones on one card run on a small 2x3 grid.
136
Feel free to pause.
137
Some in argmax work like mean and max.
138
Each one aggregates an axis away, and the at sign is the matrix multiply rows times columns.
139
Here it multiplies x by its transpose x .t, which we just met, and star multiplies cell by cell.
140
And here are a few more.
141
View changes the shape, like reshape does, and permute reorders the axes, like .t does.
142
Unsqueeze and squeeze add and drop an axis of size 1.
143
Gather picks one cell per row by index and clamp clips every number into a range.
144
The main point?
145
We pick numbers by their index and in mean, max or sum, the DIM we pass gets aggregated away.
146
We have 36 taxi rides.
147
How does a model learn from them?
148
That's the second stop done.
149
Operations pick, combine and rearrange the tensor's numbers.
150
A model learns by guessing first, so the third stop is predicting.
151
We'll pair each ride's miles with its fare and fit a line.
152
The model should guess what every rider wants to know.
153
How much will this ride cost?
154
Each ride's record also says how far it went.
155
So we have a second table of miles shaped like the fares.
156
Flatten both and we get 36 pairs, miles in, fare out.
157
We'll call them M and F.
158
How do we see M and F as a table of pairs?
159
Look at the code.
160
Torch .StackWithDim equals 1 puts the two rows side by side along a new axis.
161
That gives 36 rows of miles and fares.
162
Torch .Cat would glue them end to end instead into one row of 72.
163
So how much should a ride cost?
164
New York's rate card has an answer.
165
It's $3 to start, plus 70 cents every fifth of a mile.
166
A mile is five fifths, so that's 350 a mile.
167
Can a model find a rule like that from our 36 rides alone?
168
We know the meter's rule so we can check.
169
Almost every ride sits above the cards line, and we'll see why.
170
Let's keep the model as simple as it gets.
171
It's a line with two numbers to learn.
172
The first is a weight, W, the price per mile, which sets how steep the line is.
173
The second is a bias, B, the price to start, which lifts it.
174
Its output, f -hat, is the predicted fare.
175
From here on, we'll call all the numbers a model learns its weights.
176
PyTorch calls them parameters.
177
A line like this is called linear regression.
178
We start from a lazy guess.
179
W equals 1 and B equals 0.
180
That's a dollar a mile and nothing to start.
181
Let's work out one prediction by hand.
182
Monday's noon ride went 1 .38 miles.
183
So the guess is 1 times 1 .38 plus 0, which is $1 .38.
184
The meter charged $11 .40.
185
The miss is the guess minus the fare, so it's 1 .38 minus 11 .4.
186
That's minus 10 .02, about $10 short.
187
What about every other ride?
188
One line, W times M plus B, does that for all 36 rides at once.
189
All 36 miles times W plus B give 36 predictions?
190
PyTorch reuses the 1W and the 1B for every ride, and that's called broadcasting.
191
The math writes a prediction as F -hat, and the code calls it Pred.
192
The main point, a model turns an input into a prediction, and ours turns each ride's miles into a guess at its fare.
193
Our lazy line now predicts all 36 fares, but every guess is too low.
194
Fourth stop, the loss function.
195
How wrong is the prediction?
196
Each prediction misses by some amount.
197
So how do we turn 36 misses into one number?
198
The naive tries to just average them.
199
Right now, every prediction is below its point, so nothing cancels yet.
200
But watch what happens once the line runs through the points.
201
Misses above and below cancel out and even a bad line could average zero, so we square each miss instead.
202
Monday's noon miss is minus 10 .02.
203
Squared, that's 100 .4.
204
Square all 36 misses the same way.
205
Add up the 36 squares and we get 9 ,432 .56.
206
Divide by 36 and we get 262 .02.
207
Pause here if you want to check the division.
208
That's the mean squared error.
209
The lazy line's loss is big because it misses by a lot.
210
The main point, the loss is one number that says how wrong the model is.
211
Pop quiz.
212
One line misses one of our 36 rides by $6.
213
Another misses 3 by $2 each. Which loss is bigger?
214
Is it a, the one with the single $6 miss?
215
B, the one with the three $2 dollar misses?
216
C. They're equal? Or D.
217
It depends on the fares?
218
Pause the video here and pick one.
219
The answer is A.
220
Squaring makes one big miss count far more than several small ones.
221
One six dollar miss squares to six times six, 36, while three two dollar misses square to four each,
222
and 4 plus 4 plus 4 is 12.
223
Choice B looks fair since both lines are off by $6 in total, but its squares add up to just 12.
224
Choice C would be true if we added the misses without squaring them.
225
Choice D is wrong because only the misses go into the loss.
226
Which way should W and B move to lower the loss?
227
Our loss squares the misses and averages them into one number.
228
Its derivatives say which way to move W and B, and they lead to the fifth stop, gradient descent.
229
Compute the gradient, then step against it.
230
What is the gradient?
231
It's the derivatives of the loss, one per weight.
232
DL by DW is how fast the loss changes when only W moves and B holds still.
233
That's the slope of the loss along W, DL by DB does that for B.
234
Two weights, so the gradient is two numbers.
235
How do we compute DL by DW?
236
and dl by db.
237
Let's write down what we have.
238
Each prediction is w times m plus b, and the loss is the mean of the squared misses.
239
The derivative of x squared is 2 times x, so each squared miss has derivative 2 times the miss.
240
Moving w by 1 moves each prediction by its miles, m.
241
Moving b by 1 moves it by 1.
242
Multiply those together and then average over the 36 rides.
243
That's the chain rule, the one bit of calculus here.
244
You don't need to master it because soon Autograd does it for us.
245
Let's work out both derivatives for one ride by hand.
246
Take Monday's noon ride.
247
For W, its term is 2 times minus 10 point.
248
O, 2 times 1 .38.
249
That's minus 27 .66.
250
For B, there are no miles.
251
It's just 2 times minus 10 .02, which is minus 20 .04.
252
Now average those over all 36 rides.
253
That gives DL by DW equals minus 126 .21, and DL by DB equals minus 27 .56.
254
Pause here if you want to check the arithmetic.
255
Which way should we move?
256
Look at this map of the loss.
257
W runs across and B runs up.
258
Each curve is one loss level, and the star is the bottom.
259
We can only draw this map because there are two numbers.
260
A big model has millions, so we follow the gradient instead.
261
At our start, both derivatives are negative, so the loss goes down if both numbers go up.
262
How far do we step?
263
The naive tries to move W by its whole derivative.
264
Watch what happens.
265
It jumps past 120, much too far.
266
So we scale the gradient by a small learning rate, here 0 .01.
267
This is important.
268
We step against the gradient.
269
Our new W is 1 minus 0 .01 times minus 126 .21.
270
That's 2 .262.
271
And B becomes 0 minus 0 .01 times minus 27 .56, or about 0 .276.
272
The line swings up toward the points.
273
Here's one whole gradient descent step with every variable on screen, one row per ride.
274
Each row has the miles, the fare, the prediction, the miss, its square, and the miss times the miles.
275
The squares average to the loss.
276
Average the misses times the miles and double it, and that's dL by dw.
277
Average the misses alone and double it, and that's dL by db.
278
And the last line moves both.
279
Feel free to pause and check any row.
280
What happens if we repeat that step 100 times, still with a learning rate of 0 .01?
281
W shoots up early.
282
It's overshooting to make up for the missing B.
283
But B crawls.
284
Its derivative is much smaller because moving B by 1 changes each miss by just 1.
285
Moving W by 1 changes it by the ride's miles.
286
After 100 steps, B is only at 3 .71.
287
That's still well short of where it ends up.
288
It does get there.
289
It just needs more steps and later will meet an optimizer that fixes this crawl.
290
So let gradient descent run a thousand times.
291
The line settles on the points and the loss falls from 262 .02 to 10 .15.
292
W is 3 .76 and B is 5 .66.
293
That's about 376 a mile and 566 to start.
294
The meters card says 350 a mile so that's close.
295
The start is higher and part of the reason is on that card too.
296
In slow traffic, the meter charges by the minute, so slow rides cost more than their miles alone.
297
The main point?
298
To train a model, we predict, measure the loss, compute the gradient, step, and repeat.
299
Pop quiz.
300
A ride's prediction is too high, so its miss is positive.
301
Which way does that ride push W?
302
Is it a up, B, down, C, not at all?
303
Or D, only B moves?
304
Pause the video here and pick one.
305
The answer is B.
306
We step against the gradient, so too high means come down.
307
That ride's term in DL by DW is 2 times the miss times the miles, a positive number, so W goes down.
308
Choice A is what happens at our lazy start, where every ride is too low, so their terms are negative, and W goes up.
309
Choice C would need a miss of 0.
310
Choice D is wrong, because that term, two times the miss times the miles, isn't zero.
311
So W moves two.
312
But who works out the gradient for a model with a million weights?
313
By hand, we worked out the gradient and stepped against it.
314
A thousand times.
315
Now the sixth stop?
316
Autograd.
317
PyTorch finds the gradients for us.
318
How does Autograd find the gradients?
319
Look at the code.
320
We create W and B with requires grad equals true, so PyTorch knows we'll want their gradients.
321
From then on, it keeps recording everything we compute from them.
322
Each operation becomes a node in a graph.
323
The graph is a record of how the loss was made.
324
Here it's a multiply, an add, a subtract, a power for the square, and a mean.
325
PyTorch names each node for the step it will run on the way back, like mole backward.
326
And each result's gradfn, its gradient function, remembers the node that made it.
327
Then loss .backward walks that graph backward, from the loss to W and B.
328
It fills W .grad with DL by DW minus 126 .21, and B .grad with DL by DB minus 27 .56.
329
That's exactly what we worked out by hand.
330
On the way back, each node multiplies its own local derivative into the gradient.
331
Take Monday's noon ride again, and leave out the means 1 over 36 for now.
332
Its factors multiply to minus 27 .66 for W and minus 20 .04 for B.
333
That's what we got by hand.
334
Every ride has its own product and the mean averages them.
335
It's the chain rule done for you.
336
In our back propagation video, we take it apart step by step.
337
In a training loop, backward runs every step.
338
So what happens if we compute the loss and call backward again?
339
Does the new gradient replace the old one or add to it?
340
Vali .grad doesn't stay at minus 126 .21.
341
It doubles to minus 252 .42.
342
This is important.
343
Backward adds into .grad instead of replacing it.
344
PyTorch's documentation says it plainly.
345
Backward accumulates gradients, so you might need to zero them.
346
That's why every training step clears .grad once the update is done.
347
Then the next step doesn't add on top of old gradients.
348
Our code clears them by setting both to none.
349
Now, let's run one fresh backward and use its gradients in our loop.
350
There's one rule.
351
The update isn't part of the model's math, so it shouldn't be recorded.
352
Watch what happens if we update anyway.
353
Changing the recorded tensor in place means overwriting its own numbers, and that raises an error.
354
So we do the update inside Torch .nograd.
355
Let's put it together.
356
Here's our loop with Autograd.
357
Compute the loss, call backward, step every weight inside NoGrad, and clear the gradients for the next step.
358
And its printout matches our loop by hand.
359
Autograd records every operation, then walks the record backward to fill DotGrad.
360
Our loop now gets its gradients from Autograd.
361
And the last stop?
362
The Professional loop our training, organized with torch .optim and nn .module, where nn stands for neural network.
363
Our loop still updates and clears each weight by hand.
364
In PyTorch, that's the job of an optimizer.
365
It's an object that holds the weights and steps them for us.
366
The simplest is torch .optim .sgd plane gradient descent.
367
So its numbers match ours, crawl and all.
368
Faster optimizers come later.
369
Look at the code.
370
We give it the weights and the learning rate.
371
SGD stands for Stochastic Gradient Descent, named for the random batches we'll meet at the end.
372
Opt .step updates every weight, Opt .0Grad clears every dot grad, and the loop prints those four rows again.
373
But real models can have millions of weights.
374
So how do we hand them all to the optimizer?
375
Look at the code.
376
We subclass nn .module, which is PyTorch's base class for models.
377
Ours is called fair model.
378
We store each weight on self as an nn .parameter.
379
That's a tensor with requires grad already on.
380
Then model .parameters finds them all for you.
381
How does the model make a prediction?
382
Look at forward.
383
It says how an input becomes an output, it, and calling model of m runs it.
384
All 36 miles go in and 36 predictions come out.
385
That's what nn .module is, a container that finds every weight for you, even inside layers nested in it.
386
What's a layer?
387
It's a building block with its own weights, like nn .linear, which will open up soon.
388
A bigger model, two layers, nest two of them, and nn .module still finds all 13 parameters.
389
Every model also gets the tools on this card.
390
It can list its weights by name, count them, and save them.
391
Our mean squared error is built in too.
392
NN .MSELoss makes a loss function.
393
You call it on the prediction and the target, which is the real fare.
394
It gives 262 .02 again, and it's the one our training loop will use.
395
Here are the five lines from the start, and now you know everyone.
396
Predict.
397
Measure the loss with nn .mse loss.
398
Compute the gradients with backward, update the weights with step, and clear the gradients with zero grade.
399
Pop quiz!
400
What goes wrong if you delete opti .0 grad from the five lines?
401
Is it A.
402
Nothing changes?
403
B. The gradients pile up, so the steps grow and overshoot?
404
C. Backward raises an error? Or D.
405
The model forgets its weights?
406
Pause the video here and pick one.
407
The answer is B.
408
Backward adds into .grad.
409
Without 0 grad, step 2's .grad holds step 1's gradient plus its own, and the old ones never get cleared.
410
So old gradients keep pushing, and W shoots past the bottom, to over 8 within 5 steps.
411
Choice A would only be true if backward replaced .grad instead of adding to it.
412
Choice C is wrong because PyTorch is happy to keep adding.
413
Choice D is wrong because zero grad clears dot grad, never the weights.
414
Side by side with our loop by hand, you can see every job line up.
415
The five lines just hand the gradients and the step to PyTorch.
416
Starting from w equals 1 and b equals 0, both land on 3 .76 and 5 .66.
417
Our model is trained, so what's next?
418
Let's use it, save it, and then train it on batches.
419
To use it, we turn recording off with Torch .noGrad since we won't call backward.
420
Its exact weights are 3 .7556 a mile and 5 .658 to start.
421
With those it says a one mile ride costs about $9 .41.
422
Five miles cost $24 .44 and 10 miles cost $43 .21.
423
So before you get in, you know roughly what you'll pay.
424
How do we save the trained model?
425
State Dyke holds the trained weights by name.
426
Torch .save writes it to a file.
427
Then a fresh fair model which starts at 1 and 0 loads them back.
428
Before batches there's one more piece.
429
PyTorch has our model built in.
430
Look at the code.
431
NN .Linear is a weight times the input, plus a bias.
432
That's one number in and one number out.
433
Let's hand it our row of 36 numbers.
434
Watch what happens.
435
It fails.
436
It reads the last axis as one rides inputs.
437
So our row of 36 looks like one ride with 36 inputs.
438
It wants one input per ride.
439
How do we fix that shape error?
440
We add a new last axis with M of colon, none.
441
That turns the row into a column of 36 examples, each one number -wide.
442
Inside NN .linear is a matrix multiply.
443
The 36 by 1 column times the 1 by 1 weight, plus the bias, gives 36 predictions.
444
Our five lines train it from small random starting numbers to 3 .76 and 5 .66 again.
445
With those, Monday's noon row is 1 .38 times 3 .7556 plus 5 .658, which is $10 .84.
446
What if a dataset is too big to use all at once?
447
A data loader deals it out in shuffled batches.
448
Look at the code.
449
A tensor dataset joins each ride's miles to its fare, like our stack from earlier.
450
Then the loader deals the 36 pairs out, 12 at a time.
451
Our five lines run once per batch, on its 12 miles X and 12 fares Y.
452
One trip through all three batches is called an epoch, so 300 epochs make 900 steps, and they land near 3 .75 and 5 .69.
453
Those random batches are the stochastic and stochastic gradient descent.
454
Predict measure the loss backward step clear.
455
That's the five -line loop.
456
What else can we choose?
457
Three parts of the loop have a menu.
458
The optimizer, the model, and the loss.
459
We'll go through them fast.
460
First, the optimizer, which is where we fix the crawl.
461
At our learning rate, plain SGD needs 551 steps to settle because B crawls.
462
Momentum keeps some speed from recent steps and lets older ones fade, so B's small pushes add up and build speed,
463
and Momentum needs just 100 steps.
464
Atom gives each weight its own step size, so even B's small derivative gets a full -size step, and Atom needs just 88.
465
Here are 8 on our loss, each at a learning rate tuned for it.
466
Each badge counts its steps to the star, so feel free to pause here.
467
Switching is one line and the loop doesn't change.
468
Next we can change the model itself.
469
A linear layer only makes straight lines.
470
But some data is curved like how fast taxis move at each time of day.
471
Quick at night, slow at midday.
472
Let's put an activation, a small function applied to every number between two layers.
473
Now the line can bend.
474
Without one, two linear layers in a row still make a straight line.
475
Relu zeros the negatives, and Gelu is a smooth Relu.
476
Sigmoid squeezes every number between 0 and 1, and Tan does it between minus 1 and 1.
477
And Softmax turns a row into probabilities.
478
Let's try an activation on real data.
479
Take the average taxi speed in each 2 -hour slot from the trip records again.
480
Then give a model the time of day and ask it to predict all three days' speeds.
481
Straight lines miss by up to 7 .1 miles an hour.
482
With one ReLU in the middle, the worst miss drops to 1 .62.
483
Last, the loss is a choice too.
484
Mean squared error is for numbers like ours.
485
L1 is for when a big miss shouldn't count extra and Huber mixes the two.
486
For picking a category, there's cross entropy, or binary cross entropy for yes or no. Let's try them on two rides.
487
Our trained line misses Monday noon by minus 0 .56, and Monday's 10 o 'clock ride by minus 13 .23.
488
Mean squared error squares them.
489
The big miss counts about 175, the small one about a third, average 87 .67.
490
L1 takes their plane sizes, average 6 .90 Huber takes half the small miss squared
491
And the big one's plane size, less a half, so it's 6 .44 For numbers, swapping one in is a single line
492
For categories, cross -entropy grades made up scores for
493
which day a ride was on And binary cross -entropy grades a yes or no
494
Is it over $30?
495
And that's the whole loop Tensors hold the numbers
496
The law says how wrong we are Autograd finds the gradients.
497
Gradient descent moves the weights, and the professional loop does it in five lines.
498
Now PyTorch clicks.
Download App
Everything you need to speak fluently
AI PronunciationScore every sentence
IPA PracticeMaster every sound
VocabularyBuild your word bank
Vocab GameLearn while playing
Same Channel
FOR YOU
Recommend Video
C12
182 sentences
We Let $200 GPT-Pro Think for 7 Days on Unsolved Math
Daily ConversationUniversity StudentsWorking Adults
C11
128 sentences
Vectors and 2D Motion: Crash Course Physics #4
Daily ConversationHigh School StudentsUniversity Students
C11
54 sentences
Traceroute: You're Probably Reading the Results Wrong
Daily ConversationUniversity StudentsWorking Adults
C12
134 sentences
AI harness crash course
Daily ConversationUniversity Students
C11
68 sentences
GitHub Copilot now controls your command line...
Daily ConversationUniversity StudentsWorking Adults
C11
108 sentences
How I Learned Procedural Generation
Daily ConversationUniversity StudentsWorking Adults
C11
433 sentences
Give Me 24 minutes, I'll Give You 1,000+ hours of Claude Knowledge
Daily ConversationUniversity StudentsWorking Adults
C11
321 sentences
Mixture of Experts Explained Visually: How Trillion-Parameter Models Actually Work
Daily ConversationUniversity Students
C11
747 sentences
MIT Introduction to Deep Learning | 6.S191
Daily ConversationUniversity Students
C11
125 sentences
Introduction to statistical prediction models
Daily ConversationUniversity Students
C12
60 sentences
Mind-bending new programming language for GPUs just dropped...
Daily ConversationUniversity Students
C12
186 sentences
Learn Java in 14 Minutes (seriously)
Daily ConversationUniversity Students
Vocabulary and speaking notes for this lesson
This speaking lesson is built on the video “Give me 31 minutes, I will make PyTorch click forever”. The speaker keeps coming back to these words: ride, gradient, fare, step, mile. This video has 498 sentences and 4730 words to shadow. The speech runs for 31:50. The speaker talks at a steady 149 words per minute, a comfortable pace for shadowing. Only 77% of the words are among the 3,000 most common in English, so the vocabulary is demanding.
Key vocabulary in this video
15 words from the video worth learning, with pronunciation and meaning:
| Word | Pronunciation | Meaning |
|---|---|---|
| fare noun | /ˈfɛə/ | Money paid for a transport ticket. |
| minus | /ˈmaɪ.nəs/ | Made less or reduced by (followed by an expression of number or quantity). |
| slot noun | /slɑt/ | A narrow depression, perforation, or aperture; especially, one for the reception of a piece fitting or sliding into it. |
| backward adjective | /ˈbækwɚd/ | Situated toward or at the rear of something. |
| loop noun | /luːp/ | A length of thread, line or rope that is doubled over to make an opening. |
| torch noun | /ˈtɔːtʃ/ | A stick of wood or plant fibres twisted together, with one end soaked in a flammable substance such as resin or tallow and set on fire, which is held in the… |
| prediction noun | /pɹɪˈdɪkʃən/ | A statement of what will happen in the future. |
| grid noun | /ɡɹɪd/ | A rectangular array of squares or rectangles of equal size, such as in a crossword puzzle. |
| axis noun | /ˈæksɪs/ | An imaginary line around which an object spins (an axis of rotation) or is symmetrically arranged (an axis of symmetry). |
| predict verb | /pɹɪˈdɪkt/ | To make a prediction: to forecast, foretell, or estimate a future event on the basis of knowledge and reasoning; to prophesy a future event on the basis of… |
| pause noun | /pɔːz/ | A temporary stop or rest; an intermission of action; interruption; suspension; cessation. |
| derivative noun | /dɪˈɹɪvətɪv/ | Something derived. |
| noon noun | /nun/ | The time of day when the Sun seems to reach its highest point in the sky; solar noon. |
| aggregate noun | /ˈæɡ.ɹɪ.ɡət/ | A mass, assemblage, or sum of particulars; something consisting of elements but considered as a whole. |
| descent noun | /dəˈsɛnt/ | An instance of descending; act of coming down. |
Phrasal verbs you will hear
| Word | Pronunciation | Meaning |
|---|---|---|
| work out verb | /ˌwɝk ˈaʊt/ | To calculate. |
| run down verb | To hit someone with a car or other vehicle and injure or kill them. | |
| add up verb | To take the sum of; to total. | |
| go down verb | To descend; to move from a higher place to a lower one. | |
| go through verb | To travel from one end of something to the other. | |
| go up verb | To move upwards. | |
| leave out verb | To omit, to not include, to neglect to mention. | |
| run across verb | To cross by running. |
Sentences worth repeating
Short, complete lines from the video that you can reuse in everyday conversation:
- How do we hold these fares?
- What can we do with our table of fares?
- How do we get one day's rides?
- Which way should we move?
- Our model is trained, so what's next?
Grammar in this video
The structures the speaker uses most, with the exact words from the video:
| Structure | In the video |
|---|---|
| Passive voice be + past participle — the focus is on what happens, not who does it | is called · is left · were capped |
| Relative clauses who / which + clause — extra information about a person or thing | tensor, which is · module, which is · linear, which will |
Pronunciation to watch
The speaker uses 31 contractions and reduced forms, such as we'll, doesn't, don't. Say them the short way, as you hear them.
- The “th” sounds: mathematics /mæθ(.ə)ˈmæt.ɪks/, smooth /smuːð/, sixth /sɪksθ/, gather /ˈɡæðɚ/, arithmetic /æɹɪθˈmɛtɪk/
- The “sh” and “zh” sounds: prediction /pɹɪˈdɪkʃən/, activation /ˌæktəˈveɪʃən/, dimension /daɪˈmɛn.ʃən/, documentation /ˌdɑkjəmɛnˈteɪʃən/, overshoot /ˌoʊ.vɚˈʃut/
- Long words — get the stress right: derivative /dɪˈɹɪvətɪv/, parameter /pəˈɹæm.ə.tɚ/, mathematics /mæθ(.ə)ˈmæt.ɪks/, activation /ˌæktəˈveɪʃən/, ordinary /ˈɔɹdɪnɛɹi/
How to practise with this video
- Listen to the whole video once without speaking and note the words you do not know.
- Shadow it sentence by sentence at normal speed, repeating each one until your rhythm matches the speaker.
- Record yourself and compare with the original, paying attention to words like fare, minus, slot.
What is the Shadowing Technique?
Shadowing is a science-backed language learning technique originally developed for professional interpreter training and popularized by polyglot Dr. Alexander Arguelles. The method is simple but powerful: you listen to native English audio and immediately repeat it out loud — like a shadow following the speaker with just a 1–2 second delay. Unlike passive listening or grammar drills, shadowing forces your brain and mouth muscles to simultaneously process and reproduce real speech patterns. Research shows it significantly improves pronunciation accuracy, intonation, rhythm, connected speech, listening comprehension, and speaking fluency — making it one of the most effective methods for IELTS Speaking preparation and real-world English communication.