Shadowing Practice: Sergey Levine (Top Robotics Researcher): The Best Data For Robotics - Learn English Speaking with Video

Kies een oefenmodus

Elke modus opent een volledig scherm oefenruimte.

121 zinnen
Als zinnen te kort of te lang zijn, klik op Bewerken om ze aan te passen.
1
And I was reading there's different types of data, there's simulated data, you could collect physical interactive data,
2
and I wanted to hear your take on what's the best data to get, what's the worst, and what are the pros and cons?
3
This is, by the way, a question that is, I guess, quite, there's a lot of discussion in the robotics community about this question,
4
and some people have very opposite opinions on it.
5
My own take on this is
6
that a lot of different data sources are easier for the model to internalize
7
if it can ground them in a thorough physical understanding of the world.
8
So let me try to explain what I mean with like a few examples.
9
If you want to learn to fly an airplane, you will probably use a simulator, at least during part of your training.
10
But the simulator makes a lot of sense to you because when you start using the simulator,
11
you have a lot of world knowledge that you can use to ground what's going on.
12
You know that when you are using the flight simulator to learn how to fly the airplane, you're not just playing a video game,
13
you're trying to acquire knowledge that you will then use with a real airplane.
14
And you understand that there's sort of an abstraction there.
15
Same thing if you're playing a really cartoony Atari game or something.
16
You know that all the symbols on the screen,
17
you can sort of connect them to things that you've experienced in your life and you can make an analogy there.
18
So a lot of that, like even though it kind of seems like these simulated environments reflect aspects of the real world,
19
to us they make a lot of sense
20
because we kind of bring to bear a lot of our own prior experience
21
and we fill in the blanks that the simulation has.
22
um and also if you
23
if you want to use uh data uh yourself as a person of somebody else doing something
24
if you you watch someone uh let's say cooking a meal right um even though you don't experience every movement
25
they're experiencing you have a lot of that knowledge
26
that you bring to bear
27
and you're like okay i see they're picking up the salt shaker like i've put salt on things before
28
so i kind of roughly know what's going on there
29
and i can file
30
it away at this level of abstraction of like add salt
31
without having to like figure out all their muscle movements
32
so my point with this is that once you have
33
that understanding of how you do things physically with your own body
34
and how you experience the physical world now all these other sources of knowledge can be connected up to it
35
because that foundation you get from your experience helps you ground everything.
36
So where I'm going with this is that
37
if we have a robotic foundation model
38
that is trained on lots of real embodied data that provides
39
that grounding it might actually be much better able to absorb other sources of knowledge
40
and this is actually like a little bit upside down relative to how some people think about it
41
because it's very tempting looking at the success of like internet data for LLMs to say, well, maybe we should do the opposite.
42
Maybe we should like start with like YouTube videos and then put robot data on top of that.
43
But I think it's actually the other way around.
44
And I actually, I even have like a little bit of evidence for this.
45
So my colleague, Suraj Nair, together with Simar from Georgia Tech.
46
They had a project together a while back where they took our robot foundation model and they added human video data,
47
but they didn't start with human video data.
48
They actually started with a model train on robot data and then added video data on top of it.
49
And what they did is they looked at the representations inside the model.
50
Basically, how does the model represent human experience versus robot experience?
51
And they found that if you use a small model with a small amount of robot data,
52
predictably the human experience and the robot experience are fully separated, meaning that the feature representations are different.
53
But if you train on lots of robot data from lots of different robots,
54
then the features are grouped much more by what task is being done rather than by whether it's a human
55
or a robot.
56
And like when we looked at the feature plots,
57
it was just like mind boggling because literally when you crank up the amount of robot data to 100%, they just line up perfectly.
58
Like you see the, you know, you do this T -SNE embedding, you see the shapes of the embeddings,
59
and it's just all task identity and like minimal sensitivity to embodiment.
60
And to me, that's kind of mind -blowing because the base model wasn't trained on any human data at all.
61
But once you start adding human data, it represents it exactly the same way.
62
And I think that's really exciting.
63
And I think that to me is one of the strongest indicators that if you have that good foundation of robot experience,
64
you can put everything else on top of it.
65
It's actually better at absorbing that.
66
Is it important that that base model has data that was collected using that specific set of motors, specific set of joints?
67
So far, we've obviously put a lot of effort into cross embodiment models that can handle many different robot types.
68
But generally, you do need data of the robot you're going to be deploying on to get good performance.
69
So kind of the metric of generalization there is not can you zero shot a new robot,
70
but it's mostly can you get away with less experience from the new robot and transfer skills from other robots.
71
So that's the current state of things.
72
Now, there is a little bit of a kind of surprisingly positive read on that, which is...
73
even though you need data from these robots, the amount of special stuff that the model is doing is kind of minimal.
74
Like when we started doing all this,
75
I had like a big long list of all the cool research I wanted to do to better accommodate different morphologies.
76
Like, can you like factorize the model's representation in some way so that there's like a,
77
you know, a six degree freedom arm head and a seven degree head and a gripper head, etc. We didn't do any of that.
78
Like the model just outputs like a big vector of numbers.
79
If the robot has less degrees of freedom, Then the number it outputs, it just zero -pads it.
80
There's just like nothing fancy, and that's it.
81
And then it just train on all the robots
82
and outputs the correct actions based on what it's seen through the camera, essentially.
83
But now, to your point about whether you can handle new robots.
84
So, so far, the thing that we focused on, and I think this is showing some promise, is to be able to transfer skills between robots.
85
And this is actually where the particular choices in how the model works seem to matter.
86
For example, you can have a model that does some intermediate thinking.
87
And that thinking can be done in different modalities.
88
So you can think in text.
89
And thinking in text is really good for transferring high -level behavioral structure.
90
So that's basically how you understand that, hey, if I want to,
91
like, clean the kitchen and put away like the silhouette where first open the drawer.
92
Like that's kind of a semantic inference.
93
And you can transfer that very well because obviously that's like largely agnostic to any embodiment or anything like that.
94
But even lower level things can be transferred if you use the right representation.
95
So one experiment we did is we had a thinking stage
96
that is expressed in images where you basically like dream up an image of the next milestone in the task.
97
And with that, we could actually get a robot,
98
the UR5 robot to fold a t -shirt even though we didn't have any t -shirt folding data on the UR5.
99
Because while getting the arm motions correct is very hard
100
because the robot basically requires very different joint angles to do the task,
101
cooking up an image of what it looks like for it to fold a shirt is not that hard because,
102
like, you've seen the robot arm in all sorts of different poses, you've seen the shirt in all different stages of being folded and unfolded,
103
you know roughly where it should hold it, so getting a good generative model to cook up that image is pretty straightforward.
104
And once you have the image, then from that backing out the correct actions is easy too
105
because you can just look at the synthesized arm angle and just like back out what the angle should be.
106
So it's not changing the problem, but it's just introducing this intermediate step that makes it easier to solve.
107
Just like if you're solving a math problem, if you figure out like the right intermediate step,
108
kind of the answer is obvious from that intermediate step.
109
And I think that's really exciting because now that shows that this level of generalization across robots, and I'm sure other generalization too,
110
can be facilitated with thinking, just like in LLMs, but with a twist that you have to think in the right modality.
111
interesting so it it outputs a I guess
112
that image is what its video sensor is seeing
113
and it's like the next step yeah
114
you can almost think of it like image editing
115
you can do the same thing with video you can do it with video prediction
116
but the key is to like imagine what it would look like to progress on this task
117
that seems like a very human thing to do yeah yeah some things you can you you plan semantically
118
and some things you plan spatially like
119
if you're doing rock climbing
120
you're probably not thinking like hey left arm to rock 37 centimeters to the left
121
you're probably more like imagining your arm reaching for the rock

App downloaden

Alles wat je nodig hebt om vloeiend te spreken

AI UitspraakScoor elke zin
IPA OefeningBeheers elk geluid
WoordenschatBouw je woordenschat op
WoordenspelLeren terwijl je speelt

Zelfde Kanaal

FOR YOU

Aanbevolen Video

Woordenschat en spreektips bij deze les

Deze spreekles is gebaseerd op de video “Sergey Levine (Top Robotics Researcher): The Best Data For Robotics”. Deze woorden komen het vaakst terug: robot, model, experience, image. Deze video bevat 121 zinnen en 1737 woorden om na te spreken. Het gesproken deel duurt 8:24. De spreker praat snel, ongeveer 207 woorden per minuut, dus reken op verbonden en ingeslikte klanken. 85% van de woorden hoort bij de 3.000 meest gebruikte Engelse woorden; de rest kun je beter vooraf bekijken.

Belangrijke woorden in deze video

15 woorden uit de video die het leren waard zijn, met uitspraak en betekenis:

WoordUitspraakBetekenis
robot zelfstandig naamwoord/ˈɹoʊ.bɑt/robot
output zelfstandig naamwoord/ˈaʊtpʊt/productie
representation zelfstandig naamwoord/ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/weergave, representatie
fold werkwoord/ˈfəʊld/vouwen
intermediate bijvoeglijk naamwoord/ɪn.tə(ɹ)ˈmi.di.ət/middelmatig
angle zelfstandig naamwoord/ˈæŋ.ɡəl/hoek
airplane zelfstandig naamwoord/ˈɛɹˌpleɪ̯n/vliegtuig
skill zelfstandig naamwoord/skɪl/bekwaamheid, vaardigheid
solve werkwoord/sɒlv/oplossen
connect werkwoord/kəˈnɛkt/verbinden
collect werkwoord/ˈkɑlɪkt/verzamelen, opstapelen
roughly bijwoord/ˈɹʌf.li/grofweg
minimal bijvoeglijk naamwoord/ˈmɪn.ə.məl/minimaal
absorb werkwoord/æbˈsɔɹb/absorberen, (in zich) opnemen
meal zelfstandig naamwoord/ˈmiːl/maaltijd

Phrasal verbs die je zult horen

WoordBetekenis
back out werkwoordzich terugtrekken
figure out werkwoorduitvogelen, er achter komen
pick up werkwoordoppakken, optillen
see through werkwoorddoorheen zien

Grammatica in deze video

De structuren die de spreker het meest gebruikt, met de exacte woorden uit de video:

StructuurIn de video
Lijdende vorm be + voltooid deelwoord — het gaat om wat er gebeurt, niet om wie het doetcan be connected · is trained · are fully separated
Present perfect have/has + voltooid deelwoord — iets uit het verleden dat nu nog teltyou've experienced · i've put · we've obviously put

Uitspraak om op te letten

De spreker gebruikt 23 samentrekkingen en verkorte vormen, zoals you're, didn't, you've. Spreek ze kort uit, zoals je ze hoort.

  • De “th”-klanken: math /mæθ/, thorough /ˈθʌɹoʊ/, synthesize /ˈsɪnθəsaɪz/
  • De klanken “sh” en “zh”: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, essentially /ɪˈsɛnʃəli/, prediction /pɹɪˈdɪkʃən/, simulation /ˌsɪm.jəˈleɪ.ʃn̩/, abstraction /æbˈstɹæk.ʃn̩/
  • Lange woorden — let op de klemtoon: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, intermediate /ɪn.tə(ɹ)ˈmi.di.ət/, simulator /ˈsɪmjəˌleɪtɚ/, essentially /ɪˈsɛnʃəli/, experiment /ɪkˈspɛɹ.ə.mənt/

Klanken die Nederlandstaligen lastig vinden:

  • Stemhebbende eindmedeklinker — /b/, /d/, /g/, /z/, /v/ niet verscherpen: fold /ˈfəʊld/, solve /sɒlv/, absorb /æbˈsɔɹb/, relative /ˈɹɛl.ə.tɪv/, pose /poʊz/
  • /æ/ — opener dan de Nederlandse “e”: angle /ˈæŋ.ɡəl/, absorb /æbˈsɔɹb/, math /mæθ/, aspect /ˈæspɛkt/, fancy /ˈfæn.si/
  • /g/ — een harde plofklank, geen Nederlandse “g”: angle /ˈæŋ.ɡəl/, colleague /ˈkɒliːɡ/, agnostic /aɡˈnɒstɪk/, boggle /ˈbɒɡ.əl/, gripper /ˈɡɹɪ.pəɹ/

Zo oefen je met deze video

  1. Luister de hele video één keer zonder te spreken en noteer de woorden die je niet kent.
  2. Begin op 0,75× snelheid, spreek zin voor zin na en ga terug naar normale snelheid zodra het makkelijk gaat.
  3. Neem jezelf op en vergelijk met het origineel; let daarbij op woorden als robot, output, representation.

Wat is de Shadowing-techniek?

Shadowing is een wetenschappelijk onderbouwde taalleermethode die oorspronkelijk is ontwikkeld voor professionele tolkentraining en gepopulariseerd door polyglot Dr. Alexander Arguelles. De methode is eenvoudig maar krachtig: je luistert naar native Engelse audio en herhaalt het onmiddellijk hardop — als een schaduw die de spreker volgt met slechts 1–2 seconden vertraging. In tegenstelling tot passief luisteren of grammaticadrills, dwingt shadowing je hersenen en mondspieren om echte spraakpatronen tegelijkertijd te verwerken en te reproduceren. Onderzoek toont aan dat het de uitspraaknauwkeurigheid, intonatie, ritme, verbonden spraak, luisterbegrip en spreekvaardigheid aanzienlijk verbetert — waardoor het een van de meest effectieve methoden is voor IELTS Speaking-voorbereiding en echte Engelse communicatie.

Shadowing-techniek: lees de volledige stap-voor-stap-gids →