Pratique du Shadowing: Sergey Levine (Top Robotics Researcher): The Best Data For Robotics - Apprendre l'anglais à l'oral avec la vidéo

Choisissez un mode d'entraînement

Chaque mode ouvre une salle d'entraînement en plein écran.

121 phrases
Si les phrases sont trop courtes ou trop longues, cliquez sur Edit pour les ajuster.
1
And I was reading there's different types of data, there's simulated data, you could collect physical interactive data,
2
and I wanted to hear your take on what's the best data to get, what's the worst, and what are the pros and cons?
3
This is, by the way, a question that is, I guess, quite, there's a lot of discussion in the robotics community about this question,
4
and some people have very opposite opinions on it.
5
My own take on this is
6
that a lot of different data sources are easier for the model to internalize
7
if it can ground them in a thorough physical understanding of the world.
8
So let me try to explain what I mean with like a few examples.
9
If you want to learn to fly an airplane, you will probably use a simulator, at least during part of your training.
10
But the simulator makes a lot of sense to you because when you start using the simulator,
11
you have a lot of world knowledge that you can use to ground what's going on.
12
You know that when you are using the flight simulator to learn how to fly the airplane, you're not just playing a video game,
13
you're trying to acquire knowledge that you will then use with a real airplane.
14
And you understand that there's sort of an abstraction there.
15
Same thing if you're playing a really cartoony Atari game or something.
16
You know that all the symbols on the screen,
17
you can sort of connect them to things that you've experienced in your life and you can make an analogy there.
18
So a lot of that, like even though it kind of seems like these simulated environments reflect aspects of the real world,
19
to us they make a lot of sense
20
because we kind of bring to bear a lot of our own prior experience
21
and we fill in the blanks that the simulation has.
22
um and also if you
23
if you want to use uh data uh yourself as a person of somebody else doing something
24
if you you watch someone uh let's say cooking a meal right um even though you don't experience every movement
25
they're experiencing you have a lot of that knowledge
26
that you bring to bear
27
and you're like okay i see they're picking up the salt shaker like i've put salt on things before
28
so i kind of roughly know what's going on there
29
and i can file
30
it away at this level of abstraction of like add salt
31
without having to like figure out all their muscle movements
32
so my point with this is that once you have
33
that understanding of how you do things physically with your own body
34
and how you experience the physical world now all these other sources of knowledge can be connected up to it
35
because that foundation you get from your experience helps you ground everything.
36
So where I'm going with this is that
37
if we have a robotic foundation model
38
that is trained on lots of real embodied data that provides
39
that grounding it might actually be much better able to absorb other sources of knowledge
40
and this is actually like a little bit upside down relative to how some people think about it
41
because it's very tempting looking at the success of like internet data for LLMs to say, well, maybe we should do the opposite.
42
Maybe we should like start with like YouTube videos and then put robot data on top of that.
43
But I think it's actually the other way around.
44
And I actually, I even have like a little bit of evidence for this.
45
So my colleague, Suraj Nair, together with Simar from Georgia Tech.
46
They had a project together a while back where they took our robot foundation model and they added human video data,
47
but they didn't start with human video data.
48
They actually started with a model train on robot data and then added video data on top of it.
49
And what they did is they looked at the representations inside the model.
50
Basically, how does the model represent human experience versus robot experience?
51
And they found that if you use a small model with a small amount of robot data,
52
predictably the human experience and the robot experience are fully separated, meaning that the feature representations are different.
53
But if you train on lots of robot data from lots of different robots,
54
then the features are grouped much more by what task is being done rather than by whether it's a human
55
or a robot.
56
And like when we looked at the feature plots,
57
it was just like mind boggling because literally when you crank up the amount of robot data to 100%, they just line up perfectly.
58
Like you see the, you know, you do this T -SNE embedding, you see the shapes of the embeddings,
59
and it's just all task identity and like minimal sensitivity to embodiment.
60
And to me, that's kind of mind -blowing because the base model wasn't trained on any human data at all.
61
But once you start adding human data, it represents it exactly the same way.
62
And I think that's really exciting.
63
And I think that to me is one of the strongest indicators that if you have that good foundation of robot experience,
64
you can put everything else on top of it.
65
It's actually better at absorbing that.
66
Is it important that that base model has data that was collected using that specific set of motors, specific set of joints?
67
So far, we've obviously put a lot of effort into cross embodiment models that can handle many different robot types.
68
But generally, you do need data of the robot you're going to be deploying on to get good performance.
69
So kind of the metric of generalization there is not can you zero shot a new robot,
70
but it's mostly can you get away with less experience from the new robot and transfer skills from other robots.
71
So that's the current state of things.
72
Now, there is a little bit of a kind of surprisingly positive read on that, which is...
73
even though you need data from these robots, the amount of special stuff that the model is doing is kind of minimal.
74
Like when we started doing all this,
75
I had like a big long list of all the cool research I wanted to do to better accommodate different morphologies.
76
Like, can you like factorize the model's representation in some way so that there's like a,
77
you know, a six degree freedom arm head and a seven degree head and a gripper head, etc. We didn't do any of that.
78
Like the model just outputs like a big vector of numbers.
79
If the robot has less degrees of freedom, Then the number it outputs, it just zero -pads it.
80
There's just like nothing fancy, and that's it.
81
And then it just train on all the robots
82
and outputs the correct actions based on what it's seen through the camera, essentially.
83
But now, to your point about whether you can handle new robots.
84
So, so far, the thing that we focused on, and I think this is showing some promise, is to be able to transfer skills between robots.
85
And this is actually where the particular choices in how the model works seem to matter.
86
For example, you can have a model that does some intermediate thinking.
87
And that thinking can be done in different modalities.
88
So you can think in text.
89
And thinking in text is really good for transferring high -level behavioral structure.
90
So that's basically how you understand that, hey, if I want to,
91
like, clean the kitchen and put away like the silhouette where first open the drawer.
92
Like that's kind of a semantic inference.
93
And you can transfer that very well because obviously that's like largely agnostic to any embodiment or anything like that.
94
But even lower level things can be transferred if you use the right representation.
95
So one experiment we did is we had a thinking stage
96
that is expressed in images where you basically like dream up an image of the next milestone in the task.
97
And with that, we could actually get a robot,
98
the UR5 robot to fold a t -shirt even though we didn't have any t -shirt folding data on the UR5.
99
Because while getting the arm motions correct is very hard
100
because the robot basically requires very different joint angles to do the task,
101
cooking up an image of what it looks like for it to fold a shirt is not that hard because,
102
like, you've seen the robot arm in all sorts of different poses, you've seen the shirt in all different stages of being folded and unfolded,
103
you know roughly where it should hold it, so getting a good generative model to cook up that image is pretty straightforward.
104
And once you have the image, then from that backing out the correct actions is easy too
105
because you can just look at the synthesized arm angle and just like back out what the angle should be.
106
So it's not changing the problem, but it's just introducing this intermediate step that makes it easier to solve.
107
Just like if you're solving a math problem, if you figure out like the right intermediate step,
108
kind of the answer is obvious from that intermediate step.
109
And I think that's really exciting because now that shows that this level of generalization across robots, and I'm sure other generalization too,
110
can be facilitated with thinking, just like in LLMs, but with a twist that you have to think in the right modality.
111
interesting so it it outputs a I guess
112
that image is what its video sensor is seeing
113
and it's like the next step yeah
114
you can almost think of it like image editing
115
you can do the same thing with video you can do it with video prediction
116
but the key is to like imagine what it would look like to progress on this task
117
that seems like a very human thing to do yeah yeah some things you can you you plan semantically
118
and some things you plan spatially like
119
if you're doing rock climbing
120
you're probably not thinking like hey left arm to rock 37 centimeters to the left
121
you're probably more like imagining your arm reaching for the rock

Télécharger l'application

Tout pour parler couramment

Prononciation IAChaque phrase notée
Phonétique (IPA)Maîtrisez chaque son
VocabulaireEnrichissez votre lexique
Jeu de vocabulaireApprenez en jouant

Même chaîne

FOR YOU

Vidéo recommandée

Vocabulaire et conseils d’expression pour cette leçon

Cette leçon d’expression orale s’appuie sur la vidéo « Sergey Levine (Top Robotics Researcher): The Best Data For Robotics ». Les mots qui reviennent le plus souvent : robot, model, experience, image. Cette vidéo contient 121 phrases et 1737 mots à répéter en shadowing. La partie parlée dure 8:24. Le locuteur parle vite, environ 207 mots par minute : attendez-vous à des liaisons et à des sons réduits. 85 % des mots font partie des 3 000 mots les plus courants en anglais ; le reste mérite d’être vu avant de commencer.

Vocabulaire clé de cette vidéo

15 mots de la vidéo à retenir, avec leur prononciation et leur sens :

MotPrononciationSens
robot nom/ˈɹoʊ.bɑt/robot
output nom/ˈaʊtpʊt/sortie
fold verbe/ˈfəʊld/plier
intermediate adjectif/ɪn.tə(ɹ)ˈmi.di.ət/intermédiaire
simulator nom/ˈsɪmjəˌleɪtɚ/simulateur
angle nom/ˈæŋ.ɡəl/angle
airplane nom/ˈɛɹˌpleɪ̯n/avion, aéroplane
skill nom/skɪl/habileté, compétence
solve verbe/sɒlv/résoudre, régler
connect verbe/kəˈnɛkt/accoupler, connecter
collect verbe/ˈkɑlɪkt/rassembler, recueillir
roughly adverbe/ˈɹʌf.li/rudement
minimal adjectif/ˈmɪn.ə.məl/minimum
absorb verbe/æbˈsɔɹb/absorber
meal nom/ˈmiːl/repas

Les verbes à particule que vous entendrez

MotSens
figure out verbecerner
get away with verbes'en tirer
pick up verbesoulever
put away verberanger
see through verbevoir à travers

La grammaire de cette vidéo

Les structures que le locuteur utilise le plus, avec les mots exacts de la vidéo :

StructureDans la vidéo
Voix passive be + participe passé — l’accent est mis sur ce qui arrive, pas sur qui le faitcan be connected · is trained · are fully separated
Present perfect have/has + participe passé — une action passée qui compte encore maintenantyou've experienced · i've put · we've obviously put

Prononciation à surveiller

Le locuteur utilise 23 contractions et formes réduites, comme you're, didn't, you've. Prononcez-les sous leur forme courte, telles que vous les entendez.

  • Les sons « th »: math /mæθ/, thorough /ˈθʌɹoʊ/, synthesize /ˈsɪnθəsaɪz/
  • Les sons « sh » et « zh »: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, essentially /ɪˈsɛnʃəli/, prediction /pɹɪˈdɪkʃən/, simulation /ˌsɪm.jəˈleɪ.ʃn̩/, abstraction /æbˈstɹæk.ʃn̩/
  • Mots longs — placez bien l’accent: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, intermediate /ɪn.tə(ɹ)ˈmi.di.ət/, simulator /ˈsɪmjəˌleɪtɚ/, essentially /ɪˈsɛnʃəli/, experiment /ɪkˈspɛɹ.ə.mənt/

Les sons difficiles pour les francophones :

  • /tʃ/ et /dʒ/ — à ne pas adoucir en « ch » et « j »: analogy /əˈnæləd͡ʒi/, generative /ˈd͡ʒɛnəɹətɪv/, morphology /mɔɹˈfɑ.lə.d͡ʒi/
  • /r/ anglais — langue recourbée, sans frotter la gorge: robot /ˈɹoʊ.bɑt/, representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, intermediate /ɪn.tə(ɹ)ˈmi.di.ət/, airplane /ˈɛɹˌpleɪ̯n/, roughly /ˈɹʌf.li/

Comment s’entraîner avec cette vidéo

  1. Écoutez la vidéo en entier une fois sans parler et notez les mots que vous ne connaissez pas.
  2. Commencez à la vitesse 0,75×, répétez phrase par phrase, puis revenez à la vitesse normale quand cela devient facile.
  3. Enregistrez-vous et comparez avec l’original, en faisant attention à des mots comme robot, output, fold.

Qu'est-ce que la technique du Shadowing ?

Le Shadowing est une technique d'apprentissage des langues fondée sur la science, développée à l'origine pour la formation des interprètes professionnels. Le principe est simple mais puissant : vous écoutez de l'anglais natif et le répétez immédiatement à voix haute — comme une ombre suivant le locuteur avec un décalage de 1 à 2 secondes. Les recherches montrent une amélioration significative de la précision de la prononciation, de l'intonation, du rythme, des liaisons, de la compréhension orale et de la fluidité.

Technique du shadowing : lire le guide complet étape par étape →