Shadowing Practice: Sergey Levine (Top Robotics Researcher): The Best Data For Robotics - Learn English Speaking with Video

בחר מצב תרגול

כל מצב פותח חדר תרגול במסך מלא.

121 משפטים
אם המשפטים קצרים או ארוכים מדי, לחץ על ערוך כדי להתאים אותם.
1
And I was reading there's different types of data, there's simulated data, you could collect physical interactive data,
2
and I wanted to hear your take on what's the best data to get, what's the worst, and what are the pros and cons?
3
This is, by the way, a question that is, I guess, quite, there's a lot of discussion in the robotics community about this question,
4
and some people have very opposite opinions on it.
5
My own take on this is
6
that a lot of different data sources are easier for the model to internalize
7
if it can ground them in a thorough physical understanding of the world.
8
So let me try to explain what I mean with like a few examples.
9
If you want to learn to fly an airplane, you will probably use a simulator, at least during part of your training.
10
But the simulator makes a lot of sense to you because when you start using the simulator,
11
you have a lot of world knowledge that you can use to ground what's going on.
12
You know that when you are using the flight simulator to learn how to fly the airplane, you're not just playing a video game,
13
you're trying to acquire knowledge that you will then use with a real airplane.
14
And you understand that there's sort of an abstraction there.
15
Same thing if you're playing a really cartoony Atari game or something.
16
You know that all the symbols on the screen,
17
you can sort of connect them to things that you've experienced in your life and you can make an analogy there.
18
So a lot of that, like even though it kind of seems like these simulated environments reflect aspects of the real world,
19
to us they make a lot of sense
20
because we kind of bring to bear a lot of our own prior experience
21
and we fill in the blanks that the simulation has.
22
um and also if you
23
if you want to use uh data uh yourself as a person of somebody else doing something
24
if you you watch someone uh let's say cooking a meal right um even though you don't experience every movement
25
they're experiencing you have a lot of that knowledge
26
that you bring to bear
27
and you're like okay i see they're picking up the salt shaker like i've put salt on things before
28
so i kind of roughly know what's going on there
29
and i can file
30
it away at this level of abstraction of like add salt
31
without having to like figure out all their muscle movements
32
so my point with this is that once you have
33
that understanding of how you do things physically with your own body
34
and how you experience the physical world now all these other sources of knowledge can be connected up to it
35
because that foundation you get from your experience helps you ground everything.
36
So where I'm going with this is that
37
if we have a robotic foundation model
38
that is trained on lots of real embodied data that provides
39
that grounding it might actually be much better able to absorb other sources of knowledge
40
and this is actually like a little bit upside down relative to how some people think about it
41
because it's very tempting looking at the success of like internet data for LLMs to say, well, maybe we should do the opposite.
42
Maybe we should like start with like YouTube videos and then put robot data on top of that.
43
But I think it's actually the other way around.
44
And I actually, I even have like a little bit of evidence for this.
45
So my colleague, Suraj Nair, together with Simar from Georgia Tech.
46
They had a project together a while back where they took our robot foundation model and they added human video data,
47
but they didn't start with human video data.
48
They actually started with a model train on robot data and then added video data on top of it.
49
And what they did is they looked at the representations inside the model.
50
Basically, how does the model represent human experience versus robot experience?
51
And they found that if you use a small model with a small amount of robot data,
52
predictably the human experience and the robot experience are fully separated, meaning that the feature representations are different.
53
But if you train on lots of robot data from lots of different robots,
54
then the features are grouped much more by what task is being done rather than by whether it's a human
55
or a robot.
56
And like when we looked at the feature plots,
57
it was just like mind boggling because literally when you crank up the amount of robot data to 100%, they just line up perfectly.
58
Like you see the, you know, you do this T -SNE embedding, you see the shapes of the embeddings,
59
and it's just all task identity and like minimal sensitivity to embodiment.
60
And to me, that's kind of mind -blowing because the base model wasn't trained on any human data at all.
61
But once you start adding human data, it represents it exactly the same way.
62
And I think that's really exciting.
63
And I think that to me is one of the strongest indicators that if you have that good foundation of robot experience,
64
you can put everything else on top of it.
65
It's actually better at absorbing that.
66
Is it important that that base model has data that was collected using that specific set of motors, specific set of joints?
67
So far, we've obviously put a lot of effort into cross embodiment models that can handle many different robot types.
68
But generally, you do need data of the robot you're going to be deploying on to get good performance.
69
So kind of the metric of generalization there is not can you zero shot a new robot,
70
but it's mostly can you get away with less experience from the new robot and transfer skills from other robots.
71
So that's the current state of things.
72
Now, there is a little bit of a kind of surprisingly positive read on that, which is...
73
even though you need data from these robots, the amount of special stuff that the model is doing is kind of minimal.
74
Like when we started doing all this,
75
I had like a big long list of all the cool research I wanted to do to better accommodate different morphologies.
76
Like, can you like factorize the model's representation in some way so that there's like a,
77
you know, a six degree freedom arm head and a seven degree head and a gripper head, etc. We didn't do any of that.
78
Like the model just outputs like a big vector of numbers.
79
If the robot has less degrees of freedom, Then the number it outputs, it just zero -pads it.
80
There's just like nothing fancy, and that's it.
81
And then it just train on all the robots
82
and outputs the correct actions based on what it's seen through the camera, essentially.
83
But now, to your point about whether you can handle new robots.
84
So, so far, the thing that we focused on, and I think this is showing some promise, is to be able to transfer skills between robots.
85
And this is actually where the particular choices in how the model works seem to matter.
86
For example, you can have a model that does some intermediate thinking.
87
And that thinking can be done in different modalities.
88
So you can think in text.
89
And thinking in text is really good for transferring high -level behavioral structure.
90
So that's basically how you understand that, hey, if I want to,
91
like, clean the kitchen and put away like the silhouette where first open the drawer.
92
Like that's kind of a semantic inference.
93
And you can transfer that very well because obviously that's like largely agnostic to any embodiment or anything like that.
94
But even lower level things can be transferred if you use the right representation.
95
So one experiment we did is we had a thinking stage
96
that is expressed in images where you basically like dream up an image of the next milestone in the task.
97
And with that, we could actually get a robot,
98
the UR5 robot to fold a t -shirt even though we didn't have any t -shirt folding data on the UR5.
99
Because while getting the arm motions correct is very hard
100
because the robot basically requires very different joint angles to do the task,
101
cooking up an image of what it looks like for it to fold a shirt is not that hard because,
102
like, you've seen the robot arm in all sorts of different poses, you've seen the shirt in all different stages of being folded and unfolded,
103
you know roughly where it should hold it, so getting a good generative model to cook up that image is pretty straightforward.
104
And once you have the image, then from that backing out the correct actions is easy too
105
because you can just look at the synthesized arm angle and just like back out what the angle should be.
106
So it's not changing the problem, but it's just introducing this intermediate step that makes it easier to solve.
107
Just like if you're solving a math problem, if you figure out like the right intermediate step,
108
kind of the answer is obvious from that intermediate step.
109
And I think that's really exciting because now that shows that this level of generalization across robots, and I'm sure other generalization too,
110
can be facilitated with thinking, just like in LLMs, but with a twist that you have to think in the right modality.
111
interesting so it it outputs a I guess
112
that image is what its video sensor is seeing
113
and it's like the next step yeah
114
you can almost think of it like image editing
115
you can do the same thing with video you can do it with video prediction
116
but the key is to like imagine what it would look like to progress on this task
117
that seems like a very human thing to do yeah yeah some things you can you you plan semantically
118
and some things you plan spatially like
119
if you're doing rock climbing
120
you're probably not thinking like hey left arm to rock 37 centimeters to the left
121
you're probably more like imagining your arm reaching for the rock

הורד אפליקציה

כל מה שאתה צריך כדי לדבר בשטף

הגיית AIציון לכל משפט
תרגול IPAשלוט בכל צליל
אוצר מיליםבנה את מאגר המילים שלך
משחק אוצר מיליםלמד תוך כדי משחק

אותו ערוץ

FOR YOU

המלץ על סרטון

אוצר מילים והערות הגייה לשיעור הזה

שיעור הדיבור הזה מבוסס על הסרטון “Sergey Levine (Top Robotics Researcher): The Best Data For Robotics”. המילים שחוזרות הכי הרבה: robot, model, experience, image. בסרטון הזה יש 121 משפטים ו-1737 מילים לתרגול Shadowing. הדיבור נמשך 8:24. הדובר מדבר מהר, כ-207 מילים בדקה, אז צפו לצלילים מחוברים ומקוצרים. 85% מהמילים נמצאות בין 3,000 המילים הנפוצות ביותר באנגלית; כדאי ללמוד את השאר לפני שמתחילים.

אוצר מילים מרכזי בסרטון הזה

15 מילים מהסרטון שכדאי ללמוד, עם ההגייה והמשמעות שלהן:

מילההגייהמשמעות
robot שם עצם/ˈɹoʊ.bɑt/רוֹבּוֹט
output שם עצם/ˈaʊtpʊt/פֶּלֶט
angle שם עצם/ˈæŋ.ɡəl/פינה, זווית
airplane שם עצם/ˈɛɹˌpleɪ̯n/מָטוֹס, אווירון
skill שם עצם/skɪl/מְיֻמָּנוּת, כִּשּׁוּרִים
solve פועל/sɒlv/פָּתַר
connect פועל/kəˈnɛkt/חיבר
collect פועל/ˈkɑlɪkt/אסף
absorb פועל/æbˈsɔɹb/ספג
meal שם עצם/ˈmiːl/אֲרוּחָה
relative שם עצם/ˈɹɛl.ə.tɪv/קָרוֹב, קְרוֹבָה
muscle שם עצם/ˈmʌs.əl/שְׁרִיר
aspect שם עצם/ˈæspɛkt/הֶבֵּט
reflect פועל/ɹɪˈflɛkt/החזיר
experiment שם עצם/ɪkˈspɛɹ.ə.mənt/ניסוי, נִסּוּי

Phrasal verbs שתשמעו

מילהמשמעות
figure out פועלהֵבִין, מצא תשובה
pick up פועלהרים, אסף

דקדוק בסרטון הזה

המבנים שהדובר משתמש בהם הכי הרבה, עם המילים המדויקות מהסרטון:

מבנהבסרטון
סביל (Passive voice) be + past participle — הדגש הוא על מה שקורה, לא על מי שעושה את הפעולהcan be connected · is trained · are fully separated
הווה מושלם (Present perfect) have/has + past participle — פעולה שקרתה בעבר ועדיין יש לה חשיבות או השפעה עכשיוyou've experienced · i've put · we've obviously put

הגייה ששווה לשים לב אליה

הדובר משתמש ב-23 קיצורים וצורות מקוצרות, כמו you're, didn't, you've. אמרו אותם בצורה הקצרה, כפי שאתם שומעים אותם.

  • הצלילים של “th”: math /mæθ/, thorough /ˈθʌɹoʊ/, synthesize /ˈsɪnθəsaɪz/
  • הצלילים של “sh” ו-“zh”: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, essentially /ɪˈsɛnʃəli/, prediction /pɹɪˈdɪkʃən/, simulation /ˌsɪm.jəˈleɪ.ʃn̩/, abstraction /æbˈstɹæk.ʃn̩/
  • מילים ארוכות — שימו לב להטעמה: representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, intermediate /ɪn.tə(ɹ)ˈmi.di.ət/, simulator /ˈsɪmjəˌleɪtɚ/, essentially /ɪˈsɛnʃəli/, experiment /ɪkˈspɛɹ.ə.mənt/

צלילים שדוברי עברית מתקשים בהם:

  • /w/ — שפתיים מעוגלות, לא /v/: twist /twɪst/, acquire /əˈkwaɪɚ/, straightforward /ˌstɹeɪtˈfɔː(ɹ)wə(ɹ)d/
  • /æ/ — פתוח יותר מה-“אֶ” העברית, לא כמו “a” ב-“cup”: angle /ˈæŋ.ɡəl/, absorb /æbˈsɔɹb/, math /mæθ/, aspect /ˈæspɛkt/, fancy /ˈfæn.si/
  • /r/ — הלשון לא נוגעת בשום מקום, לא ה-“ר” הגרונית: robot /ˈɹoʊ.bɑt/, representation /ˌɹɛp.ɹə.zɛnˈteɪ.ʃən/, intermediate /ɪn.tə(ɹ)ˈmi.di.ət/, airplane /ˈɛɹˌpleɪ̯n/, roughly /ˈɹʌf.li/

איך לתרגל עם הסרטון הזה

  1. הקשיבו לכל הסרטון פעם אחת בלי לדבר, ורשמו את המילים שאתם לא מכירים.
  2. התחילו במהירות 0.75×, תרגלו Shadowing משפט אחר משפט, וחזרו למהירות רגילה כשזה מרגיש קל.
  3. הקליטו את עצמכם והשוו למקור, תוך תשומת לב למילים כמו robot, output, angle.

מהי טכניקת השאדואינג?

שאדואינג היא טכניקת למידת שפה מבוססת מדע שפותחה במקור להכשרת מתורגמנים מקצועיים ופופולרית על ידי הפוליגלוט ד"ר אלכסנדר ארגואלס. השיטה פשוטה אך עוצמתית: אתה מקשיב לאודיו באנגלית ילידית וחוזר עליו בקול רם מיד — כמו צל העוקב אחרי הדובר עם עיכוב של 1–2 שניות בלבד. בניגוד להאזנה פסיבית או תרגילי דקדוק, שאדואינג מכריח את המוח והשרירים בפה לעבד ולשחזר דפוסי דיבור אמיתיים בו זמנית. מחקרים מראים שזה משפר משמעותית את דיוק ההגייה, האינטונציה, הקצב, הדיבור המחובר, הבנת השמיעה ושטף הדיבור — מה שהופך אותו לאחת השיטות היעילות ביותר להכנה לIELTS Speaking ולתקשורת באנגלית בעולם האמיתי.

טכניקת Shadowing: קרא את המדריך המלא שלב אחר שלב →