Shadowing Practice: Why Memory Prices Are Crazy High - Learn English Speaking with Video
Creando lección...
1
This is a graphics card for home gaming, the RTX 5090.
2
On its board, the first thing you see is the GPU in the middle.
3
Around the GPU is a ring of video memory chip.
4
When the GPU is working, its instructions, data, and temporary results are stored here.
5
And this is one compute node in NVIDIA's GP300 AI data center system.
6
The node has four GPUs.
7
But around them, we cannot see memory chips like the ones on the 5090.
8
So can AI server GPU work without video memory?
9
Of course not.
10
Look closely.
11
Next to the four GPUs are two rows of small squares.
12
These are the memory chips.
13
But unlike the memory on the gaming card, they are not placed on the board by themselves.
14
They are packaged together with the GPU.
15
And this special kind of memory is the star of today's video, HBM.
16
HBM uses 3D stacking to give it a high bandwidth and pack more parts into a small space.
17
It is now a key part of high -end AI graphics card.
18
It's also one of the main reasons normal memory prices have gone up recently
19
so how does hbm work
20
and how is it made in this video we are going
21
to learn all about hbm hi everyone i'm richard from htx
22
studio this video was originally made by right not a very
23
talented chinese content creator with his permission we translate it into
24
english for you guys we hope you enjoy it hbm stands for high bandwidth memory.
25
Bandwidth means how fast data can be moved.
26
So high bandwidth simply means that data can move very fast.
27
The memory chips and the GPU are connected by wires.
28
They use voltage signals to send data.
29
For example, let's say we want to send this data 0 1 0 1 0.
30
We can use high voltage for 1 and low voltage for 0.
31
So the voltage on the wire becomes low, high, low, high, low.
32
If want to send the data faster we can make each
33
voltage state last for last time we make this low high
34
low high low become this low high low high low in other words we erase the frequency
35
but frequency is not the only way to move data faster we can also add more wires
36
and send different parts of the data at the same time
37
this increases the bus width bandwidth equals frequency times spot width
38
so to get more bandwidth we can raise the frequency increase the bus width
39
or do both But raising the frequency of video memory causes many problems.
40
The voltage on the wire cannot go from low to high in no time.
41
It also cannot go from high to low in no time.
42
The change takes time.
43
We must wait for the voltage to fully rise or fall before the next cycle can start.
44
So frequency has a clear limit.
45
A higher frequency also causes more noise between nearby data lines.
46
That is why HBM uses a very wide bus to get more bandwidth.
47
A normal GDDR7 memory chip like the ones on the RTX 5090 has a bus width of 32 bits.
48
The 5090 can use up to 16 of these memory chips.
49
That gives it a total bus width of 512 bits, but one HDBM3 memory stack has a bus width of 1024 bits.
50
This B300 GPU has 8 HDBM3 stacks around it, so its total bus width is 8192 bits.
51
But how can HBM fit such a wide bus into such a small space?
52
As we just said, memory sends data to the GPU through wires.
53
More wires means a wider bus.
54
One bit of bus width needs one wire for data.
55
A normal GDDR7 memory chip has a bus width of 32 bits.
56
This means there are 32 data lines between the memory chip and the GPU.
57
But in real life, it needs many more lines.
58
It also needs lines for power, addresses, clock signals, and more.
59
So a GDDR7 chip can have 266 connection points.
60
And that is only for a bus width of 32 bits.
61
HBM has a bus width of 1024 bits.
62
So it needs 1024 lines just for data.
63
After we add the lines for power, addresses, and other jobs, one HBM3E stack has 3982 connection points.
64
So this is HBM's first problem.
65
How can it fit so many connections into such a small space?
66
First, let's see how normal GDDR memory is made.
67
The main memory circuits are made on a thin piece of silicon, but this silicon die is very easy to break.
68
The contact points on it are also very small, so it is hard to work with and use.
69
That is why we need a packaged substrate.
70
It is made from woven glass fiber and epoxy resin.
71
It is much stronger than the silicon die.
72
The finished die is turned upside down and placed face down on the substrate.
73
The contact points on the die connect straight to the substrate.
74
Inside the substrate are many layers of wires.
75
They move the signals to the back of the substrate, where larger solder balls are added.
76
The substrate protects the silicon die.
77
It also moves the tiny contact points on the die to the back of a much stronger package.
78
This makes the next step like soldering it onto a circuit board much easier.
79
To make this kind of substrate, glass fiber cloth is first soaked in epoxy resin.
80
It is pressed at high heat until it becomes a hard board.
81
Then a layer of copper foil is placed on both sides to make the circuit.
82
A light -sensitive film is put on top of the copper.
83
A mask with the circuit pattern is placed over it.
84
Then ultraviolet light shines on the board.
85
The parts of the film hit by the light change.
86
A developing liquid then washes away the parts that were not hit by light.
87
That leaves some of the copper open.
88
The board is then placed in an etching liquid.
89
The open copper is eaten away, while the copper under the film stays.
90
This is how the circuit lines are made.
91
A real packaged substrate usually has four to eight layers.
92
They are stacked together like pancakes.
93
Small holes are made with a laser to connect the circuits on different layers.
94
But this way of making circuits has a big limit.
95
The circuit lines cannot be packed very close together.
96
The surface of glass fiber is rough and not flat.
97
So when the copper foil and the light -sensitive film are pressed onto it, small air bubbles can get trapped inside.
98
If a circuit line is too thin, one small bubble can break it.
99
There is another problem.
100
This process uses a liquid chemical to eat away the copper.
101
But the chemical does not only eat downward.
102
It also eats into the sides.
103
If a line is too thin, the sides can be eaten away until the line breaks.
104
Because of these limits, a normal packaged substrate cannot hold very dense circuit lines.
105
And there is one more problem.
106
Normal video memory must be soldered onto a PCB.
107
The PCB then connects the memory to the GPU.
108
A PCB is made in almost the same way as the substrate
109
so its circuit lines also cannot be packed very close together.
110
But HBM needs almost 4 ,000 connections.
111
A substrate made from glass fiber cannot do this.
112
Its surface is just too rough.
113
We need a new material.
114
The material is silicon.
115
Yes, the same silicon used to make chips.
116
We start with one large piece of silicon.
117
Its surface is polished until it is very smooth.
118
Then photolithography and etching are used to make very small grooves on its surface.
119
Copper is put into those grooves to form circuit lines.
120
The HBM chips no longer need to sit on a normal substrate.
121
They can be soldered straight onto this large piece of silicon.
122
The GPU is also soldered straight onto it.
123
Now the HBM and the GPU can connect through this silicon.
124
This large piece of silicon which connects the chips together is called a silicon interpulsor.
125
A silicon interpulsor is like a chip with no transistors.
126
Its surface is even flatter than a mirror.
127
This lets chip makers use photolithography, etching, and thin film deposition to make very fine metal lines.
128
On a glass fiber substrate, the lines are more than 10 micrometers wide.
129
But on a silicon interpulsor, they can be less than 1 micrometer wide.
130
They can even be only a few hundred nanometers wide.
131
This means the silicon interpulsor can hold tens or even hundreds of times more circuit lines in the same space.
132
With the silicon interpulsor, HBM can fit many more connections into a small area.
133
And that solves the speed problem.
134
But AI chips needs one more thing from video memory, more capacity.
135
An HBM stack and a normal GDDR memory chip take up about the same amount of space on the board.
136
A GDDR7 chip like the one on RTX 5090 can usually hold 2GB.
137
Even if we put 16 of them around the GPU, the total is only 32GB.
138
That is enough for gaming, but it is still too little for AI.
139
So how can we increase the capacity?
140
The answer is stacking.
141
We stack several bare memory dies on top of one another.
142
They can all use the same silicon interpulsor at the bottom.
143
This let us increase the capacity many times without using more space at the bottom.
144
But stacking many chips is not easy.
145
Putting more than 10 memory chips on top of one another may look simple.
146
But there is one big problem.
147
How does the chip on top talk to the chip below it?
148
Can we pull wires out from the sides of every chip and connect them to the bottom?
149
No, every layer would need thousands of wires.
150
There would not be enough space on the silicon interpulsor and every wire would have to be very thin.
151
Even gold wire cannot hang in the air at such a small size.
152
So engineers came up with another idea.
153
They make holes in the silicon and fill them with copper.
154
Every memory chip connects to these copper columns.
155
The data can move down through the columns and reach the silicon interposer at the bottom.
156
This is called a through silicon via, or TSV.
157
But making these holes is not easy.
158
Each hole is only about 5 to 10 micrometers wide a normal drill cannot make a hole this small.
159
The industry uses a method called deep reactive ion etching.
160
It is also called the Bosch process.
161
This is a silicon wafer that already has transistor circuits on it.
162
Now we need to make deep holes in it.
163
First, we put photoresist on the surface of the wafer.
164
Then a lithography machine shines the pattern of the holes onto the photoresist.
165
The parts hit by the lights change.
166
A developing liquid washes these parts away and opens the silicon below.
167
Next, sulfur hexafluorid gas or SF6 enters the chamber.
168
A radio frequency coil turns the gas into plasma and makes three fluorine radicals.
169
These fluorine radicals react with the open silicon.
170
This makes silicon tetrafluorid gas.
171
In simple words, the silicon is eaten away.
172
But this does not only happen downward.
173
It also happens sideways.
174
If we keep doing this, we will get a large round space instead of a straight deep hole.
175
So after FS6 has been used for a short time, it is stopped.
176
Then Octafluorocyte cubitane or C4F8 enters the chamber.
177
It is also turned into plasma.
178
This makes fluorocarbon radicals.
179
They gather on the silicon and form a protective film.
180
Then FS6 enters the chamber again.
181
But this time, the protective film stops it from eating the silicon.
182
When SF6 is turned into plasma, it makes both fluorine radicals and charged ions.
183
An electric field guides these ions straight down towards the wafer.
184
The ions hit the protective film at the bottom of the hole and remove it.
185
Because the ions move straight down, they only remove the film at the bottom.
186
The film on the sides stays in place.
187
Now the etching glass can keep eating the silicon at the bottom.
188
Then the same steps happen again.
189
Ash, protect, pit.
190
Ash, protect, pit the hole slowly gets deeper.
191
A silicon wafer may begin around 700 to 800 micrometers thick, but at this stage, the hole may only be etched about 100 micrometers deep.
192
Later, the wafer will be made thinner from the back until the hole comes out the other side, but we do not make it thinner yet.
193
First, we need to fill the hole with copper.
194
We use vapor deposition to put a layer of silicon dioxide on the inside wall of the hole.
195
This layer is very dense and does not carry electricity.
196
It It is like putting a tiny ceramic tube around the copper column.
197
It keeps the copper away from the silicon, but this is still not enough.
198
Copper items can move around.
199
Over time, they can move into the silicon.
200
If this happens, they can damage the crystal structure of the silicon.
201
So we need another layer between the silicon dioxide and the copper.
202
This is called a barrier layer.
203
It is often made from tantalum or tantalum nitride.
204
The layer is only a few nanometers thick, but it can stop copper items from moving into the silicon.
205
Next, a very thin copper seed layer is put on top of the barrier layer.
206
Then a wafer is placed in the copper sulfate liquid for electroplating.
207
Copper ions slowly build up on the bottom and the walls of the hole.
208
In the end, copper fills the entire hole.
209
The wafer is then ground thinner from the back onto the hole and the copper inside are open.
210
Now we have a copper connection that goes through the silicon.
211
This is how one layer is made.
212
Next, lithography and electroplating are used to grow tiny copper bumps on top of the open copper columns.
213
Their tops are covered with a thin layer of tin.
214
These are called micro bumps.
215
Now we can stack several memory dies together.
216
We heat them and press them.
217
The tin on the micro bumps melts and solders the layers together.
218
Now every chip is connected through the copper columns and the micro bumps.
219
Data from the top layers can move all the way down to the silicon interposer.
220
At this point, the main memory part of HBM is finished, but the chip is still very easy to break, so we cannot use it yet.
221
As we said before, every memory die has been ground down to a thickness of only a few dozen micrometers.
222
The thin dies are held up by tiny micro bumps between them.
223
It is like a thin sheet of dry seaweed hanging in the air.
224
A small force can break it.
225
The answer is to fill the spaces between the dies, so they are no longer hanging in the air.
226
There are two main ways to do this.
227
Only three companies can mass -produce HBM today, SK Hynix, Samsung, and Micron.
228
SK Hynix uses the first method.
229
It stacks all the dyes first, then it heats the whole stack and solders all the micro bombs at the same time.
230
The full stack is put into a mold.
231
Liquid epoxy resin is poured into the spaces between the dyes, heat and pressure that make the resin hard.
232
Samsung and Micron uses another method.
233
They put a thin polymer film between every two dyes.
234
Under heat and pressure, the film melts and the micro -bombs are soldered together.
235
Then the film becomes hard as it cools.
236
They repeat this process one layer at a time.
237
The SK Hynix method works better in two ways.
238
First, it solders the whole stack at once instead of doing it one layer at a time, so it is much faster to make.
239
Second, there is nothing else between the chips when the micro -bombs are being soldered.
240
This means fewer things can go wrong, so the yield is higher.
241
HBM has many chips stacked on top of one another, so heat is a big problem.
242
To help heat move out, makers add extra through silicon vias.
243
These extra vias do not carry data, they are only used to move heat.
244
SK Hynix also mixes particles that carry heat well into the epoxy Ryzen.
245
This makes the epoxy work a little like thermal paste, so their HBM can move heat away about twice as well as Samsung's.
246
This has helped SK Hynix became the biggest company in the HBM market.
247
Their market share is about 50%.
248
With a silicon interposer, HBM can have a bus width of 1 ,024 bits.
249
And by stacking memory dice, HBM can hold a lot more data.
250
In this GP300 server node, every HBM stack has 12 memory layers.
251
Each stack can hold 36GB.
252
Each GPU has 8 HBM stacks.
253
So that is 288GB for 1 GPU.
254
The node has 4 GPUs.
255
So it has 1 ,152GB of video memory in total.
256
That is more than 1TB in just one server node.
257
So how much does one node cost?
258
About $300 ,000.
259
Today, HBM3E is the version used at a large scale, but companies have already started making HBM4.
260
HBM4 has a buzz width of 2048 bits, so it can stack up to 16 memory layers.
261
So HBM4 needs even denser connections and an even taller stack.
262
HBM4 still uses micro bombs to join the layers, but it may be the last HBM generation to use them.
263
As the connections get denser, the space between the micro bombs gets smaller.
264
During soldering, two nearby micro bombs may join by mistake.
265
This can cause a short circuit, so in the future, HBM may remove the micro bombs.
266
Instead, two clean and smooth copper surfaces will be pressed straight together.
267
The copper items will move into each other and form one piece.
268
This is called hybrid bounding without micro bumps the connections can
269
be packed closer together the stack can be shorter heat can move out more easily
270
and no epoxy is needed between the layers
271
but we still do not know
272
which hbm generation will use this technology at a large scale
273
a few years ago the plan was to use it in hbm4
274
but that did not happen and
275
that is all for this video hbm has helped ai grow by giving it the key memory
276
and computing base it needs but hbm uses many of the same processes and machines as normal DRAM.
277
So making more HPM takes production space away from normal memory.
278
This means there's not enough normal memory for everyone who wants it.
279
So prices go up.
280
Do you believe in the future of HPM?
281
Do you believe in the future of AI?
282
That's it for this video.
283
See you next time.
📺 Mismo canal
✨ Recomendar Video
C11
65 oraciones
How do microSD Cards Store 2 TB in such a TINY space?
Daily ConversationUniversity StudentsWorking Adults
B21
85 oraciones
Significant slowdown in AI payoff could tip economy into a recession, says Apollo Global's Slok
Business EnglishDaily ConversationUniversity Students
C11
75 oraciones
Video
Daily ConversationUniversity StudentsWorking Adults
C11
53 oraciones
Cache Systems Every Developer Should Know
Daily ConversationUniversity StudentsWorking Adults
C11
137 oraciones
JUST $1 For Billions of Tokens?! (DeepSeek v4 Pro)
Daily ConversationUniversity StudentsWorking Adults
C185 oraciones
We Can Now Simulate a Human Brain, Scientists Show
Daily ConversationUniversity StudentsWorking Adults
C12
20 oraciones
The Semiconductor Production Process Explained Clearly
Daily ConversationUniversity StudentsWorking Adults
C11
427 oraciones
The Absurd Engineering Behind the Internet
Daily ConversationUniversity StudentsWorking Adults
C11
33 oraciones
Enhancing AV Simulation with Generative AI - DRIVE Labs Ep. 36
Daily ConversationUniversity StudentsWorking Adults
B21
28 oraciones
Gemma 4 Just Released - Whats In? In 2026!
Daily ConversationUniversity StudentsWorking Adults
B22
252 oraciones
Memory is crazy expensive so I made my own…
Business EnglishUniversity StudentsWorking Adults
B2181 oraciones
Once you see this, You’ll see Competitive Games Differently
Daily ConversationUniversity StudentsWorking Adults
Vocabulario y notas de pronunciación para esta lección
Esta lección de conversación de nivel C1 se basa en el vídeo “Why Memory Prices Are Crazy High”. Las palabras que más se repiten: silicon, memory, hbm, copper, chip. Este vídeo tiene 283 frases y 3140 palabras para practicar shadowing. La parte hablada dura 16:58. El hablante habla rápido, unas 185 palabras por minuto, así que espera sonidos enlazados y reducidos. Solo el 76 % de las palabras está entre las 3.000 más comunes del inglés, por lo que el vocabulario es exigente.
Vocabulario clave de este vídeo
Las 15 palabras más avanzadas del vídeo, con su pronunciación y significado:
| Palabra | Pronunciación | Significado |
|---|---|---|
| silicon sustantivo | /ˈsɪləˌkɑn/ | silicio |
| stack sustantivo | /stæk/ | pila, montón |
| substrate sustantivo | /ˈsʌbˌstɹeɪt/ | sustrato, substrato |
| width sustantivo | /ˈwɪtθ/ | anchura, ancheza |
| microwave sustantivo | /ˈmaɪkɹoʊˌweɪv/ | microonda |
| solder sustantivo | /ˈsɒl.də(ɹ)/ | estaño para soldar, estaño |
| wafer sustantivo | /ˈweɪ̯fɚ/ | oblea, barquillo |
| bandwidth sustantivo | /ˈbændwɪdθ/ | ancho de banda, anchura de banda |
| voltage sustantivo | /ˈvəʊltɪd͡ʒ/ | voltaje, tensión |
| epoxy adjetivo | /ɪˈpɑk.si/ | epoxi, epóxico |
| etch verbo | /ɛt͡ʃ/ | grabar, aguaforzar |
| micrometer sustantivo | /maɪˈkɹɑmɪtəɹ/ | micrómetro |
| node sustantivo | /noʊd/ | nudo, ñudo |
| bump sustantivo | /bʌmp/ | bache, bulto |
| resin sustantivo | /ˈɹɛzɪn/ | resina, zopisa |
Phrasal verbs que vas a escuchar
| Palabra | Pronunciación | Significado |
|---|---|---|
| eat away verbo | /it‿əˈweɪ/ | apolillar, cancerar |
| go up verbo | subir | |
| move out verbo | mudarse de acá, independizarse | |
| come out verbo | /ˌkʌm ˈaʊt/ | revelarse, salir a la luz |
| come up with verbo | ingeniar, ingeniarse | |
| go through verbo | atravesar | |
| hold up verbo | aguaitar, esperar | |
| move around verbo | movilizarse |
Gramática en este vídeo
Las estructuras que más usa el hablante, con las palabras exactas del vídeo:
| Estructura | En el vídeo |
|---|---|
| “Used to” used to + verbo — un hábito o estado pasado que ya no es cierto | used to make · used to move |
| Voz pasiva be + participio pasado — importa lo que ocurre, no quién lo hace | are stored · are not placed · are packaged |
| Present perfect have/has + participio pasado — una acción pasada que sigue importando ahora | have gone · has been · has helped |
Pronunciación a tener en cuenta
- Los sonidos de “th”: width /ˈwɪtθ/, bandwidth /ˈbændwɪdθ/, lithography /lɪθˈɒɡɹəfi/, thickness /ˈθɪknəs/, cloth /klɒθ/
- Palabras largas — cuida el acento: micrometer /maɪˈkɹɑmɪtəɹ/, lithography /lɪθˈɒɡɹəfi/, deposition /ˌdɛp.əˈzɪʃ.ən/, ultraviolet /ˌʌltɹəˈvaɪələt/
Sonidos difíciles para hispanohablantes:
- /v/ — no es /b/: los dientes tocan el labio inferior: microwave /ˈmaɪkɹoʊˌweɪv/, voltage /ˈvəʊltɪd͡ʒ/, groove /ɡɹuv/, ultraviolet /ˌʌltɹəˈvaɪələt/, weave /wiːv/
- /z/ — sonora, no /s/: resin /ˈɹɛzɪn/, columns /ˈkɑləmz/, plasma /ˈplæzmə/, transistor /tɹænˈzɪs.tɚ/, deposition /ˌdɛp.əˈzɪʃ.ən/
Cómo practicar con este vídeo
- Escucha el vídeo entero una vez sin hablar y anota las palabras que no conoces.
- Empieza a velocidad 0,75×, haz shadowing frase por frase y vuelve a la velocidad normal cuando te resulte fácil.
- Grábate y compara con el original, prestando atención a palabras como silicon, stack, substrate.
¿Qué es la Técnica de Shadowing?
Shadowing es una técnica de aprendizaje de idiomas respaldada por la ciencia, desarrollada originalmente para la formación de intérpretes profesionales y popularizada por el políglota Dr. Alexander Arguelles. El método es simple pero poderoso: escuchas audio en inglés nativo y lo repites en voz alta de inmediato, como una sombra que sigue al hablante con solo 1-2 segundos de retraso. A diferencia de la escucha pasiva o los ejercicios de gramática, el shadowing obliga a tu cerebro y músculos de la boca a procesar y reproducir simultáneamente patrones de habla reales. Las investigaciones muestran que mejora significativamente la precisión de la pronunciación, la entonación, el ritmo, el habla conectada, la comprensión auditiva y la fluidez al hablar, convirtiéndola en una de las metodologías más efectivas para la preparación del IELTS Speaking y la comunicación en inglés en el mundo real.