Shadowing Practice: Why Memory Prices Are Crazy High - Learn English Speaking with Video

This is a graphics card for home gaming, the RTX 5090.

On its board, the first thing you see is the GPU in the middle.

Around the GPU is a ring of video memory chip.

When the GPU is working, its instructions, data, and temporary results are stored here.

And this is one compute node in NVIDIA's GP300 AI data center system.

The node has four GPUs.

But around them, we cannot see memory chips like the ones on the 5090.

So can AI server GPU work without video memory?

Of course not.

Look closely.

Next to the four GPUs are two rows of small squares.

These are the memory chips.

But unlike the memory on the gaming card, they are not placed on the board by themselves.

They are packaged together with the GPU.

And this special kind of memory is the star of today's video, HBM.

HBM uses 3D stacking to give it a high bandwidth and pack more parts into a small space.

It is now a key part of high -end AI graphics card.

It's also one of the main reasons normal memory prices have gone up recently

so how does hbm work

and how is it made in this video we are going

to learn all about hbm hi everyone i'm richard from htx

studio this video was originally made by right not a very

talented chinese content creator with his permission we translate it into

english for you guys we hope you enjoy it hbm stands for high bandwidth memory.

Bandwidth means how fast data can be moved.

So high bandwidth simply means that data can move very fast.

The memory chips and the GPU are connected by wires.

They use voltage signals to send data.

For example, let's say we want to send this data 0 1 0 1 0.

We can use high voltage for 1 and low voltage for 0.

So the voltage on the wire becomes low, high, low, high, low.

If want to send the data faster we can make each

voltage state last for last time we make this low high

low high low become this low high low high low in other words we erase the frequency

but frequency is not the only way to move data faster we can also add more wires

and send different parts of the data at the same time

this increases the bus width bandwidth equals frequency times spot width

so to get more bandwidth we can raise the frequency increase the bus width

or do both But raising the frequency of video memory causes many problems.

The voltage on the wire cannot go from low to high in no time.

It also cannot go from high to low in no time.

The change takes time.

We must wait for the voltage to fully rise or fall before the next cycle can start.

So frequency has a clear limit.

A higher frequency also causes more noise between nearby data lines.

That is why HBM uses a very wide bus to get more bandwidth.

A normal GDDR7 memory chip like the ones on the RTX 5090 has a bus width of 32 bits.

The 5090 can use up to 16 of these memory chips.

That gives it a total bus width of 512 bits, but one HDBM3 memory stack has a bus width of 1024 bits.

This B300 GPU has 8 HDBM3 stacks around it, so its total bus width is 8192 bits.

But how can HBM fit such a wide bus into such a small space?

As we just said, memory sends data to the GPU through wires.

More wires means a wider bus.

One bit of bus width needs one wire for data.

A normal GDDR7 memory chip has a bus width of 32 bits.

This means there are 32 data lines between the memory chip and the GPU.

But in real life, it needs many more lines.

It also needs lines for power, addresses, clock signals, and more.

So a GDDR7 chip can have 266 connection points.

And that is only for a bus width of 32 bits.

HBM has a bus width of 1024 bits.

So it needs 1024 lines just for data.

After we add the lines for power, addresses, and other jobs, one HBM3E stack has 3982 connection points.

So this is HBM's first problem.

How can it fit so many connections into such a small space?

First, let's see how normal GDDR memory is made.

The main memory circuits are made on a thin piece of silicon, but this silicon die is very easy to break.

The contact points on it are also very small, so it is hard to work with and use.

That is why we need a packaged substrate.

It is made from woven glass fiber and epoxy resin.

It is much stronger than the silicon die.

The finished die is turned upside down and placed face down on the substrate.

The contact points on the die connect straight to the substrate.

Inside the substrate are many layers of wires.

They move the signals to the back of the substrate, where larger solder balls are added.

The substrate protects the silicon die.

It also moves the tiny contact points on the die to the back of a much stronger package.

This makes the next step like soldering it onto a circuit board much easier.

To make this kind of substrate, glass fiber cloth is first soaked in epoxy resin.

It is pressed at high heat until it becomes a hard board.

Then a layer of copper foil is placed on both sides to make the circuit.

A light -sensitive film is put on top of the copper.

A mask with the circuit pattern is placed over it.

Then ultraviolet light shines on the board.

The parts of the film hit by the light change.

A developing liquid then washes away the parts that were not hit by light.

That leaves some of the copper open.

The board is then placed in an etching liquid.

The open copper is eaten away, while the copper under the film stays.

This is how the circuit lines are made.

A real packaged substrate usually has four to eight layers.

They are stacked together like pancakes.

Small holes are made with a laser to connect the circuits on different layers.

But this way of making circuits has a big limit.

The circuit lines cannot be packed very close together.

The surface of glass fiber is rough and not flat.

So when the copper foil and the light -sensitive film are pressed onto it, small air bubbles can get trapped inside.

If a circuit line is too thin, one small bubble can break it.

There is another problem.

100

This process uses a liquid chemical to eat away the copper.

101

But the chemical does not only eat downward.

102

It also eats into the sides.

103

If a line is too thin, the sides can be eaten away until the line breaks.

104

Because of these limits, a normal packaged substrate cannot hold very dense circuit lines.

105

And there is one more problem.

106

Normal video memory must be soldered onto a PCB.

107

The PCB then connects the memory to the GPU.

108

A PCB is made in almost the same way as the substrate

109

so its circuit lines also cannot be packed very close together.

110

But HBM needs almost 4 ,000 connections.

111

A substrate made from glass fiber cannot do this.

112

Its surface is just too rough.

113

We need a new material.

114

The material is silicon.

115

Yes, the same silicon used to make chips.

116

We start with one large piece of silicon.

117

Its surface is polished until it is very smooth.

118

Then photolithography and etching are used to make very small grooves on its surface.

119

Copper is put into those grooves to form circuit lines.

120

The HBM chips no longer need to sit on a normal substrate.

121

They can be soldered straight onto this large piece of silicon.

122

The GPU is also soldered straight onto it.

123

Now the HBM and the GPU can connect through this silicon.

124

This large piece of silicon which connects the chips together is called a silicon interpulsor.

125

A silicon interpulsor is like a chip with no transistors.

126

Its surface is even flatter than a mirror.

127

This lets chip makers use photolithography, etching, and thin film deposition to make very fine metal lines.

128

On a glass fiber substrate, the lines are more than 10 micrometers wide.

129

But on a silicon interpulsor, they can be less than 1 micrometer wide.

130

They can even be only a few hundred nanometers wide.

131

This means the silicon interpulsor can hold tens or even hundreds of times more circuit lines in the same space.

132

With the silicon interpulsor, HBM can fit many more connections into a small area.

133

And that solves the speed problem.

134

But AI chips needs one more thing from video memory, more capacity.

135

An HBM stack and a normal GDDR memory chip take up about the same amount of space on the board.

136

A GDDR7 chip like the one on RTX 5090 can usually hold 2GB.

137

Even if we put 16 of them around the GPU, the total is only 32GB.

138

That is enough for gaming, but it is still too little for AI.

139

So how can we increase the capacity?

140

The answer is stacking.

141

We stack several bare memory dies on top of one another.

142

They can all use the same silicon interpulsor at the bottom.

143

This let us increase the capacity many times without using more space at the bottom.

144

But stacking many chips is not easy.

145

Putting more than 10 memory chips on top of one another may look simple.

146

But there is one big problem.

147

How does the chip on top talk to the chip below it?

148

Can we pull wires out from the sides of every chip and connect them to the bottom?

149

No, every layer would need thousands of wires.

150

There would not be enough space on the silicon interpulsor and every wire would have to be very thin.

151

Even gold wire cannot hang in the air at such a small size.

152

So engineers came up with another idea.

153

They make holes in the silicon and fill them with copper.

154

Every memory chip connects to these copper columns.

155

The data can move down through the columns and reach the silicon interposer at the bottom.

156

This is called a through silicon via, or TSV.

157

But making these holes is not easy.

158

Each hole is only about 5 to 10 micrometers wide a normal drill cannot make a hole this small.

159

The industry uses a method called deep reactive ion etching.

160

It is also called the Bosch process.

161

This is a silicon wafer that already has transistor circuits on it.

162

Now we need to make deep holes in it.

163

First, we put photoresist on the surface of the wafer.

164

Then a lithography machine shines the pattern of the holes onto the photoresist.

165

The parts hit by the lights change.

166

A developing liquid washes these parts away and opens the silicon below.

167

Next, sulfur hexafluorid gas or SF6 enters the chamber.

168

A radio frequency coil turns the gas into plasma and makes three fluorine radicals.

169

These fluorine radicals react with the open silicon.

170

This makes silicon tetrafluorid gas.

171

In simple words, the silicon is eaten away.

172

But this does not only happen downward.

173

It also happens sideways.

174

If we keep doing this, we will get a large round space instead of a straight deep hole.

175

So after FS6 has been used for a short time, it is stopped.

176

Then Octafluorocyte cubitane or C4F8 enters the chamber.

177

It is also turned into plasma.

178

This makes fluorocarbon radicals.

179

They gather on the silicon and form a protective film.

180

Then FS6 enters the chamber again.

181

But this time, the protective film stops it from eating the silicon.

182

When SF6 is turned into plasma, it makes both fluorine radicals and charged ions.

183

An electric field guides these ions straight down towards the wafer.

184

The ions hit the protective film at the bottom of the hole and remove it.

185

Because the ions move straight down, they only remove the film at the bottom.

186

The film on the sides stays in place.

187

Now the etching glass can keep eating the silicon at the bottom.

188

Then the same steps happen again.

189

Ash, protect, pit.

190

Ash, protect, pit the hole slowly gets deeper.

191

A silicon wafer may begin around 700 to 800 micrometers thick, but at this stage, the hole may only be etched about 100 micrometers deep.

192

Later, the wafer will be made thinner from the back until the hole comes out the other side, but we do not make it thinner yet.

193

First, we need to fill the hole with copper.

194

We use vapor deposition to put a layer of silicon dioxide on the inside wall of the hole.

195

This layer is very dense and does not carry electricity.

196

It It is like putting a tiny ceramic tube around the copper column.

197

It keeps the copper away from the silicon, but this is still not enough.

198

Copper items can move around.

199

Over time, they can move into the silicon.

200

If this happens, they can damage the crystal structure of the silicon.

201

So we need another layer between the silicon dioxide and the copper.

202

This is called a barrier layer.

203

It is often made from tantalum or tantalum nitride.

204

The layer is only a few nanometers thick, but it can stop copper items from moving into the silicon.

205

Next, a very thin copper seed layer is put on top of the barrier layer.

206

Then a wafer is placed in the copper sulfate liquid for electroplating.

207

Copper ions slowly build up on the bottom and the walls of the hole.

208

In the end, copper fills the entire hole.

209

The wafer is then ground thinner from the back onto the hole and the copper inside are open.

210

Now we have a copper connection that goes through the silicon.

211

This is how one layer is made.

212

Next, lithography and electroplating are used to grow tiny copper bumps on top of the open copper columns.

213

Their tops are covered with a thin layer of tin.

214

These are called micro bumps.

215

Now we can stack several memory dies together.

216

We heat them and press them.

217

The tin on the micro bumps melts and solders the layers together.

218

Now every chip is connected through the copper columns and the micro bumps.

219

Data from the top layers can move all the way down to the silicon interposer.

220

At this point, the main memory part of HBM is finished, but the chip is still very easy to break, so we cannot use it yet.

221

As we said before, every memory die has been ground down to a thickness of only a few dozen micrometers.

222

The thin dies are held up by tiny micro bumps between them.

223

It is like a thin sheet of dry seaweed hanging in the air.

224

A small force can break it.

225

The answer is to fill the spaces between the dies, so they are no longer hanging in the air.

226

There are two main ways to do this.

227

Only three companies can mass -produce HBM today, SK Hynix, Samsung, and Micron.

228

SK Hynix uses the first method.

229

It stacks all the dyes first, then it heats the whole stack and solders all the micro bombs at the same time.

230

The full stack is put into a mold.

231

Liquid epoxy resin is poured into the spaces between the dyes, heat and pressure that make the resin hard.

232

Samsung and Micron uses another method.

233

They put a thin polymer film between every two dyes.

234

Under heat and pressure, the film melts and the micro -bombs are soldered together.

235

Then the film becomes hard as it cools.

236

They repeat this process one layer at a time.

237

The SK Hynix method works better in two ways.

238

First, it solders the whole stack at once instead of doing it one layer at a time, so it is much faster to make.

239

Second, there is nothing else between the chips when the micro -bombs are being soldered.

240

This means fewer things can go wrong, so the yield is higher.

241

HBM has many chips stacked on top of one another, so heat is a big problem.

242

To help heat move out, makers add extra through silicon vias.

243

These extra vias do not carry data, they are only used to move heat.

244

SK Hynix also mixes particles that carry heat well into the epoxy Ryzen.

245

This makes the epoxy work a little like thermal paste, so their HBM can move heat away about twice as well as Samsung's.

246

This has helped SK Hynix became the biggest company in the HBM market.

247

Their market share is about 50%.

248

With a silicon interposer, HBM can have a bus width of 1 ,024 bits.

249

And by stacking memory dice, HBM can hold a lot more data.

250

In this GP300 server node, every HBM stack has 12 memory layers.

251

Each stack can hold 36GB.

252

Each GPU has 8 HBM stacks.

253

So that is 288GB for 1 GPU.

254

The node has 4 GPUs.

255

So it has 1 ,152GB of video memory in total.

256

That is more than 1TB in just one server node.

257

So how much does one node cost?

258

About $300 ,000.

259

Today, HBM3E is the version used at a large scale, but companies have already started making HBM4.

260

HBM4 has a buzz width of 2048 bits, so it can stack up to 16 memory layers.

261

So HBM4 needs even denser connections and an even taller stack.

262

HBM4 still uses micro bombs to join the layers, but it may be the last HBM generation to use them.

263

As the connections get denser, the space between the micro bombs gets smaller.

264

During soldering, two nearby micro bombs may join by mistake.

265

This can cause a short circuit, so in the future, HBM may remove the micro bombs.

266

Instead, two clean and smooth copper surfaces will be pressed straight together.

267

The copper items will move into each other and form one piece.

268

This is called hybrid bounding without micro bumps the connections can

269

be packed closer together the stack can be shorter heat can move out more easily

270

and no epoxy is needed between the layers

271

but we still do not know

272

which hbm generation will use this technology at a large scale

273

a few years ago the plan was to use it in hbm4

274

but that did not happen and

275

that is all for this video hbm has helped ai grow by giving it the key memory

276

and computing base it needs but hbm uses many of the same processes and machines as normal DRAM.

277

So making more HPM takes production space away from normal memory.

278

This means there's not enough normal memory for everyone who wants it.

279

So prices go up.

280

Do you believe in the future of HPM?

281

Do you believe in the future of AI?

282

That's it for this video.

283

See you next time.

📺 Zelfde Kanaal

Is Fast Charging Killing the Battery? A 2-Year Test on 40 Phones

Business EnglishUniversity StudentsWorking Adults

We Built the Product Apple Gave Up On...

Business EnglishUniversity StudentsWorking Adults

✨ Aanbevolen Video

65 zinnen

How do microSD Cards Store 2 TB in such a TINY space?

Daily ConversationUniversity StudentsWorking Adults

85 zinnen

Significant slowdown in AI payoff could tip economy into a recession, says Apollo Global's Slok

Business EnglishDaily ConversationUniversity Students

75 zinnen

Woordenschat en spreektips bij deze les

Deze spreekles op niveau C1 is gebaseerd op de video “Why Memory Prices Are Crazy High”. Deze woorden komen het vaakst terug: silicon, memory, hbm, copper, chip. Deze video bevat 283 zinnen en 3140 woorden om na te spreken. Het gesproken deel duurt 16:58. De spreker praat snel, ongeveer 185 woorden per minuut, dus reken op verbonden en ingeslikte klanken. Slechts 76% van de woorden hoort bij de 3.000 meest gebruikte Engelse woorden, dus de woordenschat is pittig.

Belangrijke woorden in deze video

De 15 moeilijkste woorden uit de video, met uitspraak en betekenis:

Woord	Uitspraak	Betekenis
silicon zelfstandig naamwoord	/ˈsɪləˌkɑn/	silicium
stack zelfstandig naamwoord	/stæk/	stapel
substrate zelfstandig naamwoord	/ˈsʌbˌstɹeɪt/	substraat
width zelfstandig naamwoord	/ˈwɪtθ/	breedte
microwave zelfstandig naamwoord	/ˈmaɪkɹoʊˌweɪv/	microgolf
solder zelfstandig naamwoord	/ˈsɒl.də(ɹ)/	soldeersel
wafer zelfstandig naamwoord	/ˈweɪ̯fɚ/	wafel
bandwidth zelfstandig naamwoord	/ˈbændwɪdθ/	bandbreedte
voltage zelfstandig naamwoord	/ˈvəʊltɪd͡ʒ/	spanning
etch werkwoord	/ɛt͡ʃ/	etsen
micrometer zelfstandig naamwoord	/maɪˈkɹɑmɪtəɹ/	micrometer
bump zelfstandig naamwoord	/bʌmp/	botsing
resin zelfstandig naamwoord	/ˈɹɛzɪn/	hars
dense bijvoeglijk naamwoord	/ˈdɛns/	dicht
fluorine zelfstandig naamwoord	/ˈflʊ(ə)ɹˌin	fluor

Phrasal verbs die je zult horen

Woord	Uitspraak	Betekenis
come out werkwoord	/ˌkʌm ˈaʊt/	uitkomen
come up with werkwoord		bedenken, verzinnen

Grammatica in deze video

De structuren die de spreker het meest gebruikt, met de exacte woorden uit de video:

Structuur	In de video
“Used to” used to + werkwoord — een vroegere gewoonte of toestand die niet meer klopt	used to make · used to move
Lijdende vorm be + voltooid deelwoord — het gaat om wat er gebeurt, niet om wie het doet	are stored · are not placed · are packaged
Present perfect have/has + voltooid deelwoord — iets uit het verleden dat nu nog telt	have gone · has been · has helped

Uitspraak om op te letten

De “th”-klanken: width /ˈwɪtθ/, bandwidth /ˈbændwɪdθ/, lithography /lɪθˈɒɡɹəfi/, thickness /ˈθɪknəs/, cloth /klɒθ/
Lange woorden — let op de klemtoon: micrometer /maɪˈkɹɑmɪtəɹ/, lithography /lɪθˈɒɡɹəfi/, deposition /ˌdɛp.əˈzɪʃ.ən/, ultraviolet /ˌʌltɹəˈvaɪələt/

Klanken die Nederlandstaligen lastig vinden:

Stemhebbende eindmedeklinker — /b/, /d/, /g/, /z/, /v/ niet verscherpen: microwave /ˈmaɪkɹoʊˌweɪv/, voltage /ˈvəʊltɪd͡ʒ/, node /noʊd/, columns /ˈkɑləmz/, downward /ˈdaʊnwɚd/
/æ/ — opener dan de Nederlandse “e”: stack /stæk/, bandwidth /ˈbændwɪdθ/, plasma /ˈplæzmə/, tantalum /ˈtæntələm/, transistor /tɹænˈzɪs.tɚ/
/g/ — een harde plofklank, geen Nederlandse “g”: lithography /lɪθˈɒɡɹəfi/, groove /ɡɹuv/

Zo oefen je met deze video

Luister de hele video één keer zonder te spreken en noteer de woorden die je niet kent.
Begin op 0,75× snelheid, spreek zin voor zin na en ga terug naar normale snelheid zodra het makkelijk gaat.
Neem jezelf op en vergelijk met het origineel; let daarbij op woorden als silicon, stack, substrate.

Wat is de Shadowing-techniek?

Shadowing is een wetenschappelijk onderbouwde taalleermethode die oorspronkelijk is ontwikkeld voor professionele tolkentraining en gepopulariseerd door polyglot Dr. Alexander Arguelles. De methode is eenvoudig maar krachtig: je luistert naar native Engelse audio en herhaalt het onmiddellijk hardop — als een schaduw die de spreker volgt met slechts 1–2 seconden vertraging. In tegenstelling tot passief luisteren of grammaticadrills, dwingt shadowing je hersenen en mondspieren om echte spraakpatronen tegelijkertijd te verwerken en te reproduceren. Onderzoek toont aan dat het de uitspraaknauwkeurigheid, intonatie, ritme, verbonden spraak, luisterbegrip en spreekvaardigheid aanzienlijk verbetert — waardoor het een van de meest effectieve methoden is voor IELTS Speaking-voorbereiding en echte Engelse communicatie.

Shadowing-techniek: lees de volledige stap-voor-stap-gids →

Shadowing Practice: Why Memory Prices Are Crazy High - Learn English Speaking with Video

Is Fast Charging Killing the Battery? A 2-Year Test on 40 Phones

We Built the Product Apple Gave Up On...

How do microSD Cards Store 2 TB in such a TINY space?

Significant slowdown in AI payoff could tip economy into a recession, says Apollo Global's Slok

Video

Cache Systems Every Developer Should Know

JUST $1 For Billions of Tokens?! (DeepSeek v4 Pro)

We Can Now Simulate a Human Brain, Scientists Show

The Semiconductor Production Process Explained Clearly

The Absurd Engineering Behind the Internet

Enhancing AV Simulation with Generative AI - DRIVE Labs Ep. 36

Gemma 4 Just Released - Whats In? In 2026!

Memory is crazy expensive so I made my own…

Once you see this, You’ll see Competitive Games Differently