Shadowing Practice: Why Memory Prices Are Crazy High - Learn English Speaking with Video

This is a graphics card for home gaming, the RTX 5090.

On its board, the first thing you see is the GPU in the middle.

Around the GPU is a ring of video memory chip.

When the GPU is working, its instructions, data, and temporary results are stored here.

And this is one compute node in NVIDIA's GP300 AI data center system.

The node has four GPUs.

But around them, we cannot see memory chips like the ones on the 5090.

So can AI server GPU work without video memory?

Of course not.

Look closely.

Next to the four GPUs are two rows of small squares.

These are the memory chips.

But unlike the memory on the gaming card, they are not placed on the board by themselves.

They are packaged together with the GPU.

And this special kind of memory is the star of today's video, HBM.

HBM uses 3D stacking to give it a high bandwidth and pack more parts into a small space.

It is now a key part of high -end AI graphics card.

It's also one of the main reasons normal memory prices have gone up recently

so how does hbm work

and how is it made in this video we are going

to learn all about hbm hi everyone i'm richard from htx

studio this video was originally made by right not a very

talented chinese content creator with his permission we translate it into

english for you guys we hope you enjoy it hbm stands for high bandwidth memory.

Bandwidth means how fast data can be moved.

So high bandwidth simply means that data can move very fast.

The memory chips and the GPU are connected by wires.

They use voltage signals to send data.

For example, let's say we want to send this data 0 1 0 1 0.

We can use high voltage for 1 and low voltage for 0.

So the voltage on the wire becomes low, high, low, high, low.

If want to send the data faster we can make each

voltage state last for last time we make this low high

low high low become this low high low high low in other words we erase the frequency

but frequency is not the only way to move data faster we can also add more wires

and send different parts of the data at the same time

this increases the bus width bandwidth equals frequency times spot width

so to get more bandwidth we can raise the frequency increase the bus width

or do both But raising the frequency of video memory causes many problems.

The voltage on the wire cannot go from low to high in no time.

It also cannot go from high to low in no time.

The change takes time.

We must wait for the voltage to fully rise or fall before the next cycle can start.

So frequency has a clear limit.

A higher frequency also causes more noise between nearby data lines.

That is why HBM uses a very wide bus to get more bandwidth.

A normal GDDR7 memory chip like the ones on the RTX 5090 has a bus width of 32 bits.

The 5090 can use up to 16 of these memory chips.

That gives it a total bus width of 512 bits, but one HDBM3 memory stack has a bus width of 1024 bits.

This B300 GPU has 8 HDBM3 stacks around it, so its total bus width is 8192 bits.

But how can HBM fit such a wide bus into such a small space?

As we just said, memory sends data to the GPU through wires.

More wires means a wider bus.

One bit of bus width needs one wire for data.

A normal GDDR7 memory chip has a bus width of 32 bits.

This means there are 32 data lines between the memory chip and the GPU.

But in real life, it needs many more lines.

It also needs lines for power, addresses, clock signals, and more.

So a GDDR7 chip can have 266 connection points.

And that is only for a bus width of 32 bits.

HBM has a bus width of 1024 bits.

So it needs 1024 lines just for data.

After we add the lines for power, addresses, and other jobs, one HBM3E stack has 3982 connection points.

So this is HBM's first problem.

How can it fit so many connections into such a small space?

First, let's see how normal GDDR memory is made.

The main memory circuits are made on a thin piece of silicon, but this silicon die is very easy to break.

The contact points on it are also very small, so it is hard to work with and use.

That is why we need a packaged substrate.

It is made from woven glass fiber and epoxy resin.

It is much stronger than the silicon die.

The finished die is turned upside down and placed face down on the substrate.

The contact points on the die connect straight to the substrate.

Inside the substrate are many layers of wires.

They move the signals to the back of the substrate, where larger solder balls are added.

The substrate protects the silicon die.

It also moves the tiny contact points on the die to the back of a much stronger package.

This makes the next step like soldering it onto a circuit board much easier.

To make this kind of substrate, glass fiber cloth is first soaked in epoxy resin.

It is pressed at high heat until it becomes a hard board.

Then a layer of copper foil is placed on both sides to make the circuit.

A light -sensitive film is put on top of the copper.

A mask with the circuit pattern is placed over it.

Then ultraviolet light shines on the board.

The parts of the film hit by the light change.

A developing liquid then washes away the parts that were not hit by light.

That leaves some of the copper open.

The board is then placed in an etching liquid.

The open copper is eaten away, while the copper under the film stays.

This is how the circuit lines are made.

A real packaged substrate usually has four to eight layers.

They are stacked together like pancakes.

Small holes are made with a laser to connect the circuits on different layers.

But this way of making circuits has a big limit.

The circuit lines cannot be packed very close together.

The surface of glass fiber is rough and not flat.

So when the copper foil and the light -sensitive film are pressed onto it, small air bubbles can get trapped inside.

If a circuit line is too thin, one small bubble can break it.

There is another problem.

100

This process uses a liquid chemical to eat away the copper.

101

But the chemical does not only eat downward.

102

It also eats into the sides.

103

If a line is too thin, the sides can be eaten away until the line breaks.

104

Because of these limits, a normal packaged substrate cannot hold very dense circuit lines.

105

And there is one more problem.

106

Normal video memory must be soldered onto a PCB.

107

The PCB then connects the memory to the GPU.

108

A PCB is made in almost the same way as the substrate

109

so its circuit lines also cannot be packed very close together.

110

But HBM needs almost 4 ,000 connections.

111

A substrate made from glass fiber cannot do this.

112

Its surface is just too rough.

113

We need a new material.

114

The material is silicon.

115

Yes, the same silicon used to make chips.

116

We start with one large piece of silicon.

117

Its surface is polished until it is very smooth.

118

Then photolithography and etching are used to make very small grooves on its surface.

119

Copper is put into those grooves to form circuit lines.

120

The HBM chips no longer need to sit on a normal substrate.

121

They can be soldered straight onto this large piece of silicon.

122

The GPU is also soldered straight onto it.

123

Now the HBM and the GPU can connect through this silicon.

124

This large piece of silicon which connects the chips together is called a silicon interpulsor.

125

A silicon interpulsor is like a chip with no transistors.

126

Its surface is even flatter than a mirror.

127

This lets chip makers use photolithography, etching, and thin film deposition to make very fine metal lines.

128

On a glass fiber substrate, the lines are more than 10 micrometers wide.

129

But on a silicon interpulsor, they can be less than 1 micrometer wide.

130

They can even be only a few hundred nanometers wide.

131

This means the silicon interpulsor can hold tens or even hundreds of times more circuit lines in the same space.

132

With the silicon interpulsor, HBM can fit many more connections into a small area.

133

And that solves the speed problem.

134

But AI chips needs one more thing from video memory, more capacity.

135

An HBM stack and a normal GDDR memory chip take up about the same amount of space on the board.

136

A GDDR7 chip like the one on RTX 5090 can usually hold 2GB.

137

Even if we put 16 of them around the GPU, the total is only 32GB.

138

That is enough for gaming, but it is still too little for AI.

139

So how can we increase the capacity?

140

The answer is stacking.

141

We stack several bare memory dies on top of one another.

142

They can all use the same silicon interpulsor at the bottom.

143

This let us increase the capacity many times without using more space at the bottom.

144

But stacking many chips is not easy.

145

Putting more than 10 memory chips on top of one another may look simple.

146

But there is one big problem.

147

How does the chip on top talk to the chip below it?

148

Can we pull wires out from the sides of every chip and connect them to the bottom?

149

No, every layer would need thousands of wires.

150

There would not be enough space on the silicon interpulsor and every wire would have to be very thin.

151

Even gold wire cannot hang in the air at such a small size.

152

So engineers came up with another idea.

153

They make holes in the silicon and fill them with copper.

154

Every memory chip connects to these copper columns.

155

The data can move down through the columns and reach the silicon interposer at the bottom.

156

This is called a through silicon via, or TSV.

157

But making these holes is not easy.

158

Each hole is only about 5 to 10 micrometers wide a normal drill cannot make a hole this small.

159

The industry uses a method called deep reactive ion etching.

160

It is also called the Bosch process.

161

This is a silicon wafer that already has transistor circuits on it.

162

Now we need to make deep holes in it.

163

First, we put photoresist on the surface of the wafer.

164

Then a lithography machine shines the pattern of the holes onto the photoresist.

165

The parts hit by the lights change.

166

A developing liquid washes these parts away and opens the silicon below.

167

Next, sulfur hexafluorid gas or SF6 enters the chamber.

168

A radio frequency coil turns the gas into plasma and makes three fluorine radicals.

169

These fluorine radicals react with the open silicon.

170

This makes silicon tetrafluorid gas.

171

In simple words, the silicon is eaten away.

172

But this does not only happen downward.

173

It also happens sideways.

174

If we keep doing this, we will get a large round space instead of a straight deep hole.

175

So after FS6 has been used for a short time, it is stopped.

176

Then Octafluorocyte cubitane or C4F8 enters the chamber.

177

It is also turned into plasma.

178

This makes fluorocarbon radicals.

179

They gather on the silicon and form a protective film.

180

Then FS6 enters the chamber again.

181

But this time, the protective film stops it from eating the silicon.

182

When SF6 is turned into plasma, it makes both fluorine radicals and charged ions.

183

An electric field guides these ions straight down towards the wafer.

184

The ions hit the protective film at the bottom of the hole and remove it.

185

Because the ions move straight down, they only remove the film at the bottom.

186

The film on the sides stays in place.

187

Now the etching glass can keep eating the silicon at the bottom.

188

Then the same steps happen again.

189

Ash, protect, pit.

190

Ash, protect, pit the hole slowly gets deeper.

191

A silicon wafer may begin around 700 to 800 micrometers thick, but at this stage, the hole may only be etched about 100 micrometers deep.

192

Later, the wafer will be made thinner from the back until the hole comes out the other side, but we do not make it thinner yet.

193

First, we need to fill the hole with copper.

194

We use vapor deposition to put a layer of silicon dioxide on the inside wall of the hole.

195

This layer is very dense and does not carry electricity.

196

It It is like putting a tiny ceramic tube around the copper column.

197

It keeps the copper away from the silicon, but this is still not enough.

198

Copper items can move around.

199

Over time, they can move into the silicon.

200

If this happens, they can damage the crystal structure of the silicon.

201

So we need another layer between the silicon dioxide and the copper.

202

This is called a barrier layer.

203

It is often made from tantalum or tantalum nitride.

204

The layer is only a few nanometers thick, but it can stop copper items from moving into the silicon.

205

Next, a very thin copper seed layer is put on top of the barrier layer.

206

Then a wafer is placed in the copper sulfate liquid for electroplating.

207

Copper ions slowly build up on the bottom and the walls of the hole.

208

In the end, copper fills the entire hole.

209

The wafer is then ground thinner from the back onto the hole and the copper inside are open.

210

Now we have a copper connection that goes through the silicon.

211

This is how one layer is made.

212

Next, lithography and electroplating are used to grow tiny copper bumps on top of the open copper columns.

213

Their tops are covered with a thin layer of tin.

214

These are called micro bumps.

215

Now we can stack several memory dies together.

216

We heat them and press them.

217

The tin on the micro bumps melts and solders the layers together.

218

Now every chip is connected through the copper columns and the micro bumps.

219

Data from the top layers can move all the way down to the silicon interposer.

220

At this point, the main memory part of HBM is finished, but the chip is still very easy to break, so we cannot use it yet.

221

As we said before, every memory die has been ground down to a thickness of only a few dozen micrometers.

222

The thin dies are held up by tiny micro bumps between them.

223

It is like a thin sheet of dry seaweed hanging in the air.

224

A small force can break it.

225

The answer is to fill the spaces between the dies, so they are no longer hanging in the air.

226

There are two main ways to do this.

227

Only three companies can mass -produce HBM today, SK Hynix, Samsung, and Micron.

228

SK Hynix uses the first method.

229

It stacks all the dyes first, then it heats the whole stack and solders all the micro bombs at the same time.

230

The full stack is put into a mold.

231

Liquid epoxy resin is poured into the spaces between the dyes, heat and pressure that make the resin hard.

232

Samsung and Micron uses another method.

233

They put a thin polymer film between every two dyes.

234

Under heat and pressure, the film melts and the micro -bombs are soldered together.

235

Then the film becomes hard as it cools.

236

They repeat this process one layer at a time.

237

The SK Hynix method works better in two ways.

238

First, it solders the whole stack at once instead of doing it one layer at a time, so it is much faster to make.

239

Second, there is nothing else between the chips when the micro -bombs are being soldered.

240

This means fewer things can go wrong, so the yield is higher.

241

HBM has many chips stacked on top of one another, so heat is a big problem.

242

To help heat move out, makers add extra through silicon vias.

243

These extra vias do not carry data, they are only used to move heat.

244

SK Hynix also mixes particles that carry heat well into the epoxy Ryzen.

245

This makes the epoxy work a little like thermal paste, so their HBM can move heat away about twice as well as Samsung's.

246

This has helped SK Hynix became the biggest company in the HBM market.

247

Their market share is about 50%.

248

With a silicon interposer, HBM can have a bus width of 1 ,024 bits.

249

And by stacking memory dice, HBM can hold a lot more data.

250

In this GP300 server node, every HBM stack has 12 memory layers.

251

Each stack can hold 36GB.

252

Each GPU has 8 HBM stacks.

253

So that is 288GB for 1 GPU.

254

The node has 4 GPUs.

255

So it has 1 ,152GB of video memory in total.

256

That is more than 1TB in just one server node.

257

So how much does one node cost?

258

About $300 ,000.

259

Today, HBM3E is the version used at a large scale, but companies have already started making HBM4.

260

HBM4 has a buzz width of 2048 bits, so it can stack up to 16 memory layers.

261

So HBM4 needs even denser connections and an even taller stack.

262

HBM4 still uses micro bombs to join the layers, but it may be the last HBM generation to use them.

263

As the connections get denser, the space between the micro bombs gets smaller.

264

During soldering, two nearby micro bombs may join by mistake.

265

This can cause a short circuit, so in the future, HBM may remove the micro bombs.

266

Instead, two clean and smooth copper surfaces will be pressed straight together.

267

The copper items will move into each other and form one piece.

268

This is called hybrid bounding without micro bumps the connections can

269

be packed closer together the stack can be shorter heat can move out more easily

270

and no epoxy is needed between the layers

271

but we still do not know

272

which hbm generation will use this technology at a large scale

273

a few years ago the plan was to use it in hbm4

274

but that did not happen and

275

that is all for this video hbm has helped ai grow by giving it the key memory

276

and computing base it needs but hbm uses many of the same processes and machines as normal DRAM.

277

So making more HPM takes production space away from normal memory.

278

This means there's not enough normal memory for everyone who wants it.

279

So prices go up.

280

Do you believe in the future of HPM?

281

Do you believe in the future of AI?

282

That's it for this video.

283

See you next time.

📺 Aynı Kanal

Is Fast Charging Killing the Battery? A 2-Year Test on 40 Phones

Business EnglishUniversity StudentsWorking Adults

We Built the Product Apple Gave Up On...

Business EnglishUniversity StudentsWorking Adults

✨ Video Öner

65 cümleler

How do microSD Cards Store 2 TB in such a TINY space?

Daily ConversationUniversity StudentsWorking Adults

85 cümleler

Significant slowdown in AI payoff could tip economy into a recession, says Apollo Global's Slok

Business EnglishDaily ConversationUniversity Students

75 cümleler

Bu dersin kelimeleri ve konuşma notları

Bu C1 seviyesindeki konuşma dersi “Why Memory Prices Are Crazy High” videosuna dayanıyor. En çok tekrarlanan kelimeler: silicon, memory, hbm, copper, chip. Bu videoda gölgeleme çalışması için 283 cümle ve 3140 kelime var. Konuşma bölümü 16:58 sürüyor. Konuşmacı hızlı konuşuyor, dakikada yaklaşık 185 kelime; bu yüzden birbirine bağlanan ve zayıflayan sesler duyacaksınız. Kelimelerin yalnızca %76’i İngilizcede en sık kullanılan 3.000 kelime arasında, bu yüzden kelime dağarcığı zorlayıcı.

Bu videodaki önemli kelimeler

Videodaki en ileri düzey 15 kelime, telaffuzu ve anlamıyla:

Kelime	Telaffuz	Anlam
silicon isim	/ˈsɪləˌkɑn/	silisyum
stack isim	/stæk/	yığın
width isim	/ˈwɪtθ/	genişlik, en
microwave isim	/ˈmaɪkɹoʊˌweɪv/	mikrodalga
solder isim	/ˈsɒl.də(ɹ)/	lehim
wafer isim	/ˈweɪ̯fɚ/	gofret
voltage isim	/ˈvəʊltɪd͡ʒ/	gerilim
etch fiil	/ɛt͡ʃ/	oymak
bump isim	/bʌmp/	çarpma, vuruş
resin isim	/ˈɹɛzɪn/	reçine
dense sıfat	/ˈdɛns/	yoğun, sık
fluorine isim	/ˈflʊ(ə)ɹˌin	flor
dye isim	/daɪ/	boya
plasma isim	/ˈplæzmə/	plazma
compute fiil	/kəmˈpjuːt/	hesaplamak

Duyacağınız deyimsel fiiller

Kelime	Telaffuz	Anlam
move out fiil		evden çıkmak, evi boşaltmak
come out fiil	/ˌkʌm ˈaʊt/	çıkmak

Bu videodaki dil bilgisi

Konuşmacının en çok kullandığı yapılar, videodaki sözcüklerin aynısıyla:

Yapı	Videoda
“Used to” used to + fiil — artık geçerli olmayan eski bir alışkanlık ya da durum	used to make · used to move
Edilgen yapı be + fiilin üçüncü hâli — kimin yaptığı değil, ne olduğu önemli	are stored · are not placed · are packaged
Present perfect have/has + fiilin üçüncü hâli — geçmişte olan ama şimdi de önemli olan bir eylem	have gone · has been · has helped

Dikkat edilecek telaffuzlar

“th” sesleri: width /ˈwɪtθ/, bandwidth /ˈbændwɪdθ/, lithography /lɪθˈɒɡɹəfi/, thickness /ˈθɪknəs/, cloth /klɒθ/
Uzun kelimeler — vurguyu doğru yere koyun: micrometer /maɪˈkɹɑmɪtəɹ/, lithography /lɪθˈɒɡɹəfi/, deposition /ˌdɛp.əˈzɪʃ.ən/, ultraviolet /ˌʌltɹəˈvaɪələt/

Türkçe konuşanların zorlandığı sesler:

/w/ — dudaklar yuvarlak, /v/ değil: width /ˈwɪtθ/, microwave /ˈmaɪkɹoʊˌweɪv/, wafer /ˈweɪ̯fɚ/, bandwidth /ˈbændwɪdθ/, downward /ˈdaʊnwɚd/
Kelime başındaki ünsüz kümesi — araya ünlü eklemeyin: stack /stæk/, fluorine /ˈflʊ(ə)ɹˌin, plasma /ˈplæzmə/, transistor /tɹænˈzɪs.tɚ/, groove /ɡɹuv/
/æ/ — “e”den daha açık: stack /stæk/, bandwidth /ˈbændwɪdθ/, plasma /ˈplæzmə/, tantalum /ˈtæntələm/, transistor /tɹænˈzɪs.tɚ/

Bu videoyla nasıl çalışılır

Videonun tamamını konuşmadan bir kez dinleyin ve bilmediğiniz kelimeleri not edin.
0,75× hızla başlayın, cümle cümle tekrar edin ve kolaylaşınca normal hıza dönün.
Kendinizi kaydedin ve orijinaliyle karşılaştırın; silicon, stack, width gibi kelimelere özellikle dikkat edin.

Gölgeleme Tekniği Nedir?

Gölgeleme, başlangıçta profesyonel tercüman eğitimi için geliştirilen ve çok dilli Dr. Alexander Arguelles tarafından popüler hale getirilen, bilim destekli bir dil öğrenme tekniğidir. Yöntem basit ama güçlüdür: ana dili İngilizce olan bir sesi dinler ve hemen yüksek sesle tekrar edersiniz — konuşmacıyı 1-2 saniye gecikmeyle takip eden bir gölge gibi. Pasif dinleme veya dilbilgisi alıştırmalarının aksine, gölgeleme beyninizi ve ağız kaslarınızı gerçek konuşma kalıplarını eşzamanlı olarak işlemeye ve yeniden üretmeye zorlar. Araştırmalar, telaffuz doğruluğu, tonlama, ritim, bağlı konuşma, dinleme anlama ve konuşma akıcılığını önemli ölçüde geliştirdiğini göstermektedir — bu da onu IELTS Konuşma hazırlığı ve gerçek dünya İngilizce iletişimi için en etkili yöntemlerden biri yapar.

Shadowing tekniği: adım adım eksiksiz rehberi okuyun →

Shadowing Practice: Why Memory Prices Are Crazy High - Learn English Speaking with Video

Is Fast Charging Killing the Battery? A 2-Year Test on 40 Phones

We Built the Product Apple Gave Up On...

How do microSD Cards Store 2 TB in such a TINY space?

Significant slowdown in AI payoff could tip economy into a recession, says Apollo Global's Slok

Video

Cache Systems Every Developer Should Know

JUST $1 For Billions of Tokens?! (DeepSeek v4 Pro)

We Can Now Simulate a Human Brain, Scientists Show

The Semiconductor Production Process Explained Clearly

The Absurd Engineering Behind the Internet

Enhancing AV Simulation with Generative AI - DRIVE Labs Ep. 36

Gemma 4 Just Released - Whats In? In 2026!

Memory is crazy expensive so I made my own…

Once you see this, You’ll see Competitive Games Differently