シャドーイング練習: Intensity, Loudness, and Timbre - 動画で英語スピーキングを学ぶ

Hi everybody and welcome to another video in the audio processing for machine learning series.

Last time we talked about basic features of sounds, so what sound is, waveforms and introduce the concept of frequency.

This time we continue delving into the features of sound, talking about intensity, power, loudness and timbre.

So first of all I want to talk about a couple of concepts, so intensity and power.

And these are like connected together.

So let's start from the power of sound.

Well, this is, we all know the power of sound, right?

But we don't want to talk about the power of sound here.

We actually want to talk about the reverse of that, the sound power.

So what's the sound power? well this is a physical measure

that we can express like this it's basically the rate at

which the energy is transferred this is basically the idea of the concept of power in general but

if we talk about sound power specifically this is the energy

per unit of time emitted by a sound source in all directions

across I mean like the air across like the the medium

that the sound is traveling in and we measure power in watts okay

and we indicate that with a capital W okay

so this is like the idea of sound power now connected

with sound power we have another thing that's called sound intensity And sound intensity is simply sound power per unit area, right?

And we can measure that as watts divided by squared meters, okay?

So the higher, like, the intensity and obviously the higher, the kind of, like, perception,

like, of loudness of that sound that we're going to have.

Okay, so now I want to ask you, like, a question.

So how much power do you think a thunder, like a super heavy thunder has?

Or how much power do you think a concert orchestra has?

Well, you may think like quite a lot, right?

Because like we perceive it as very loud.

Definitely like the heavy thunder, right?

And it gave us like a few scares, like I guess in the past when we were a child.

Okay, but that's not really the case.

So both a concert orchestra and a heavy thunder usually have a sound power of one watt.

Now if you are not familiar with like any reference like with water I'm gonna give you one.

So your usual like a light bulb used to be a hundred watts.

OK, so you like a whole orchestra is a hundredth, basically, of a light bulb in terms of power.

Same thing for this heavy thunder.

So yeah, this is quite mesmerizing, isn't it?

OK, but basically this tells us another very interesting thing, which is connected with a concept called the threshold of hearing.

So humans are capable of perceiving sounds which have extremely small intensities.

And the threshold of hearing is at 10 to the minus 12 watt over squared meters, right?

So this is like an extremely small number, but still we're capable of hearing that.

So obviously like the threshold of hearing like is the minimum intensity of sound that we can appreciate.

Okay, so now another kind of threshold which is like very important for us is called the threshold of pain.

So which is like the threshold beyond which we start to have like hearing pain, right?

Because like the sound like is like too intense.

okay and that's it at 10 watts over squared meters so

if you just like calculate the difference between 10 to the minus 12

and 10 you appreciate that like the amount the range of intensity

that we can perceive is enormous it's basically like 13 orders of magnitudes which is incredible

And that's why we use the so-called intensity level,

like for describing the intensity of a sound.

And that's why we put the intensity level on a logarithmic scale, because like the range is really, really enormous.

And the intensity level is connected with this measure,

which we call this unit of measure, which we call decibels.

I'm sure like you may be familiar with this but you may be wondering, you may have wondered, okay but what's a decibel?

Well it's a measure of intensity level and it's a ratio between two intensity values.

I just like found that I have like a small like typo here,

it's not ration because I don't know what that is but it's a ratio between two intensity values, right?

and so we use an intensity of reference

which sometimes is the threshold of hearing and then we compare

that against a current like intensity and we applied

that ratio a logarithm and so

if we want to take a look at the function

that describe decibels as a function of intensity is this one right

so we have 10 by log of an intensity divided by

the intensity of the threshold of hearing okay cool okay

so now you may be wondering but what's zero decibels and

so for that we need to pass in uh the intensity of the threshold of hearing

and so if we do

that this ratio uh basically like goes down to uh to one

and logarithm of one is equal to zeros

and we multiply 10 by 0 which is 0 okay

so a 0 at 0 decibel we are basically appreciating the

uh intensity of the threshold of hearing now this is not a universal because it really depends on the

kind of like reference intensity that you use in the intensity level

but usually like you'll find a lot of people like using

the threshold of hearing like as a as an intensity um a level of reference okay

so another thing uh to notice about the intensity level is

that every time we go up by three decibels the intensity tends to double

so if we are at zero decibel we are at the um

intensity of like the threshold of hearing but if we go up by three we are just like doubling

uh that uh intensity cool okay

so now i want to show you a cool table

which by the way is but i like i took it from a book

which is an amazing book like on audio music processing that's called fundamentals of music pro assessing it

and it's by mueller so many of the like uh pictures like

graphs and tables i'm going to use here are just like taken from

that book and i highly suggest you to go check

that out because it's great and it will give you like way more context

that one i'm covering here okay

but here you have a table where we have a lot of like different sound sources

and we have like the intensity measured in watts over a squared meters the

and the relative intensity level and that's like compared in terms of like the threshold of hearing.

Okay so like when we whisper we have an intensity which is at 10 to the minus 10

which is basically 20 decibels so a normal conversation happens at an intensity of 10 to to the minus six,

which is 60 decibels.

And the threshold of pain, which is we know is already intensity 10,

it's 130 decibels and the jet.

So when you have like a jet engine at takeoff,

this is at intensity 10 to two, and it's 140 decibels.

100

Okay, so this gives you like more

101

or less like an idea of like the different sounds

102

and the relative intensity but now let's move on to another aspect

103

that i think like it's extremely fascinating which is loudness now while intensity

104

and power are two objective measures loudness is a subjective measure

105

of intensity it is really like how loud we perceive a sound

106

and as we've seen for um a frequency pitch which is like the relative like um

107

kind of like subjective measure of frequency obviously it correlates with frequency

108

but like there's some level of like uh subjectivity

109

and some level of like mapping that's not like linear not the direct

110

and the same thing happens here between intensity and loudness okay so um

111

loudness so first of all depends on duration and the frequency of the sound in terms of duration

112

So we usually hear as at equal intensity,

113

we hear shorter sounds as less loud than longer sounds.

114

So for example, if I have a sound which is at,

115

say, 3 decibels and it only lasts for 100 milliseconds,

116

I'll hear that as less loud than a similar sound with the same 3 decibel intensity level,

117

but which lasts for 600 milliseconds.

118

So this is like kind of like very fascinating because at the end of the day the intensity is the same

119

but the perception of loudness is different.

120

And a similar thing happens also with frequency.

121

So frequency, depending on which frequency the sound is, we're going to hear that sound at a different loudness level.

122

And obviously loudness is also correlated with age.

123

So people of different age are going to hear like sound with the same intensity, but like with different loudness, right?

124

And for loudness, so what researchers have done is basically they went out there

125

and they did a lot of experiments to understand how we perceive sound and measured that with psychological experiments.

126

And they came out with this measure, unit of measure called phones.

127

So now let's take a look at phones.

128

So here you have a very interesting chart that's called like equal loudness contours.

129

So now on the y-axis you have the sound pressure level measured in decibels.

130

So that's the intensity level.

131

Here on the x-axis you have

132

frequency in a logarithmic scale and it's measured obviously in Hertz you should know

133

that by now and here like this curves

134

that you see like in red so these are called equal loudness contours

135

so along these curves we hear a sound to at a same intensity so for example on this curve here

136

we hear sound at 80 phones, which basically is like we hear it at the same perceived loudness.

137

However, what changes is the intensity level across the frequencies.

138

So for example, when we are like at a very low frequency, say 20 hertz, we need a lot of intensity,

139

so 120 decibels to perceive that sound at 80 phones.

140

But then when we go up, say, to a thousand hertz,

141

then to perceive that sound at 80 hertz, we need way less intense of a sound, which is around 80 decibels.

142

And so, which basically means that, like, at lower frequencies, we perceive sounds,

143

like, as less loud if they're equal in intensity.

144

And this is like interesting.

145

And if you look at like where we are the most efficient with loudness here,

146

it's around like from a few hundred Hertz up until like 5,000, I would say 7,000 Hertz,

147

which is like more or less like the range of like human speaking and human like singing, which may not be like a case at all.

148

Right.

149

So we are kind of like tailored to be the most efficient at that frequency range.

150

And then like when the frequency goes up again,

151

so we still, yeah, we are less efficient like at hearing sound, at perceiving like the loudness.

152

And so we need more intensity to perceive like the same level of loudness.

153

Okay, so this is like extremely, extremely interesting.

154

And I'm always like fascinated about like the correlation between like physical measures and kind of like perceptual measures.

155

Like and the way like these two things, yeah, are connected together.

156

And now we enter like yet another aspect of sound, which is kind of like highly subjective and difficult to grasp.

157

And that's timbre.

158

okay so what's timbre? well I wish we knew

159

because like researchers interested in sound music

160

and all of these kind of things like I've studied timbre for a long time the problem is

161

that we don't have like a real and comprehensive definition of timbre

162

but we have like a few hints so

163

if you ask a musician what timbre is like they would probably tell you like is the color of sound

164

but that's a weird definition like for some something like which we hope like could be like a measurable thing right and but

165

if we talk about this like from a kind of like

166

more like programmatic like logical perspective we could think of like uh timbre as the diff between two sounds

167

which have like all aspects equal like in terms of like intensity frequency

168

and duration so you do a diff

169

and what remains like is uh sound there uh timbra sorry

170

it remains like timbra well uh the point is like

171

if you have like the the same uh like notes like c5 for example uh played on a trumpet trumpet

172

and on a violin with the same intensity you still hear

173

that there's a difference there uh

174

but it's not intensity it's not uh frequency it's not duration it's timbre okay so uh

175

and timbre is usually described with words like bright dark dull

176

harsh worm all of this like sounds a little bit fuzzy right

177

and that's because it is fuzzy

178

but now we don't know what our timbre is perfectly

179

but we have a clear idea that timbre is somewhat multi-dimensional

180

so many features come into place to define timbre this is different from other aspects of sound like frequency

181

or intensity where we just have like one uh like one measurable one observable there okay

182

so what are like some of these features

183

that come into place with timbre so here i've listed three which which are like the most important ones.

184

So one is called the sound envelope, then we have the harmonic contents, and then amplitude and frequency modulation.

185

Now, I'm quite sure that most of you are completely oblivious about like what these things are,

186

but don't worry, because I'm gonna cover like all of these things in depth, and we're gonna have like quite a lot of fun going through all of these aspects of sound.

187

Okay, so let's start with sound envelope, which is called auto amplitude envelope.

188

So what's that?

189

Well, when we think of a sound, definitely like this is true for like musical sounds,

190

like notes and things like that on different instruments.

191

So the sound usually has like an envelope, right?

192

And so this envelope can be divided with a model that's called the ADSR model, which stands for attack, decay, sustain, release model.

193

So basically the idea is that the amplitude of these sounds has an initial spike, and that's like the attack.

194

So for example, it's when you strike a key on a piano, you have like a spike in amplitude,

195

which also has some kind of like noisy sounds due to the hummers like of the piano.

196

And that's like a type of like transient sound.

197

Then you have like a decay, which is where the sound stabilizes,

198

a sustain period, which is where the the sound like remains like more

199

or less like constant in amplitude and then you have a release

200

which is like just like the fading out phase of a sound now the interesting thing is

201

that different types of like sounds have different envelopes

202

and this is like definitely true for musical instruments

203

so let's take a look at this once again I'm using

204

like this uh figure from the mueller's like book um fundamentals of music processing okay

205

so here we can uh like um compare two different envelopes for a piano sound

206

so here like you just like press a key

207

and you wait for it like to to just like decay and here you have a violin sound okay

208

and you see that the adsr

209

and the envelope of this two um sounds it's like very very different

210

so with the piano you have like a short attack a

211

transient here then you have like a little bit of a period of decay

212

and then you have a sustain and finally you have a release down here right

213

and this is different for for violin as you can appreciate

214

here the attack is way longer in terms of time and that's

215

because like the attack is less sharp okay then you really

216

don't have like a period of decay here you just have like some kind of like sustain

217

and then you have the release when uh yeah just like end the sound okay

218

so sound envelope is an important feature

219

that determines the um the sound uh okay so one thing

220

that i that i want to tell you here is

221

that for example

222

if you remove the attack part from a key sound from a from a piano sound what happens is

223

that it's difficult to recognize that

224

that is like a piano that's like playing right that's

225

because like we we associate like a piano sound with its like sharp transient quite a lot okay

226

so now let's move on

227

to the second aspect which is harmonic content

228

but in order to understand how what harmonic content is we

229

need to understand what a complex sound is made up of

230

so a complex sound is made up of a superposition of uh many sinus

231

or like fundamental uh like simple uh like sounds right so So we can think of like a partial as a sinusoid,

232

which is used to describe a sound.

233

So the many different sinusoid that we have,

234

which are superimposed together to create a complex sound are called partials or harmonic partials.

235

So the lowest partial is called the fundamental frequency.

236

And this is usually like the one that gives the pitch name to a note,

237

if we are talking about like a note, for example, musical note.

238

And now the harmonic partial,

239

so the harmonic partials are frequencies that are like an integer multiple of the fundamental frequency.

240

So let's say we have a fundamental frequency, which is at 440 Hertz, which is a four four now

241

if we if we uh take a look at the second partial there

242

so a harmonic partial what happens is that we have to multiply

243

that 440 by 2 and this gives us like 880 hertz now the third partial

244

will be 3 multiplied by 440

245

which is um 1320 hertz now you can move on

246

and basically the idea is that's like the complex sound is

247

going to made up of all of these like partials

248

and the harmonic content tells us how much energy we have in each of these uh partials

249

and that determines somehow the timbre of the sound.

250

Now not all sounds are perfectly harmonic

251

so there's a lot of inharmonicity in some sounds and

252

that would determine inharmonicity we indicate a deviation from a harmonic partial

253

so that's for example if we had like some frequencies that are not perfect harmonic partials of the fundamental frequency.

254

Now if we take a look at music instruments for example we know

255

that um pitch pitched instruments tend to be like harmonic whereas

256

like percussive instruments tend to have a lot of inharmonicity right

257

so and usually obviously like noise are are highly completely like inharmonic right cool

258

so this gives you like a an idea

259

but now let's try to like listen to some of these things

260

so for showing uh the harmonic content we should go

261

and check out like a jupiter notebook that i just like uh broached okay

262

so here i'm not gonna go through like the code

263

because like the the point here is not like to show you like some of these things

264

because we're gonna cover them like in future videos anyways but i want to to show you

265

like some sounds and then the relative spectrograms okay so yeah

266

so here just like i load some sounds

267

and here like i have like a nice function

268

that i can use to plot a spectrogram now what's a spectrogram

269

so you'll hear about this like a lot moving forward

270

but basically the idea is that with a spectrogram we get a snapshot of like the different energy in different frequencies

271

of a sound across the duration of that sound

272

and we can visualize that with a nice um plot

273

which is this one okay so now let's try to um listen to a sound

274

and so here what i want to show you like is two sounds

275

and we'll start with a violin sound uh which is at i believe at c4 okay

276

so that's middle c and so let's listen to this sound

277

okay yeah let's okay good so that's c4 on a um violin okay

278

so now let's plot the spectrogram for this

279

and here you have it okay in all of its glory

280

so basically the idea here is that on the x-axis you have time

281

measured usually in seconds on the y-axis you have hertz

282

and this is like a logarithmic scale

283

and here like each point here has a color

284

and the color tells you uh like the uh intensity of a um yeah of

285

that frequency at that like specific time and

286

so here like you have this color bar

287

that tells you like how to reach like the intensity based on color like the redder

288

and the more intense like the that frequency is okay

289

so as you can see here

290

so we we have uh a lot of like energy here in these like frequency band

291

which is around 256 precisely this is a 261

292

which i believe like should be like the hertz for c4 for the frequency for c4

293

and yes and here we expect a lot of energy

294

but then we have like the the partial the harmonic partial here

295

which is double that frequency which is a 512

296

and as you can see here we have like uh that coming in and then we multiply 256

297

61 by 3 and we have the third partial

298

and you can see it here the fourth the fifth cool it's super cool

299

and you can go up and up and up and up

300

and you still get like all of these partials okay and

301

so as you can see there's not much like in harmonicity here

302

so the partials like are very well uh define here okay

303

so now let's compare this with a piano sound okay

304

so now i have i believe like it's a c5 piano sound

305

so let's listen that okay so you you heard that

306

so this is c5 so it's an octave above the violin sound

307

that we just heard and so let's take a look at the

308

at the spectrum here for this piano sound and so as you can see here we have a lot of activity

309

around 512 and this is like a correct right because this is like the fundamental frequency that we expect at

310

311

and then over here you have like the first like harmonic

312

partial you go up the second the third the fourth

313

but as you can see the the higher you go and the less present like the partials are

314

and if you compare this like against this right you see the difference okay

315

so this is like way more sustained across

316

so the energy is more spread out across the whole like

317

partials whereas here like with the piano like is it seems to be like a little bit less

318

so right and so by looking at this you can say okay

319

so one of the reasons why like this two sounds are different is because of the

320

distribution of the energy across the different partials being different okay cool okay now don't worry like

321

if you don't have an idea about like what spectrograms are

322

or how we we got to them or anything like that

323

because this is gonna be a key topic

324

and we're gonna cover this like in a lot of detail

325

because like it's all about spectrograms

326

and believe me spectrograms are key to analyze sound okay well

327

i didn't want to go here i just like spoiled what's coming next

328

but let's move on okay

329

so now we have an idea about harmonic content now we have like one final aspect of timbre

330

so we said envelope harmonic content and now frequency and amplitude modulation

331

so basically modulation in the uh the sound like itself so what's frequency modulation so frequency modulation

332

is also called in music circles as vibrato right and basically here you have um periodic variations

333

in frequency and in music you have this for expressive purposes

334

but uh like we can create an effect a vibrato effect by using like frequency modulation

335

and this can be formalized and but basically the idea behind this is

336

that you start like with a signal

337

that you want to use to modulate the frequency of a a carrier signal

338

which is this one like in blue down in the middle

339

so you apply like this message signal signal on the carrier signal

340

and you get these results down below

341

and as you can see here that's a frequency modulation

342

because the frequency starts uh like this

343

but then the frequency kind of like um goes down okay

344

and then it goes up and then it goes down right

345

and so here we have a frequency modulation

346

so now you may be wondering but how does like

347

that sound like now if you're a musician obviously you know what a vibrato is

348

but if you're not i'm gonna show you and this is where

349

that amazing video comes in and

350

so here we have a violinist who explains how to play vibrato on the violin

351

so the first thing you'll hear is like the uh kind of like alternating like two notes right

352

and then after that you'll hear like how that can become a vibrato

353

yeah that's very dull right it's very very dull but

354

if you change the speed there you'll hear like the typical violin sound the vibrato sound let's hear

355

and until you reach normal vibrato speed

356

and you need to right that's super cool now every time like you listen to violins

357

or orchestras like the strings usually use like this vibrato sound quite uh intensively okay

358

so uh one thing

359

that i wanted to say is like as a keyboard player

360

unfortunately we don't have this luxury of having like a vibrato

361

but sometimes like you'll see pianists who like kind of like strike a key

362

and then after that they kind of try to do a vibrato obviously

363

that doesn't have like any effect on the sound

364

but still like you have that

365

so it's a it's like a nice like quirk okay but

366

if you have a midi keyboard uh midi keyboards have this thing

367

which is like after touch like

368

and after like you strike a key then you can like vibrate the sound

369

and it's gonna like vibrate but not on a piano i can assure you

370

that i've tried it doesn't work okay uh

371

so now the other type of like modulation

372

that we have is called amplitude modulation

373

and in musical terms this is called tremolo

374

and this is still like a periodic variation but it's not in frequency

375

but in amplitude and once again music we can use this like for expressive purposes okay

376

so how do we uh get to amplitude modulation basically it's the same idea

377

so we have a signal message signal we have a carrier signal

378

and we apply the signal on the carrier signal

379

but this time not on the frequency but on the amplitude

380

and so you you're gonna have like this effect that does like ah so you like

381

like bursts and release lack of amplitude, like at regular intervals.

382

Now, I want to show you this once again.

383

So let me just like go here.

384

So I have a sound down here that I want to show you with a tremolo, right?

385

Amplitude modulation.

386

So let's hear that.

387

But yeah, let's go back to zero.

388

Okay, so yeah, you get the idea.

389

So there like you have like this tremolo effect

390

and probably you are familiar with that because like you've heard it like in many different like musical pieces and whatnot.

391

Right, but basically the idea here is that when we modulate the sound like this

392

both like on the amplitude and frequency uh dimensions what happens is

393

that we we change the timbre of sound now we don't really change

394

that much like the perception like of the of the frequency

395

itself like of the amplitude is more like a quirk

396

that changes the time breath it's as if like these things were like localized

397

and had an effect like on time breath okay

398

so here you have it like all the different things like

399

that determine like timbre that we kind of like know of

400

or have like reconstructed

401

so timbre for sure is a multifactorial sound uh like dimension dimension of sound

402

and it has like three main things

403

so we already reviewed them say amplitude envelope the harmonic content

404

or distribution of energy across different partials

405

and then the signal modulation both like in frequency and amplitude okay

406

so by now you should have like a good idea of sound like

407

and all the different aspects of sound so we last time like we saw

408

that sound like is a mechanical wave

409

that propagates through air it is characterized by frequency like the intensity we can describe it with frequency intensity timbre

410

and obviously like some of these things are also like quite subjective like pitch loudness

411

and of course uh time breath okay

412

so now you have like this very nice introduction

413

and we are now ready to go to the next level

414

which is introducing audio signals

415

and specifically next time we'll are gonna tackle two very interesting

416

like topics well for for sure we are gonna introduce like all your signals

417

and then talk about audio to digital conversion called ADC the acronym

418

and digital to audio conversion or DAC which is kind of like the inverse

419

process okay so i hope like you've enjoyed this video

420

if that's the case remember to leave like

421

if you haven't subscribed yet please do so if you have any questions or doubts or anything

422

please like just like leave a comment in the section below

423

and talking about questions i just want to remind you

424

if you haven't or well if you if you are already here that's great

425

but if you are not please uh join the sound of ai slack community

426

so there you'll find a lot of people interested in like

427

audio processing machine learning ai music all this kind of stuff you can get feedback

428

and talk with other people

429

so i'll leave you a link to sign up to the slack community in the description below

430

so i hope like you really enjoyed this video it's all for today until the next time cheers

シャドーイングとは？英語上達に効果的な理由

シャドーイング（Shadowing）は、もともとプロの通訳者養成プログラムで開発された言語学習法で、多言語習得者として知られるDr. Alexander Arguelles によって広く普及されました。方法はシンプルですが非常に効果的：ネイティブスピーカーの英語を聞きながら、1〜2秒の遅延で声に出してすぐに繰り返す——まるで「影（shadow）」のように話者を追いかけます。文法ドリルや受動的なリスニングと異なり、シャドーイングは脳と口の筋肉が同時にリアルタイムで英語を処理・再現することを強制します。研究により、発音精度、抑揚、リズム、連音、リスニング力、そして会話の流暢さが大幅に向上することが確認されています。IELTSスピーキング対策や自然な英語コミュニケーションを目指す方に特におすすめです。

シャドーイング練習: Intensity, Loudness, and Timbre - 動画で英語スピーキングを学ぶ

この動画で身につける英語スピーキングスキル

聞き取りに注意すべき発音のポイント

ネイティブのように話すコツ

シャドーイングとは？英語上達に効果的な理由