Shadowing Practice: Physics Informed Neural Networks (PINNs) [Physics Informed Machine Learning] - Learn English Speaking with Video

Choose a practice mode

Each mode opens a full-screen practice room.

400 sentences

If sentences are too short or too long, click Edit to adjust them.

Welcome back.

So, in this video, I'm going to introduce this idea of a physics -informed neural network,

or a PIN, which was first presented in this paper by Maziarraisi, Paris Peridicaris, and George Carniadakis.

And since its introduction, it's become one of the workhorse algorithms and kind of ideas in physics -informed machine learning.

So it's based on a kind of neural network idea

that also incorporates known physics in the form of a partial differential equation in the loss function.

So I'm going to walk you through this, some examples of how it's been extended, some kind of caveats and cautionary tales,

and some big success stories as well of what it can do.

So kind of a traditional approach to using a neural network to predict something like a fluid flow field,

you might build a neural network, some kind of, you know, deep feed -forward network that predicts the field variables of interest,

maybe u, v, w, and p.

These would be the x, the y, and the z velocity field components, and p might be the pressure of a fluid.

And those are functions of the spatial coordinates x, y, z, and time t.

This picture is just a two -dimensional fluid, but you can imagine if I wanted to model a three -dimensional fluid with three velocity components

and a pressure in terms of three spatial coordinates and time, I might be able to do that by building a big feed -forward neural network

and training it on a bunch of examples of flow data.

I might be able to learn that mapping from space and time to some coordinates u, v, w, and p.

Okay, this is kind of the naive approach to using a neural network to predict some physics.

And what these authors did in this kind of classic now PINS paper is they extended

this naive neural network picture where you have just a simple loss function

of your kind of predicted fields from your true measured fields.

They extend this by computing things like partial derivatives of these output variables in terms of the input spatial and temporal variables.

And this is a really simple but very powerful idea

that you can compute these terms essentially using the same kind of auto differentiation,

automatic differentiation, and back propagation that you would normally use to train a neural network,

you can use that same idea in a modern language like PyTorch

or Jax to compute these partial derivatives of the outputs with respect to the inputs.

And if you have those partial derivatives computed,

you can add in this additional loss function that your partial differential equation should be satisfied.

So if you know, for example, that this velocity field should be divergence -free,

you can add a term in purple here that is literally the divergence of your velocity field.

And if it's non -zero, it means that that physics is being violated.

So I have an example here of the Navier -Stokes equations, which is one of the most popular examples used with pins.

And again, just to reiterate,

what this big picture does is it takes a standard kind of neural network representation of the field variables

you want to predict or estimate, things like velocity fields and pressure fields as a function of space and time.

It takes that standard kind of generic neural network model

and it computes the partial derivatives that are relevant for the physics in this problem,

again, using automatic differentiation and back propagation, things like that.

And from those computed partial derivatives, you can create this additional loss term that tells you,

kind of, if this known physics is being violated or not.

Basically, you know, if there are things that should be zero,

like the divergence, or you could take the right -hand side of this equation and move it over to the left -hand,

so all of that adds up to zero, those would be terms in this purple virtual loss function.

And what that does is it adds this penalty when your neural network is violating known physics.

So this works really well for systems where we know something about the physics.

We know we're dealing with a fluid flow or a quantum system or an electromagnetic system.

And we're still trying to predict kind of our field variables as a function of space and time.

So it's a really, really cool idea.

And you'll notice that I have these sums labeled.

This is a sum over actual data.

So that's training data.

That's actual, you know, velocity fields that you have either measured or computed.

And then this purple sum is summed over virtual points.

So this is one of the cool things about this network, is if you know the physics,

if you know that you're looking at an Avier -Stokes fluid flow, then even if I only have pretty limited data here,

I can still test if my network is physical, meaning does it satisfy these equations?

Is this loss function small?

I can evaluate that on test points that are not actually in my training data.

So that's a huge and powerful idea also, is that oftentimes I've seen these pins trained on very small amounts of actual measurement data,

and you offload a lot of the training to these kind of virtual points where you can evaluate, is this network physical on those virtual points?

Because even on a virtual point where I don't have real measurement data, I can still evaluate if my network is,

you know, conserving mass and momentum.

Okay, pretty cool idea here.

And if we kind of go back to this big picture of physics -informed machine learning, it's pretty clear that PINs,

or this physics -informed neural network,

sits here in this kind of stage four of crafting a loss function that promotes what we know about the physics.

Now of course there's some architecture, you know, we're computing those partial derivatives, and we're modeling field variables as a function of space and time.

But I think the real kind of special aspect of PINs is here in this stage four crafting a loss function.

Okay, so back to this picture here.

Now a couple of things I want to point out.

There are reasons I like this architecture a lot, and there's things

that I think you should be aware are maybe a little

bit of like a cautionary tale about using an architecture like this. So, in general, I think this is a really clever idea

because it takes something that is a very simple and powerful idea in machine learning, this kind of big feed -forward neural network,

a big MLP, and it adds a really simple way of incorporating physics into that problem.

So this loss function here and the fact

that we're using kind of automatic differentiation to compute these quantities of interest that go into this virtual PDE loss function,

it's a very, very clever and also a very simple way of adding in known physics to this neural network process.

So I like that.

And I think that's probably one of the reasons that this has been so widely adopted, you know, cited thousands and thousands of times

and used by tons of researchers across the world is because it's a really simple and intuitive way of adding physics into,

you know, the standard way you would build a neural network to model physics.

Another thing I really like, and I mentioned this before, is that this really allows you to work with relatively small data sets.

You don't need necessarily that much data here for the actual database loss function.

You can use virtual points to evaluate if your network is physical or not.

And so I've seen lots of these networks trained with tons of virtual particles and not that much data.

And so I think that's a really, really powerful part of this as well.

Now the things that, this isn't, you know, exactly a negative, it's just a feature of the way that this is,

that this is developed, is that because the physics is added as a loss function, that's a strength because it's very intuitive and easy,

but it also only suggests that your physics is being satisfied.

These two terms are always going to be fighting with each other.

You're always going to be having this pull between your possibly noisy data measurements

and this kind of virtual term here that says that your PDE should be satisfied, your physics should be satisfied.

And so when you train this thing, it might be pretty stiff and difficult optimization,

and you're almost never going to actually have this purple term here be exactly zero.

So your physics is not being exactly satisfied by this pin.

Mass is not exactly being conserved.

Momentum is not exactly being conserved.

100

And you know, that may or may not become important in different applications.

101

In tons of applications, it is perfectly fine to have a small but non -zero physics loss,

102

and it still is an improvement over not adding that loss at all.

103

But in some applications, you might want to have a more exact enforcing of that physics,

104

maybe through an architectural choice or through an actual constrained optimization.

105

And that's something we've talked about a little bit before and we'll talk about again.

106

But I just want to point that out here.

107

Very simple and intuitive.

108

It allows you to work with relatively small data data sets, and those are both, you know, huge positives.

109

But the fact that this is a loss function means you're only suggesting that the physics is informed.

110

You're not enforcing that the physics is informed.

111

And so you should, or sorry, enforced.

112

And so you should just be aware of that.

113

I think that's super important.

114

And I want us to be thinking about what this kind of architecture is going to be good for.

115

Okay, so it's really, really good for estimating things like whole flow fields with fairly sparse sensor measurements.

116

It's a really good way of doing that, and it improves on some of the past methods that we used to do before pins existed.

117

I'll talk a little bit about that.

118

Data assimilation and flow estimates from limited or sparse measurements, that's a pretty cool application of this.

119

You can picture maybe you have a fusion reactor, some kind of SphereMAC fusion reactor,

120

and you can't put sensors everywhere inside of that reactor because they'll melt, so you maybe only have a few probes on the surface,

121

this could be a pretty cool architecture to take

122

that limited data on the surface and add in the physics loss

123

because we know that the physics is magnetohydrodynamics or some kind of plasma physics, and estimate the internal flow that's most consistent with those measurements

124

and with the fact that we know our physics is governed by this PDE.

125

So that's just one example.

126

Good.

127

Good.

128

It might not be the most useful thing in the world for speeding up your high fidelity simulations though.

129

That's something I think you should really be thinking about.

130

How can I use this architecture and what are ways that I maybe don't want to use this architecture?

131

Good.

132

And so actually one of the best and kind of most interesting papers

133

that was published around the same time as the original

134

Pins paper is this paper kind of, I think it's called something about hidden physics by many of the same authors in science,

135

so a very high profile paper, where they essentially show that if you have relatively limited data from a fluid flow,

136

I think this is some kind of a fluid flow in an artery or like a blood flow,

137

I don't know if this is called like a thrombosis, but it looks like there's some recirculation of blood flow here,

138

You can take very limited information and sometimes information that is

139

not even like a velocity field like a smoke visualization or some kind of like a movie and

140

you can infer what the actual velocity field the quantitative velocity field

141

that is kind of closest to satisfying the Navier -Stokes equations

142

and satisfying the measurement data that you have and you can see that you actually get relatively good reconstruction

143

of the kind of pins reconstruction and the ground truth reference velocity field.

144

And you can kind of see quantitatively here, you should download this paper and actually read it.

145

It's a very interesting paper, and it tells you how to take other types of measurements and infer these velocity fields.

146

But again, I think that's one of the things

147

that pins are really really good for is taking limited sensor data and and inferring between the lines,

148

kind of reading between the lines of what the velocity field

149

should look like to be consistent with your measurement data and to be consistent with your governing equations.

150

And it's funny, that's actually pretty close to what a Kalman filter does if you think about it.

151

It kind of optimally balances this idea between you want to agree with your data,

152

but you also want to agree with your model and depending on how much you trust your model or your data,

153

you might change this kind of hyperparameter that balances those two.

154

So there's also a hyperparameter in pins

155

that balances how much you trust your data versus how much

156

you kind of want to be enforcing your governing partial differential equation.

157

Now I'll point out because we're computing those derivatives, those partial derivatives that go into the kind of physics -informed loss function here,

158

this is going to have a harder time on systems that are, you know, discontinuous with things like shock waves or,

159

you know, chaotic convecting flows where this neural network function of time might be really diverging and really chaotic.

160

It's not going to probably be amazing at those kinds of flows.

161

So again, in the back of your mind, you have to be thinking this is not magic.

162

This is a neural network with a custom loss function.

163

And so you need to know when it is and is not going to work.

164

Okay.

165

But it, you know, does really, really cool things in certain circumstances for things like flow reconstruction from limited measurements.

166

Okay.

167

And in the past, we used to use a method called 4D var.

168

It was like kind of a variational method in space and time.

169

So that's where the 4D comes from.

170

And it was this horrific physics -based constrained optimization.

171

It was an optimization constrained by the physics to, again, find a flow field that was consistent with your measurements and with the physics.

172

All of that now in pins is abstracted from the human.

173

It uses modern machine learning, architectures and optimizations, and it typically does as well in a lot of cases.

174

For some of you who remember 4DVAR, you will see the parallel pretty quickly.

175

For those of you who don't,

176

maybe just do a quick Google on 4DVAR and you will see kind of morally how these are related and different.

177

Okay.

178

Now I'll put some links in the video description.

179

There's some great resources out there that I'm going to link to.

180

So a few really good videos here.

181

This is one by Juan Toscano and I believe this is Benjamin Mosley on this YouTube channel.

182

These are both really good kind of tutorial videos.

183

You know, this one, Juan actually works this training, how to train a physics -informed neural network out in code.

184

So I would highly recommend this one.

185

And there's some great tutorials online also for how to code this up in Torch.

186

Okay, so I'll put links to these in the video description.

187

But really it's super important for you to try this yourself.

188

I think that's one of the main points here is

189

if you want to know what this is good for and what this is not good for,

190

you've got to actually try it on a couple of examples and see, oh, did it take forever to train?

191

Was it fast to train?

192

Did it take forever to execute?

193

Was it fast to execute?

194

Did it generalize?

195

Did I need a bunch of data or a little data?

196

Those are all the things you would play around with.

197

And roughly speaking, these methods,

198

again, they're good for filling in lots of missing data or kind of inferring if you have very sparse measurement data.

199

They're good for speeding up the inference step.

200

So training might be expensive compared to something like CFD or fluid flow simulations.

201

And you in fact need the training data from those high fidelity sources.

202

But the inference step, once you have a trained model, can be very, very fast.

203

And so that's another good use of these methods.

204

And among these resources, again, this video was by Benjamin Mosley.

205

And they have a really, really nice kind of blog you should check out all about scientific machine learning.

206

And they have a really cool blog post specifically about pins with code on how to actually, you know, train these and test these yourself.

207

So I think this is one of my favorites to follow along.

208

And they contrast pins with the kind of classic naive neural network approach where you don't add that physics -informed loss.

209

And so I believe the example

210

that they train on here is a really simple one -dimensional

211

physics where you basically take a mass on a spring and you let that spring go,

212

or sorry, you let the mass go and you have this kind of decaying oscillation of your spring mass damper system.

213

And there is some pretty sparse training data, these blue points here.

214

And essentially, these are movies, so go to the website and actually, you know, check out the movies.

215

But essentially this shows the training of a naive neural network over a training step, and it basically never trains.

216

It goes into this kind of sub -optimal, kind of crummy solution, because it can't generalize past the blue data that it was trained on,

217

because it doesn't know anything about the physics.

218

And in contrast, if you have this kind of physics -informed loss function where you actually say, hey, I know that this is a,

219

you know, f equals ma spring system.

220

Now from those blue training points, you can train a physics -informed neural network that is much,

221

much better at generalizing into the future because it got the physics right, because that was in the loss function.

222

Okay.

223

So really simple example, but it's one you can code up yourself, you can follow, it all makes intuitive sense,

224

and it's a little easier than the PDE version.

225

Now I will point out, I noticed here that this is trained to 620 steps and this is trained to 12 ,000 steps.

226

So again, that should be making you wonder, hey, how much training do I really have to do to get this pin to converge?

227

But again, maybe the training is an offline computation.

228

Maybe you're willing to spend a lot of resources to train this thing as long as it learns the physics

229

and executes quickly in the future.

230

And again, this generalizes way better because it incorporates the physics.

231

That's kind of the upshot here.

232

Good.

233

And there's a lot of cool extensions to pins.

234

That's one of the things I like.

235

My lab and my collaborators, we develop a lot of methods, similar kind of machine learning and physics informed machine learning methods.

236

And when I think of my favorite methods, things like DMD and CINDY and pins and things like that,

237

they tend to be highly extensible.

238

They're such a simple idea

239

that you can often add a lot of kind of add -ons to make them more performant for different kinds of scenarios.

240

And so this is just a very, you know, select few examples out in the literature.

241

There are dozens of custom types of pins for different types of physics, for different types of data, different types of scenarios.

242

One of the ones I think is really cool are fractional pins for partial differential equations that have things like fractional derivatives.

243

What if I have the one half power diffusion term or some hyper diffusion term,

244

or I have an integral differential partial differential equation?

245

This happens all the time.

246

Like not everything is as simple as Navier Stokes, just simple mass and momentum conservation with easy integer power differential operators.

247

Oftentimes you have these nasty integral differential equations with fractional derivatives in your physics. And

248

so there's this really cool kind of split architecture here where your traditional PINs terms here can be handled using auto differentiation,

249

using kind of the automatic differentiation of the PyTorch or Jack's environment.

250

And then the other terms here, your fractional derivatives, your integral terms, things like that, those can be handled using your traditional numerical discretization schemes.

251

Things like finite difference, finite element, spectral methods, and you can kind of combine those in this fractional pin architecture.

252

So pretty cool idea and actually that broadens it out to a massive, massive class of PDs.

253

Another one I really, really like is this idea of delta pins.

254

And this is an idea that's been kind of floating around a lot.

255

I know some of my postdocs and collaborators talk about similar ideas where if you have a geometry,

256

like if I know that my physics is evolving on a sphere or on this spiral or on,

257

like let's say I look at the cochlea of an ear, I know that that's kind of this spiral shape.

258

So if I know the geometry kind of that determine the boundary conditions and where my PDE is living,

259

you can actually, you have a lot of prior information you can bake into the pins or other machine learning algorithms.

260

And so specifically, if you look at the Laplace Beltrami operator on that geometry,

261

its eigenfunctions give you a very good natural coordinate system in which to represent solution functions of that PDE.

262

So kind of the simplest example that we've all seen before is

263

that if I look at something like the heat equation on a rectangular geometry,

264

the eigenfunctions of the Laplace operator on that geometry are sines and cosines.

265

And that we know that the Fourier transform, those sines and cosines, are a good coordinate system that linearize or simplify that heat equation.

266

So it's very similar idea, but for more complex geometries,

267

that that eigenfunctions of the Laplace -Beltrami operator on an interesting geometry will be a very good coordinate system in

268

which to represent your solutions of your PDE.

269

This can be useful in things like cardiac flows.

270

If I take a weird, irregular geometry like my heart with all the veins and arteries,

271

again, I can look at the eigenfunctions of the Laplace -Beltrami operator,

272

and that gives me a good basis in which to represent and sum to approximate my PDE solution.

273

And so there's a lot going on under the hood here I'm not going to get into,

274

but this delta pins uses that idea to incorporate some idea of the geometry into the representation of the PDE,

275

and you get these improved representations.

276

Some kind of cool examples here.

277

This is a heat transfer problem where you have some heat convection and some geometry with specified boundary conditions.

278

And you can see that the regular pin doesn't agree with ground truth as well as this delta pin.

279

So having some knowledge of the geometry

280

and of the PDE physics lets you do better than just having one or the other.

281

This is a cool example where they use this method to compute geodesics on this kind of this geometry of a bunny.

282

Geodesics is just a fancy word for the shortest distance path on a manifold.

283

So, you know, if I look at the shortest distance path from my location to,

284

you know, Switzerland, it's the great circle that connects those two points on our sphere of Earth,

285

you know, if you believe that the Earth is in fact a sphere.

286

And so these geodesics are actually the solution of a certain partial differential equation.

287

I'm not going to get into that.

288

But so, you know, there is some PDE that governs geodesics that you can add as a loss function.

289

And this just shows...

290

that when you use this kind of geometry, these Laplace -Beltrami eigenfunctions for this geometry,

291

you can very accurately get those solutions for this geodesic problem.

292

It's a really cool example, I think, and it's kind of a clever example to benchmark their problem on.

293

And I don't really know exactly what this example is, but I think it's super cool that there is,

294

you know, a dragon in their paper, which I like.

295

Okay, good.

296

So those are just two of many, many, many examples of powerful extensions to this idea of pins.

297

Again, because it's such a simple idea.

298

You're just adding a loss function that is literally the PDE, how much it's being satisfied or not.

299

There's a lot of ways of extending this to other kind of more complicated systems.

300

But there are some caveats.

301

These don't always train that well.

302

Sometimes they're stiff.

303

Sometimes they overfit.

304

There's issues.

305

Anybody who has worked a lot with pins knows there's ways of massaging them to work and sometimes it doesn't work.

306

And that's like every method.

307

No method is a silver bullet.

308

That just doesn't exist.

309

But it's good to understand what are the limitations and actually study them in a very methodical and process -driven way.

310

And so that's one of the things I really,

311

really like about this paper out of Michael Mahoney's kind of group and collaboration on characterizing the actual failure modes in PINs.

312

And I'm not going to expect you to read all of this.

313

You should download the paper and check it out because they have open source code and you can play with it yourself.

314

But some of the main upshots here is that the main contributions as stated in this paper

315

is that they tried to train PINs on relatively simple but relevant problems,

316

things like convection, reaction, reaction diffusion, you know, simple PDEs.

317

And they basically found that a standard PIN often doesn't train and gives very large error except for,

318

you know, really, really simple parameter regimes of these PDEs.

319

Okay, so they basically set up a problem, a set of physics PDE problems,

320

where they can show under certain parameter regimes the method fails to train.

321

And then essentially they looked at adding how, what if you balance that PDE loss term,

322

so you can crank that PDE loss term up or down, that's a hyperparameter of how important the physics is or is not.

323

And they found that the loss function by increasing or decreasing

324

that PDE -based soft constraint actually makes it more complex to optimize and harder to find a good solution in some cases.

325

And I think this is a really careful paper.

326

I really like the work by Michael Mahoney and his colleagues because they are very, very careful in setting up this problem.

327

I don't think they're just criticizing pins.

328

They're actually trying to systematically understand when they don't work and fix the problem.

329

And so they look at, you know, the actual neural network itself without the physics informed loss.

330

And they do in fact find that it does have the capacity to represent the solutions.

331

So it's not a capacity issue, it's not the neural network's fault.

332

There's something about having this kind of dueling data and physics loss

333

that makes it stiff or difficult or prone to overfitting in some cases, not all cases, but some cases.

334

And again, it's easy to show when something doesn't work, but one of the cool things about this paper is

335

that they actually propose two concrete suggested paths to improve the PIN's training process.

336

One of them is through curriculum regularization, where they basically slowly increase that physics loss from zero to some larger value.

337

That's a very clever and kind of straightforward idea.

338

And then their second idea is posing the learning problem as a sequence -to -sequence learning task.

339

So two concrete ideas for how to actually fix these failure modes, and all of their code is available in PyTorch.

340

So a lot of work has gone into analyzing and understanding how to train these networks, when they will and will not train,

341

when they will generalize, and how to make them more robust.

342

So I think that's also really important to know, is that if you just design a vanilla pin,

343

it might work, but you might run into some issues

344

that other researchers have spent a lot of time and effort trying to diagnose.

345

There's another cool paper I like that essentially looks at the Pareto front.

346

They actually did a bunch of hyperparameter characterization where they have, this looks to me like a heat equation,

347

so this is some true velocity field, there is a predicted velocity field with a pin, and there's some amount of error.

348

This is just generic.

349

What they did was they looked at, if I change the hyperparameter,

350

the balance in my loss function between the data loss and the physics -informed loss,

351

so you can sweep through that hyperparameter and make one bigger or smaller, you can essentially get different error accuracies of your reconstructed fields.

352

And so they did an extensive test where, you know, on a relatively simple heat equation example,

353

but they did an extensive test where they looked at the different physics loss

354

and database loss for different example scenarios and initial conditions and data where they change,

355

they swept through the loss weight where, you know, if it's a small loss weight,

356

it's I forget if U is physics or F is data or vice versa.

357

But one of these regimes makes it so only your data matters, and the other regime makes it so that only your physics loss matters.

358

And you can see this kind of really interesting landscape of different amounts of loss here.

359

So in some cases, maybe your physics is actually being really well satisfied, but your data error is large,

360

or maybe your physics is actually not being satisfied very well, but your data loss is small,

361

and there's this clear balance between those two loss functions.

362

Again, you're never going to get a perfect loss where your physics is exactly satisfied

363

and your data is exactly fit if there's any error on your data.

364

Good.

365

And again, just a pretty extensive view of what happens to this relative error for different values of those hyperparameters,

366

and they give a lot of suggestions on how to choose and tune those hyperparameters.

367

So it's kind of also a practical guide.

368

You might want to read this before you train your PIN.

369

Okay, that was a mile -high overview of this very important and very widely used physics -informed neural network,

370

very often called PINs.

371

That's the acronym.

372

And again, just to recap, it is a really simple

373

and intuitive way of taking just a standard vanilla neural network

374

to predict some physical fields as a function of space

375

and time by using the automatic differentiability of these modern machine learning frameworks, PyTorch and jacks and so on,

376

to compute partial derivatives and add a custom loss function that is quantifying if the physics is or is not being satisfied.

377

So for example, if I know divergence should be zero for my velocity field,

378

I can literally add a loss term that is the norm of the divergence of these predicted velocity fields.

379

And I can compute those on arbitrary points, so I don't need to have this physics -informed loss trained on data.

380

I can train it on virtual points.

381

So that means I can use these pins to estimate flow fields and velocity fields, even if I have relatively sparse training data.

382

So it's a really cool idea.

383

It's very widely used.

384

It's highly extended.

385

There's lots of extensions to it.

386

But again, really important for you to remember, because it is a loss function, you're adding physics into the loss function,

387

you're adding the suggestion that your solution is physical.

388

You're not enforcing that your solution conserves mass or momentum or energy, you're just promoting that physicality through this loss term.

389

And again, sometimes it's pretty stiff, sometimes it doesn't generalize, sometimes it doesn't work for chaotic systems, like no method is perfect,

390

but it is a really simple and intuitive way of adding physics into your problem,

391

and I think that's probably one of the reasons it's been so widely adopted in the community.

392

And it's still, you know, especially at the sweet spot, machine learning people love it because it's easy for them.

393

Physics people love it because it's easy, you know, to add neural networks into their, you know, physics kind of workflows.

394

And so it's really at that sweet spot where it's kind of easy and pretty powerful.

395

Okay, more on this.

396

Definitely check out the resources in the description.

397

Try to code it up yourself.

398

Try to see where it does and doesn't work.

399

and we'll look at other methods soon.

400

All right, thank you.

Download App

Everything you need to speak fluently

AI PronunciationScore every sentence

IPA PracticeMaster every sound

VocabularyBuild your word bank

Vocab GameLearn while playing

Scan QR

Download on theApp Store Get it onGoogle Play

FOR YOU

Recommend Video

128 sentences

Vectors and 2D Motion: Crash Course Physics #4

Daily ConversationHigh School StudentsUniversity Students

68 sentences

Controlled Multi-modal Image Generation for Plant Growth Modeling

Daily ConversationUniversity Students

63 sentences

What Is AI? | Artificial Intelligence | What is Artificial Intelligence? | AI In 5 Mins |Simplilearn

Daily ConversationUniversity Students

78 sentences

Vanishing & Exploding Gradient explained | A problem resulting from backpropagation

Daily ConversationUniversity Students

125 sentences

Types of Proximity Sensors! #automation #engineering #science #robotics #electronics

Daily ConversationUniversity StudentsWorking Adults

Vocabulary and speaking notes for this lesson

This speaking lesson is built on the video “Physics Informed Neural Networks (PINNs)”. The speaker keeps coming back to these words: physics, loss, pin, network, function. This video has 400 sentences and 5622 words to shadow. The speech runs for 34:30. The speaker talks at a natural 163 words per minute, close to everyday conversation. Only 80% of the words are among the 3,000 most common in English, so the vocabulary is demanding.

Key vocabulary in this video

15 words from the video worth learning, with pronunciation and meaning:

Word	Pronunciation	Meaning
physics noun	/ˈfɪz.ɪks/	The branch of science concerned with the study of the properties and interactions of space, time, matter and energy.
pin noun	/pɪn/	A needle without an eye (usually) made of drawn-out steel wire with one end sharpened and the other flattened or rounded into a head, used for fastening.
neural adjective	/ˈnʊɹəl/	Of, or relating to the nerves, neurons or the nervous system.
velocity noun	/vəˈlɑsəti/	A vector quantity that denotes the rate of change of position with respect to time, combining speed with a directional component.
inform verb	/ɪnˈfɔɹm/	To communicate knowledge to.
equation noun	/ɪˈkweɪ.ʒən/	The act or process of equating two or more things, or the state of those things being equal (that is, identical).
partial adjective	/ˈpɑɹʃəl/	Existing as a part or portion; incomplete.
satisfy verb	/ˈsætɪsfaɪ/	To be enough (for)
geometry noun	/d͡ʒiˈɑ.mə.tɹi/	The branch of mathematics dealing with spatial relationships.
measurement noun	/ˈmɛʒ.ə.mənt/	The act of measuring.
derivative noun	/dɪˈɹɪvətɪv/	Something derived.
virtual adjective	/ˈvɜɹt͡ʃuəl/	In effect or essence, rather than in fact or reality; also, imitated, simulated.
architecture noun	/ˈɑː.kɪˌtɛk.t͡ʃə/	The art and science of designing and managing the construction of buildings and other structures, particularly if they are well proportioned and decorated.
fluid noun	/ˈflu.ɪd/	Any substance which can flow with relative ease, tends to assume the shape of its container, and obeys Bernoulli's principle; a liquid, gas or plasma.
differential adjective	[dɪfəˈɹənʃəɫ]	Of or pertaining to a difference.

Phrasal verbs you will hear

Word	Pronunciation	Meaning
check out verb		To record the departure or withdrawal of someone or something (such as guests, employees, books, etc.).
point out verb		To identify among a group of similar subjects, or in a scene where the subject might not be readily seen or noticed, with a gesture of the body.
set up verb	/ˌsɛt ˈʌp/	To make ready for use.
speed up verb	/spiːdˈʌp/	To accelerate; to increase speed.
add up to verb		To have a particular effect in combination.
follow along verb		To continue behind or beside an animal, person, vehicle or other object, especially as a companion.
go back to verb		To stem from; to originate in.

Sentences worth repeating

Short, complete lines from the video that you can reuse in everyday conversation:

You're not enforcing that the physics is informed.
I think that's super important.
I'll talk a little bit about that.
Did I need a bunch of data or a little data?
That's one of the things I like.

Grammar in this video

The structures the speaker uses most, with the exact words from the video:

Structure	In the video
“Used to” used to + verb — a past habit or state that is no longer true	used to do · used to use
Present perfect have/has + past participle — a past action that still matters now	it's been extended · I've seen · has been
Passive voice be + past participle — the focus is on what happens, not who does it	been extended · being violated · is summed
Relative clauses who / which + clause — extra information about a person or thing	PIN, which was · equations, which is · Anybody who has

Pronunciation to watch

The speaker uses 66 contractions and reduced forms, such as you're, don't, doesn't. Say them the short way, as you hear them.

The “sh” and “zh” sounds: equation /ɪˈkweɪ.ʒən/, partial /ˈpɑɹʃəl/, measurement /ˈmɛʒ.ə.mənt/, differential [dɪfəˈɹənʃəɫ], essentially /ɪˈsɛnʃəli/
Long words — get the stress right: velocity /vəˈlɑsəti/, geometry /d͡ʒiˈɑ.mə.tɹi/, derivative /dɪˈɹɪvətɪv/, architecture /ˈɑː.kɪˌtɛk.t͡ʃə/, differential [dɪfəˈɹənʃəɫ]

How to practise with this video

Listen to the whole video once without speaking and note the words you do not know.
Start at 0.75× speed, shadow it sentence by sentence, then go back to normal speed once it feels easy.
Record yourself and compare with the original, paying attention to words like physics, pin, neural.

What is the Shadowing Technique?

Shadowing is a science-backed language learning technique originally developed for professional interpreter training and popularized by polyglot Dr. Alexander Arguelles. The method is simple but powerful: you listen to native English audio and immediately repeat it out loud — like a shadow following the speaker with just a 1–2 second delay. Unlike passive listening or grammar drills, shadowing forces your brain and mouth muscles to simultaneously process and reproduce real speech patterns. Research shows it significantly improves pronunciation accuracy, intonation, rhythm, connected speech, listening comprehension, and speaking fluency — making it one of the most effective methods for IELTS Speaking preparation and real-world English communication.

Shadowing technique: read the full step-by-step guide →

Shadowing Practice: Physics Informed Neural Networks (PINNs) [Physics Informed Machine Learning] - Learn English Speaking with Video

Choose a practice mode

Download App

Vectors and 2D Motion: Crash Course Physics #4

Controlled Multi-modal Image Generation for Plant Growth Modeling

What Is AI? | Artificial Intelligence | What is Artificial Intelligence? | AI In 5 Mins |Simplilearn

Vanishing & Exploding Gradient explained | A problem resulting from backpropagation

The Feynman Technique Actually Works But You're Doing It Wrong

Introduction to Optoelectronics and Photonics

[TALK 13] Structure Prediction and Design using AlphaFold – Sami Chaaban

Naive Bayes

Neural Radiance Fields in 2 Minutes

Parallel Pump Operation

MIT Introduction to Deep Learning | 6.S191

Types of Proximity Sensors! #automation #engineering #science #robotics #electronics

Vocabulary and speaking notes for this lesson

Key vocabulary in this video

Phrasal verbs you will hear

Sentences worth repeating

Grammar in this video

Pronunciation to watch

How to practise with this video

What is the Shadowing Technique?