Shadowing Practice: Analyze SEC filings with Transformers for fun and profit - Learn English Speaking with Video

Choose a practice mode

Each mode opens a full-screen practice room.

310 sentences

If sentences are too short or too long, click Edit to adjust them.

Hi everybody, this is Julian from Hugging Face.

Processing financial documents is a very strong use case for natural language processing and transformers.

In fact, most organizations in financial services and beyond need to work with a mountain of documents from company filings

to analyst reports and more in order to extract information, classify the documents,

run sentiment analysis and summarize documents etc etc so there are really

so many things you could do here

so in this video i'm going to work with company filings

that i download from the sec website and we'll use a pretty cool sdk

that i found to do this and then we'll process the filings

and we will run a few transformer models namely summarization

and emotion detection or sentiment analysis okay so let's get started so let's take a look at the raw material the documents

so starting on the sec website we go to filings

and we select edgar which is the name of the database

that includes company filings

and we could look for any particular company just to take a look maybe at one of those documents before

we start processing them so why not pick amazon

and so we see different filing types 10k

which are the annual reports 10q quarterly reports and 8k current reports

and we'll be able to work with all of these in our example

so let's maybe take a look at the latest quarterly report

the structure of this document is is well defined the table of contents is always the same

so we'll find lots of financial information then we'll find this section called management discussion

and analysis of financial condition and results of operation

and this is uh usually very interesting

because you get you know the management's opinion on how things are going

and and they highlight you know stuff that's going well stuff that's not going well

etc etc so for financial analysis this is pretty pretty interesting

and we'll we'll take a look at this section again

when we start uh processing documents okay so uh you can go

and have fun and find the information that you need here.

But of course, what we want to do is automate that process, download the documents, process them,

extract the text sections that we're interested in, and run transformer models on them.

And there is an API for Edgar documents, but as it turns out,

I found

recently i should say that some of my former colleagues at aws have actually built an sdk

that lets you easily retrieve scc filings and uh

and break down those filings into different sections

that we can save in a csv file and process so this is a huge time saver i mean

Personally, I'm not a big fan of text processing and regular expression,

so these people have done the heavy lifting for us.

So thank you very much, and I'll be happy to reuse what you've built.

They've written also a couple of blog posts on the SDK and how to use it for different use cases.

and the notebooks are actually integrated in SageMaker Jumpstart, which lets you easily deploy machine learning solutions on SageMaker.

So I'll put all the links in the video description and you can go and read about it.

I'll just show you how to use the SDK.

But yeah, thank you for building this.

It's quite useful.

And of course, this SDK is on GitHub and it's called the Jumpstart Industry SDK,

although it's only about financial documents for now.

But maybe we'll see other industries integrated there.

Okay.

All right.

So that's what we're going to do.

Grab some filings from the SEC website,

process them with this SDK so that we get a data frame with individual document sections in different columns,

and then we can apply some transformer magic to that.

Okay, so let's get started.

I've broken down my example in two notebooks.

So the first notebook is data prep.

Okay, so downloading, processing, and saving to CSV files.

And the second notebook is about running transformers.

It makes it a little easier to keep track of everything.

All right, here we go.

So I include for reference the name of the Jumpstart solution that I grabbed some code from, and it's called Dashboarding SCCText for Financial NLP.

And again, that's the SDK we're using.

So maybe let me zoom in a little bit.

So of course we need to install the SageMaker SDK and the Jumpstart Industry SDK.

Okay, so import them.

Create some objects that we're going to need, like an S3 bucket to download filings, the IAM role for permissions,

I mean the usual stuff.

Okay.

And next we can configure the downloading job, so to speak.

And of course, this is based on SageMaker.

And SageMaker users will easily guess that this is based on SageMaker processing,

which is an easy way to run batch jobs on SageMaker.

And it's really pretty easy to set up.

So we need a bucket to store the process documents.

So it's downloading the raw documents to one bucket, saving the process documents to another bucket.

And okay, this is the file name

that we're going to save the documents too

and the first step is to create this edgar dataset config object the first line here is

which tickers or company ids do we want to include so here i'm only fetching amazon documents

and these are the stock tickers right so

if you wanted google you'd say you know gog and

if you wanted tesla you'd have you know tsla etc etc and you can have as many as you want form types.

So here I'm grabbing 10

Ks and 10 Qs.

The SDK actually support other document types, and you'll find the information in the SDK, right?

The time range, so the start date, end date.

So then I'm grabbing old docs for 2019, 2020.

And this is just a bogus email that we need to retrieve documents.

You don't need a user account, by the way, to go and grab at your documents, although throttling is a thing,

so don't expect that you can pull 2 million documents in 5 minutes and not being throttled.

So you have to be considerate here.

Okay, so that's what we want to get.

Now that's the infrastructure setup, And this is where you can tell it's based on SageMaker processing.

So we'll run this on one ML C5 to Excel instance, which should be large enough.

No problem here.

And then we just launch that data loading job with the buckets,

100

the file name to save everything to.

101

And that's it.

102

Right?

103

And so you run this and, you know, I hidden the log.

104

It runs for a little bit.

105

It's based on PySpark, which SageMaker Processing supports.

106

And funny enough, the internal code name for this was Gecko.

107

And if you've seen the movie

108

Wall Street with Michael Douglas, you know why.

109

Okay, so good name for this.

110

Okay, and let's skip the log, which is really, really long.

111

And so at the end of that job, all I have is this CSV file in S3.

112

Okay, so I can go and grab it, read it, and this is what I get.

113

The ticker code, the form type, some numbers and dates, the text,

114

the full text of the filing, and the management discussion and analysis,

115

which is broken down as a separate section, because as I've said before, this is probably the most interesting part of the document.

116

Now, if we want to go one step further, we can actually break down this text column into individual sections.

117

So, going back to my example here, let's go to the contents.

118

Okay, we want to break down every single item here as a separate column, because we may want to run some analysis on that.

119

And this is where you really have to like the fact that it's all done for you, right?

120

Because you don't want to run those regular expressions.

121

And so they've included that code in a notebook, which is called SCC functions, whatever.

122

So I cleaned it up a bit because it included other stuff that I didn't need for visualization,

123

etc. But all that code is also a part of the SageMaker Jumpstart solutions.

124

So they extract all the items, and you can see the regex here is pretty intense.

125

Good thing I didn't have to write that.

126

Yeah, let's not spend too much time on that.

127

Basically, we'll just use those functions and map the column names to the right names,

128

etc. So it's all done here.

129

and this is the only bit that I wrote or adapted.

130

I just put together a simple function where I can pass a data frame with filings, the form type we want to process,

131

and then just do everything automatically, right?

132

And then it applies all that regexp and returns a data frame with one section in each column, okay?

133

So that's what we have for data prep

134

And the only thing we need to do, starting from this initial data frame, is this, which is pretty nice.

135

So just pass this data frame, say, hey, process all the 10K here.

136

And then it returns another data frame.

137

And now I can see in this data frame

138

that the individual sections of the filing have been broken down into individual columns,

139

which makes it easier to zoom in and examine and process particular things here,

140

except of that huge blob of text.

141

So I do the same for the 10Q filings.

142

So it's a different data frame, different names.

143

And then I just save those two process data frames to CSV files.

144

Simple enough.

145

Okay.

146

So now I've got my two files, right?

147

I could open one of them, I guess.

148

And we'll see exactly the same thing, right?

149

All nicely broken down into text sections.

150

Perfect.

151

So now we can move on to data processing.

152

And the original blog post runs some NLP functions on the text,

153

so you may want to read about that, although it feels more like traditional

154

statistics

155

and not there's no machine learning at work i think there's there's a bit of a hugging face summarization somewhere

156

but generally they're just using traditional techniques here but still

157

and that's part of the sdk so if you want to go

158

and compute those uh those scores those nlp scores yeah you can go

159

and do that but i've i've ignored that bit okay so i installed some dependencies

160

pandas of course i'm going to need nltk i'm going to need to break down text sections into individual sentences

161

and of course transformers for modeling okay fine

162

so this is the tokenizer i'm going to use to break down text into individual sentences

163

i'm loading my two data frames i should say

164

i'm loading my two files into data frames here

165

we just extracted very few documents to keep the demo very short you can go

166

and extract much more okay so i have two 10ks

167

because 2019 2020 and i have six 10qs

168

because there are three quarters in between the the 10ks right for each year makes sense

169

so um which model are we gonna use here so um

170

and this is where you can start adapting my code of course here

171

i just figured hey let's do a classification so i'm gonna analyze

172

uh individual sentences in the uh in the the management analysis section and i'm gonna keep

173

the most significant ones so the ones that are either highly positive or highly negative and ignore everything else.

174

And then I'm going to concatenate all the positive sentences and summarize them and concatenate all the negative sentences and summarize them.

175

But again, feel free to explore.

176

You have this data frame.

177

You can easily work with it.

178

So for summarization, I'm going to use the T5 model, T5 base.

179

And for classification, I've tried a couple of models.

180

I've tried FinBert, which is, as the name implies, a BERT version fine-tuned on financial documents.

181

And it can do sentiment analysis with three classes.

182

But I've found that this one is also very interesting.

183

It's a distilled BERT model fine-tuned on emotion detection.

184

And it has a wider range.

185

Anger, disgust, fear, joy, sadness, surprise.

186

And it does pick up interesting things.

187

We'll give it a go.

188

So just use a summarizer, use a classifier, and you can see I'm using the pipeline API from the Transformers library,

189

so it's really a one-liner to go and grab the model.

190

And here's how I'm going to process those docs.

191

So I have the first function called find emotional sentences where I pass text section,

192

and it's going to tokenize that text section into sentences, okay, break it down into individual sentences.

193

And then for each sentence, it's going to classify it.

194

And if it's not neutral, and if the score is actually higher than a certain threshold,

195

okay, so the really negative ones or the really positive ones or the really angry ones, maybe, I'm going to keep, okay?

196

And I just append them to a list stored in a dictionary, right?

197

So I'll have a list for positive sentences, a list for negative sentences, and so on.

198

So I'm just grouping those, okay?

199

And printing a summary and returning that dictionary.

200

So that will return a dictionary with a list of sentences for each emotion, right?

201

And then I have a summarization function where starting from that same dictionary,

202

I just join all the sentences for each emotion and I summarize them and print the summary, okay?

203

Just, you know, you could elaborate on that.

204

You could certainly improve this function here

205

because if you concatenate many sentences that actually exceed the max sequence size of the model,

206

then it gets truncated and you lose information.

207

So if you want to chunk this bit of text into,

208

let's say, 512 tokens, that would certainly yield a better summary.

209

Okay, exercise for the reader.

210

Okay, so that's what we do here.

211

So let's just try it.

212

And let's just start from...

213

We'll take one of those 10 Qs and point at the management discussion section.

214

And we'll try to find...

215

Let's run this.

216

We'll try to find the non-neutral sentences with a score higher than .95.

217

Okay?

218

So we can see even this section is already quite long, right?

219

over 40k characters and 174 sentences.

220

So that's a lot of text, right?

221

And I've got nine highly positive sentences and nine highly negative sentences.

222

Okay, so maybe we could just take a look at those, by the way.

223

So why don't we do this?

224

Let's look at the negative ones.

225

Whoops.

226

Here we go.

227

Okay, so yeah, these assumptions about future disposition of inventory are inherently uncertain.

228

Yeah, that's all good.

229

Liquidity is also affected by restricted cash balances, blah, blah, blah.

230

Changes in foreign currency exchange rates impacted net sales by 814 million.

231

Okay, so these are all pretty negative things, okay?

232

So now, as I've said before, we're going to concatenate all the negative sentences, all the positive sentences, and we're going to summarize.

233

So let's just go and run this.

234

And here I'm running this on a GPU instance in SageMaker Studio,

235

because that will just run faster.

236

Okay, and it's done.

237

So the positive summary is, as we utilize our federal net operating losses and tax credit, we expect cash paid for taxes to increase.

238

So you could argue that's actually positive.

239

Yeah, they're using credits, which is probably good, but taxes increase, right?

240

Nothing's free.

241

AWS sales increased 37%.

242

Okay, so I'm guessing that's good news.

243

And the negative summary is change in foreign exchange rates impacted sales.

244

Decrease in North America operated income is primarily due to increased marketing expenses.

245

Ha ha!

246

Marketing expenses.

247

Increase in international operating loss is due to increase in marketing expense at sales.

248

Okay.

249

All right.

250

So these are negative things.

251

And you can control the output, of course, with the minimum length and the maximum length of the summarizer.

252

Okay, so now let's try the emotion detection model and see what it picks up.

253

So just remove this one now.

254

Okay, just run this one, run this one.

255

Yes.

256

yeah this is a very high threshold with uh six or seven emotions so maybe i need to uh

257

lower it a little bit let's see what it picks up

258

yeah it didn't pick up much so let's just say maybe point i don't know 0.7

259

is there any angry sentence in here oh yeah all right oh

260

we have a lots of joyful sentences who thought

261

we could have thought

262

you know amazon reports would be joyful interesting uh yeah let's look at the angry ones now i'm curious

263

We are also currently subject to tax controversies in various jurisdictions.

264

Okay.

265

Developments in an audit investigation or other tax controversy could have a material effect on our operating results.

266

Okay.

267

I will refrain from any comment here.

268

But this is pretty funny, I think.

269

Okay, so why don't we summarize that stuff and see what happens?

270

Okay, so summarizing anger.

271

Yeah, we only have two sentences, so I'm guessing we'll see.

272

It does look like a summary.

273

Tax controversy could have a material impact on our results.

274

Yeah, it's actually not in here.

275

Yeah, okay, so that's a good summary.

276

yeah so joy um what do we have uh so fear and joy and sadness

277

okay

278

so see we have yeah the input for fear is all

279

is higher than the maximum token length for the model

280

so this is where you know we'd want to chunk that fear bit into uh 512 blocks and summarize

281

each one of them but you know i didn't go

282

that far okay yeah so joy again is probably too long to be properly summarized

283

and sadness gives us this guidance anticipates an unfavorable impact of approximately 30 basis points for foreign exchange rates okay

284

so that's yeah that's the good one that's a good one that's definitely not a happy fact okay

285

so as you can see you know you need to tweak this a little bit

286

and probably do a little more processing on the text um

287

and and and chunk it to get a better summary but this isn't too bad for just a few lines of code.

288

So that's pretty much what I wanted to show you today.

289

So if you thought it was crazy difficult to go and build a solution that analyzes SSD filings,

290

it's actually not.

291

a lot of the again a lot of the heavy lifting is done by this jumpstart SDK

292

and then you get your CSV file and you can just grab models

293

and these are off the shelf I didn't fine-tune anything here

294

and just start processing that stuff right

295

and of course you could you could even train your own language model on that right

296

if you download enough enough data about it yeah you could do I have to say it gets big really quickly.

297

I've downloaded, I think, a full year of 10Ks and 10Qs for all S&P 500 companies.

298

So that's about, yeah, that's a few thousand documents, and it's almost already a gigabyte of data.

299

So if you want to go back in time and download, I don't know, 20 years, it's going to be a long download,

300

which you probably need to break into

301

different jobs because you'll be throttled

302

and yeah it's going to be a lot of data

303

but yeah this data is out there you can process it very easily and

304

so you can build pretty cool models maybe we'll see more

305

financial models on the hogging face up thanks to this right and

306

if you need help ping me okay all right

307

i'll put all the links in the video description i hope this was informative and fun who knows for ICC filings.

308

I'll see you soon with more content.

309

Until next time, have fun, keep learning.

310

Bye bye!

Download App

Everything you need to speak fluently

AI PronunciationScore every sentence

IPA PracticeMaster every sound

VocabularyBuild your word bank

Vocab GameLearn while playing

Scan QR

Download on theApp Store Get it onGoogle Play

FOR YOU

Recommend Video

398 sentences

Vocabulary and speaking notes for this lesson

This speaking lesson is built on the video “Analyze SEC filings with Transformers for fun and profit”. The speaker keeps coming back to these words: document, sentence, filing, section, text. This video has 310 sentences and 3365 words to shadow. The speech runs for 25:46. The speaker talks at a steady 131 words per minute, a comfortable pace for shadowing. 82% of the words are among the 3,000 most common in English; the rest is worth studying before you start.

Key vocabulary in this video

15 words from the video worth learning, with pronunciation and meaning:

Word	Pronunciation	Meaning
filing noun	/ˈfaɪ.lɪŋ/	Any particle that has been removed by a file or similar implement; a shaving.
summary noun	/ˈsʌm.ə.ɹi/	An abstract or a condensed presentation of the substance of a body of material.
emotion noun	/ɪˈməʊ.ʃən/	A person's internal state of being and involuntary physiological response to an object or a situation, based on or tied to physical state and sensory data.
bucket noun	/ˈbʌkɪt/	A container made of rigid material, often with a handle, used to carry liquids or small items.
extract noun	/ˈɛkstɹækt/	Something that is extracted or drawn out.
notebook noun	/ˈnoʊtˌbʊk/	A book (physical or digital) in which notes or memoranda are written.
column noun	/ˈkɒləm/	A solid upright structure designed usually to support a larger structure above it, such as a roof or horizontal beam, but sometimes for decoration.
dictionary noun	/ˈdɪk.ʃəˌnɛ.ɹi/	A reference work listing words or names from one or more languages, usually ordered alphabetically, explaining each word's meanings or senses, oftentimes also…
tune noun	/tjuːn/	A melody.
controversy noun	/kənˈtɹɒv.ə.si/	A debate or discussion of opposing opinions; (generally) strife.
expense noun	/ɪkˈspɛns/	A spending or consuming, often a disbursement of funds.
detection noun	/dɪˈtɛk.ʃən/	The act or process of detecting, uncovering, or finding out, the discovery of something new, hidden, or disguised.
sentiment noun	/ˈsɛn.tɪ.mənt/	A general thought, feeling, or sense.
sadness noun	/ˈsædnəs/	The state or emotion of being sad.
chunk noun	/t͡ʃʌŋk/	A part of something that has been separated; a generally squat, thick, irregular piece of something, e.g. wood or stone.

Phrasal verbs you will hear

Word	Pronunciation	Meaning
break down verb		To stop functioning.
pick up verb		To lift; to grasp and raise.
face up verb		To confront an unpleasant situation
go back to verb		To stem from; to originate in.
put together verb		To assemble, construct, build, or formulate.
set up verb	/ˌsɛt ˈʌp/	To make ready for use.
turn out verb		To end up; to result.

Sentences worth repeating

Short, complete lines from the video that you can reuse in everyday conversation:

I've broken down my example in two notebooks.
Because you don't want to run those regular expressions.
Good thing I didn't have to write that.
We'll give it a go.
So why don't we do this?

Grammar in this video

The structures the speaker uses most, with the exact words from the video:

Structure	In the video
Present perfect have/has + past participle — a past action that still matters now	have actually built · have done · you've built
Passive voice be + past participle — the focus is on what happens, not who does it	are actually integrated · is based · being throttled
Relative clauses who / which + clause — extra information about a person or thing	edgar which is · log, which is · notebook, which is

Pronunciation to watch

The speaker uses 90 contractions and reduced forms, such as I'm, we'll, I've. Say them the short way, as you hear them.

The “th” sounds: threshold /ˈθɹɛʃ(h)oʊld/, throttle /ˈθɹɔ.tl̩/
The “sh” and “zh” sounds: emotion /ɪˈməʊ.ʃən/, dictionary /ˈdɪk.ʃəˌnɛ.ɹi/, detection /dɪˈtɛk.ʃən/, classification /ˌklæsɪfɪˈkeɪʃən/, threshold /ˈθɹɛʃ(h)oʊld/
Long words — get the stress right: dictionary /ˈdɪk.ʃəˌnɛ.ɹi/, controversy /kənˈtɹɒv.ə.si/, classification /ˌklæsɪfɪˈkeɪʃən/, primarily /ˈpɹaɪ.mə.ɹə.li/, infrastructure /ˈɪnfɹəˌstɹʌkt͡ʃə/

How to practise with this video

Listen to the whole video once without speaking and note the words you do not know.
Shadow it sentence by sentence at normal speed, repeating each one until your rhythm matches the speaker.
Record yourself and compare with the original, paying attention to words like filing, summary, emotion.

What is the Shadowing Technique?

Shadowing is a science-backed language learning technique originally developed for professional interpreter training and popularized by polyglot Dr. Alexander Arguelles. The method is simple but powerful: you listen to native English audio and immediately repeat it out loud — like a shadow following the speaker with just a 1–2 second delay. Unlike passive listening or grammar drills, shadowing forces your brain and mouth muscles to simultaneously process and reproduce real speech patterns. Research shows it significantly improves pronunciation accuracy, intonation, rhythm, connected speech, listening comprehension, and speaking fluency — making it one of the most effective methods for IELTS Speaking preparation and real-world English communication.

Shadowing technique: read the full step-by-step guide →

Shadowing Practice: Analyze SEC filings with Transformers for fun and profit - Learn English Speaking with Video

Choose a practice mode

Download App

Sec Growth DataScience staff meeting Sep 14 2022

Warren Buffett: How To Analyze a BALANCE SHEET

Amazon Athena explained in 120 seconds💡

Introduction to statistical prediction models

The Complete Guide to Hybrid Search in RAG (BM25 + Embeddings + Reranker)

What are Capital Markets? | Intro to Capital Markets (Part 1)

JUST $1 For Billions of Tokens?! (DeepSeek v4 Pro)

Executing Monitoring, Evaluation and Learning (MEAL) | Complete Practical Guide

Data Analysis Essentials in Excel

IAS 7 Statement of Cash Flows: Summary - applies in 2026

GATHERING IN THE DIGITAL ERA

The Anatomy of Fraud

Vocabulary and speaking notes for this lesson

Key vocabulary in this video

Phrasal verbs you will hear

Sentences worth repeating

Grammar in this video

Pronunciation to watch

How to practise with this video

What is the Shadowing Technique?