쉐도잉 연습: Kafka Deep Dive - Technical Discussions | English Speaking Practice | EnglishNotNull - 영상으로 영어 말하기 배우기

연습 모드 선택

각 모드는 전체 화면 연습실로 열립니다.

331 문장
문장이 너무 짧거나 길면 Edit를 눌러 조정하세요.
1
From clean code to clear communication, welcome to English Not Null, where you learn to present your work with confidence and precision.
2
Welcome to English Not Null.
3
I'm the guide, and today we're going to explore one of the most important technologies in distributed systems, Apache Kafka.
4
What is Kafka?
5
Why do 80 % of Fortune 100 companies use it?
6
And most importantly, how can you use this knowledge in your system design interviews?
7
In today's video, we'll follow a conversation between
8
Emma, a curious junior developer, and John, a senior engineer with years of experience.
9
Together they'll explore Kafka from the most basic concepts to advanced techniques.
10
What's special about this video is
11
that we'll learn through a real conversation you'll not only learn about kafka
12
but also improve your listening
13
and speaking skills through technical discussions throughout the conversation you'll learn what kafka is
14
and why it's important core concepts like producers consumers topics
15
and partitions how to use kafka in real world systems strategies for scaling
16
and handling failures and best practices that big tech companies are applying.
17
I encourage you to listen actively.
18
Pay attention to how
19
Emma asks questions and how John explains complex concepts using simple examples.
20
This is exactly how you'll need to communicate in international work environments.
21
Are you ready?
22
Let's begin our journey exploring Kafka with Emma and John.
23
Hey John, I've been hearing about Kafka everywhere wear lately, especially in system design discussions.
24
Can you help me understand what it actually is and why it's so important?
25
Oh, absolutely.
26
Kafka is a really fascinating piece of technology.
27
So, at a high level, Kafka is what we call an event streaming platform.
28
Think of it like a super -powered message queue that can handle massive amounts of data.
29
Okay, so it's like a queue, but what makes it special?
30
I mean, we have regular queues, right?
31
Great question.
32
Let me give you a concrete example that might help.
33
Imagine you're building a website that provides real -time updates for World Cup games.
34
Every time there's a goal, a booking, or a substitution, you need to update your website.
35
Right, so events are happening and we need to process them somehow.
36
Exactly.
37
So in Kafka terminology, we have what's called a producer.
38
That's the server or process that puts events onto the queue.
39
In our case, it could be someone with a laptop at each game sending updates.
40
And on the other side we have something reading those events?
41
Yes, that's called a consumer.
42
The consumer reads events from the queue and updates your website.
43
Now here's where it gets interesting.
44
What happens when FIFA suddenly decides to expand the World Cup from 48 teams to, let's say a thousand teams,
45
and all games start at the same time?
46
Oh, wow, that's a lot more events all at once.
47
I'm guessing our single server would struggle?
48
Exactly.
49
The server holding the queue would run out of space, and you know what the solution is, right?
50
It's something we hear about all the time in system design.
51
Scale horizontally?
52
Add more servers?
53
Perfect.
54
But now we have a new problem.
55
If we just randomly distribute events across multiple servers, we might end up with a mess.
56
A goal could be processed before the match even started, Or players could be booked for fouls they never committed.
57
Ah, I see the issue.
58
The ordering gets messed up when we split things across multiple servers.
59
Exactly.
60
And this is where Kafka introduces a key concept called partitioning.
61
Instead of randomly distributing events, we can distribute them based on something meaningful, like the game they're associated with.
62
So all events from the Brazil vs. Argentina game would go to one queue
63
and all events from the France game would go to another queue?
64
Bingo!
65
This way, we can guarantee that events from a single game are processed in order.
66
We can't guarantee that the 81st -minute goal from France happens after the 80th -minute goal from Argentina.
67
But that's okay, because what we really care about is the order within each game.
68
That makes sense.
69
So this partitioning concept is fundamental to how Kafka works?
70
Absolutely.
71
In Kafka, these individual queues are called partitions, and they're just append -only log files.
72
Think of them like a log file on disk where we keep adding messages to the end.
73
We have partitions, but what happens when we have so many events that even our consumer can't keep up?
74
Great question!
75
The logical solution.. is to add more consumers.
76
But if we just add more consumers without any coordination, we might have multiple consumers reading the same event.
77
Right, so we'd end up with duplicate processing.
78
That would be bad.
79
Exactly.
80
So Kafka introduces something called consumer groups.
81
When consumers are part of the same group, Kafka ensures that each event is only processed by one consumer in that group.
82
That's clever.
83
So it's like having a team of workers where each task is only given to one person.
84
Perfect analogy.
85
Now, let's say FIFA decides to expand to other sports.
86
Basketball, tennis, whatever.
87
We don't want our soccer website showing basketball events, right?
88
Of course not.
89
So we need some way to separate different types of events.
90
Exactly.
91
And that's where Kafka's topics come in.
92
Topics are just logical groupings of partitions.
93
So consumers can subscribe to specific topics.
94
Our soccer consumers only get soccer events.
95
Basketball consumers only get basketball events.
96
Okay, so let me make sure I understand the hierarchy.
97
We have topics which contain partitions, which contain the actual messages?
98
You've got it!
99
And here's an important distinction that confuses people.
100
Topics are logical groupings that exist in code, while partitions are physical.
101
They're the actual log files on disk.
102
Topics are for organizing your data.
103
Partitions are for scaling your data.
104
That's a really helpful way to think about it.
105
So when I'm using Kafka, I publish to a topic, but behind the scenes, it's actually going to a specific partition?
106
Exactly.
107
And how it chooses which partition is really important.
108
If your message has a key, we call it the partition key,
109
Kafka will hash that key and use the result to determine which partition to use.
110
So messages with the same key always go to the same partition?
111
Yes.
112
That's how we maintain ordering for related events.
113
In our soccer example, we might use the game ID as the partition key, so all events from the same game stay together and in order.
114
What happens if I don't specify a key?
115
Then Kafka will just distribute messages randomly across partitions.
116
This works fine if you don't care about ordering, but if you need related messages to be processed in sequence,
117
you definitely want to use a key.
118
This is really cool.
119
So, when would I actually use Kafka in a real system?
120
Are there specific scenarios where it shines?
121
Oh, there are tons of use cases.
122
The first big category is anytime you need a message queue.
123
For example, think about YouTube.
124
When you upload a video, they need to transcode it into different resolutions.
125
480p, 720p, 1080p, and so on.
126
Right, and that transcoding probably takes a while.
127
Exactly, and the user doesn't need to wait for it.
128
So they can put a message on Kafka saying transcode video ID 123 along with the S3 location of the video.
129
Then consumer services can pull these messages and do the transcoding asynchronously.
130
That makes sense.
131
The processing can happen in the background while the user continues using the site.
132
Exactly.
133
Another great use case is when you need in -order processing.
134
Think about Ticketmaster, when a really popular concert goes on sale.
135
They might put users in a waiting queue and only let people through in groups to avoid overwhelming the system.
136
Oh, so they'd put user requests on Kafka and then periodically pull batches of them off the queue?
137
Precisely.
138
And because Kafka maintains order within a partition, they can ensure fairness, first come, first served.
139
What about when you want to decouple producers and consumers so they can scale independently?
140
That's another perfect use case.
141
Imagine a coding competition platform like LeadCode.
142
You might have 100 ,000 users all submitting code at once during a contest.
143
So you'd want to scale up the servers handling submissions, but maybe not the servers actually running the code?
144
Exactly.
145
The submission servers can scale horizontally to handle the burst, put all the submissions on Kafka,
146
and then you can have just a few workers pulling submissions off the queue and executing them.
147
This way you can scale each part of your system independently based on its specific needs.
148
That's really smart.
149
What about streaming use cases?
150
I keep hearing about Kafka being used for real -time data processing.
151
Great question.
152
Streaming is when you need to process a continuous flow of data in real -time.
153
A classic example is ad -click tracking.
154
Every time someone clicks an ad, you put that event on Kafka
155
And then you have stream processors like Flink consuming those events to give advertisers real -time statistics.
156
So it's like having a fire hose of data that you need to process as it comes in.
157
Perfect analogy.
158
And another streaming pattern is PubSub.
159
where you want multiple consumers to process the same stream of events.
160
Think about Facebook live comments.
161
When someone comments on a live video, that comment needs to be sent to all viewers watching that stream.
162
So you'd put the comment on Kafka as a topic, and all the servers connected to viewers would subscribe to that topic?
163
Exactly.
164
Each server would see the comment and send it to their connected users, making it appear in real time across all viewers.
165
This is really powerful stuff.
166
But what about when things go wrong?
167
How does Kafka handle failures and ensure data isn't lost?
168
Great question.
169
Kafka has really strong durability guarantees.
170
Each partition has a leader replica that handles all reads and writes, plus multiple follower replicas that act as backups.
171
A leader goes down.
172
One of the followers can take over.
173
Exactly!
174
And there are configuration settings you can tune.
175
The replication factor determines how many copies of each partition you keep.
176
The default is three.
177
And the Axe setting determines how many replicas need to acknowledge a message before it's considered successfully written.
178
So there's a trade -off between durability and performance?
179
Exactly!
180
If you set
181
Axe to All, every replica must acknowledge the message, which is maximum durability, but slower.
182
If you set it to One, only the leader needs to acknowledge, which is faster, but less durable.
183
What about when consumers fail?
184
How does Kafka handle that?
185
Good question!
186
Consumers periodically commit their offset, basically telling Kafka, I've successfully processed message number X.
187
If a consumer crashes, it can restart and pick up from where it left off.
188
And if you have a consumer group and one consumer crashes…
189
Kafka will automatically rebalance the remaining consumers to cover the partitions that the crashed consumer was handling.
190
It's pretty seamless from the application's perspective.
191
That's impressive.
192
Now what about scaling?
193
What happens when my Kafka cluster needs to handle more data than it can manage?
194
Well, first you should do the math.
195
A single, well -configured broker can typically handle about 10 ,000 messages per second and store about a terabyte of data.
196
If your system stays within those bounds, you might not need to scale at all.
197
But if I do need to scale?
198
Then you add more brokers and, this is crucial, you need to choose your partition key carefully.
199
A bad partition key can create hot partitions where all your data goes to one partition and overwhelms it.
200
Can you give me an example of a bad partition key?
201
Sure.
202
Let's say you're building an ad click aggregator and you partition by ad ID.
203
If Nike launches a popular LeBron James ad, suddenly one partition gets overwhelmed with clicks while others sit idle.
204
So how would you fix that?
205
A few options.
206
First, you could remove the key entirely and distribute randomly.
207
This works if you don't need ordering.
208
Second, you could use a compound key, like addID colon random number 1 through 10, to spread hot ads across multiple partitions.
209
Third, you could implement backpressure to slow down producers when partitions get overwhelmed.
210
The compound key approach is interesting.
211
So you'd take that hot
212
Nike ad and spread it across 10 partitions instead of 1?
213
Exactly, though you'd lose strict ordering across those 10 partitions.
214
It's always a trade -off.
215
What about message sizes?
216
Are there any constraints I should be aware of?
217
Great question.
218
While there's no hard limit, it's highly recommended to keep messages under one megabyte for optimal performance.
219
A common mistake I see is people putting entire files on Kafka.
220
What's the right way to handle large files then?
221
Instead of putting the file itself on Kafka, store the file somewhere like S3 and put just the S3 URL on Kafka.
222
So your message might be video ID 123 and S3 URL.
223
The consumer can then download the file from S3 when needed.
224
That makes sense.
225
Keep Kafka for coordination and metadata, not for bulk data storage.
226
Exactly.
227
Now, what about error handling?
228
Kafka handles most reliability internally, but your application code might fail when getting messages into or out of Kafka.
229
So I need to handle retries on both the producer and consumer sides?
230
Right.
231
On the producer side, it's straightforward.
232
The Kafka client libraries have built -in retry logic.
233
You can configure how many times to retry and how long to wait between retries.
234
What about consumer retries?
235
That seems more complex.
236
You're right, it is.
237
Kafka doesn't have built -in consumer retry support, so you have to implement it yourself.
238
The common pattern is to use a retry topic.
239
How does that work?
240
Let's say you're building a web crawler.
241
You pull a URL from the main topic, try to fetch it, but the website is down.
242
Instead of losing that URL, you put it on a retry topic with a retry count.
243
Another consumer can pull from the retry topic and try again.
244
And if it keeps failing?
245
After a certain number of retries, you put it on a dead letter queue, basically a topic where failed messages go to die.
246
Engineers can look at these later to debug what went wrong.
247
That's a neat pattern.
248
What about performance?
249
If I'm processing real -time data, how can I make sure I'm getting maximum throughput?
250
Great question.
251
There are several techniques.
252
First, you can batch messages in the producer.
253
Instead of sending one message at a time, you can accumulate messages and send them in batches.
254
And I'm guessing you can also compress messages?
255
Exactly.
256
The producer can compress messages using algorithms like gzip before sending them.
257
Smaller messages mean more throughput.
258
But the biggest impact is still the partition key choice, right?
259
Absolutely.
260
Good partitioning ensures your load is evenly distributed across all brokers and partitions, maximizing parallelism.
261
It's always the first thing you should optimize.
262
One last question.
263
How long does Kafka keep messages around?
264
I assume it doesn't store them forever.
265
Good question.
266
This is controlled by the retention policy, which you can configure per topic.
267
There are two settings.
268
Retention .ms controls how long to keep messages.
269
The default is seven days.
270
And Retention .books controls the maximum size before old messages are deleted.
271
The default is one gigabyte.
272
So whichever limit is hit first triggers the cleanup?
273
Exactly.
274
And in some systems, you might need to store messages for much longer.
275
Maybe months for audit purposes or event replay.
276
You can configure longer retention, but just be aware of the storage costs.
277
This has been incredibly helpful, John.
278
Kafka seems like such a powerful tool for building scalable distributed systems.
279
It really is.
280
And the great thing is that nowadays, you don't even have to manage the infrastructure yourself.
281
Cloud providers offer managed Kafka services like AWS MSK and Confluent Cloud that handle most of the operational complexity for you.
282
But it's still important to understand these underlying concepts, especially for system design interviews.
283
Absolutely.
284
Understanding when to use Kafka, how to partition your data, and how to handle failures will really set you apart in interviews.
285
Plus, it's knowledge you'll use all the time in real world systems.
286
Thanks so much for walking through all of this with me.
287
I feel like I have a much better understanding of Kafka now,
288
from the basic concepts all the way to the advanced topics like scaling and error handling.
289
My pleasure.
290
Kafka is one of those technologies that seems complex at first.
291
But once you understand the core concepts, producers, consumers, topics, partitions, everything else builds on that foundation.
292
And remember, it's used by 80 % of Fortune 100 companies, so it's definitely worth understanding well.
293
I'll definitely be practicing with these concepts and thinking about how to apply them in different system design scenarios.
294
That's the perfect approach.
295
Try to think of Kafka not just as a message queue, but as a fundamental building block for distributed systems.
296
Whether you're building real -time analytics, event -driven architectures, or just need to decouple your services, Kafka is often the right tool for the job.
297
Perfect.
298
I'm excited to start incorporating these concepts into my system design practice.
299
Thanks again, John.
300
Anytime, Emma.
301
Good luck with your interviews.
302
And that was a very detailed conversation about Kafka between
303
Emma and John.
304
I hope you now have a comprehensive understanding of one of the most important technologies in distributed systems.
305
We've explored Kafka fundamentals like producers, consumers, topics, and partitions.
306
Why partitioning is important and how to choose partition keys.
307
Real -world use cases like message queues, streaming, and pubsub.
308
Strategies for scaling and handling failures.
309
As well as performance optimizations and error handling patterns.
310
One thing I want to emphasize, Kafka is not just a simple message queue.
311
It's a fundamental building block for building modern distributed systems.
312
From Netflix to Uber, from LinkedIn to Spotify, they all use Kafka to process millions of events per second.
313
For those preparing for system design interviews, remember that Kafka often appears in questions about real -time data processing,
314
event -driven architectures, microservices communication, and handling high -throughput systems.
315
Now, practice what we've just learned.
316
Try to explain Kafka concepts in English with friends or colleagues.
317
Draw diagrams and walk through different scenarios.
318
Remember, the best way to learn is by doing.
319
In the comments below, let me know which concept in today's video you found most interesting,
320
if you can think of any other use cases for Kafka, and what technology you'd like to learn about next.
321
Don't forget to subscribe to the channel so you don't miss our upcoming videos.
322
We'll continue exploring other technologies like Redis, Elasticsearch, and many more exciting topics.
323
And, especially, if you like this dialogue format, please like the video so I know.
324
I'll create more conversations between
325
Emma and John about different topics.
326
Finally, remember, technology is just a tool.
327
What's most important is the ability to communicate and collaborate effectively.
328
By improving your English skills through technical content, you're building two important skills at the same time.
329
Keep learning, keep practicing, and most importantly, keep coding.
330
I'll see you in the next video.
331
Until then, happy coding!

앱 다운로드

유창하게 말하는 데 필요한 모든 것

AI 발음 평가문장마다 점수 확인
IPA 발음 연습모든 소리 마스터
단어 학습나만의 단어장 만들기
단어 게임놀면서 외우기

같은 채널

FOR YOU

추천 영상

이 레슨의 어휘와 말하기 포인트

이 말하기 레슨은 영상 “Kafka Deep Dive - Technical Discussions”을(를) 바탕으로 합니다. 가장 자주 반복되는 단어는 다음과 같습니다: kafka, partition, message, consumer, event. 이 영상에는 섀도잉할 문장 331개와 단어 3237개가 있습니다. 말하는 구간의 길이는 20:09입니다. 화자는 분당 약 161단어로, 일상 대화에 가까운 자연스러운 속도로 말합니다. 영어에서 가장 많이 쓰이는 3,000단어에 속하는 단어가 78%뿐이라 어휘가 어려운 편입니다. 전체 331문장 중 53문장이 질문이어서 의문문 억양과 짧은 대답을 연습하기에 좋습니다.

이 영상의 핵심 어휘

영상에 나오는 익혀 둘 만한 단어 15개를 발음, 뜻과 함께 정리했습니다.

단어발음뜻
topic 명사/ˈtɑpɪk/화제, 주제
queue 명사/kju/줄, 대기줄
server 명사/ˈsɝvɚ/서버, 봉사기
producer 명사/pɹəˈdjuːsə/생산자, 제작자
explore 동사/ɪkˈsploɹ/탐험하다
soccer 명사/ˈsɑk.ɚ/축구, 풋볼
submission 명사/səbˈmɪʃən/복종, 굴복
guarantee 명사/ˌɡæɹ.ənˈtiː/보증
fundamental 형용사/ˌfʌn.dəˈmɛn.təl/기본적
default 명사/dɪˈfɔlt/디폴트
scenario 명사/sɪˈnɛəɹioʊ/각본, 대본
acknowledge 동사/əkˈnɑ.lɪd͡ʒ/승인하다, 인정하다
subscribe 동사/səbˈskɹaɪb/구독하다
broker 명사/ˈbɹoʊkɚ/중개인, 브로커
viewer 명사/ˈvju.ɚ/관람자

영상에 나오는 구동사

단어뜻
go down 동사내려가다

따라 말해 볼 만한 문장

영상에 나오는 짧고 완결된 문장으로, 일상 대화에서 그대로 쓸 수 있습니다:

  • I'm guessing our single server would struggle?
  • Right, so we'd end up with duplicate processing.
  • We have topics which contain partitions, which contain the actual messages?
  • What happens if I don't specify a key?
  • Are there any constraints I should be aware of?

이 영상의 문법

화자가 가장 많이 쓰는 문형을 영상 속 실제 표현과 함께 정리했습니다.

문형영상 속 표현
수동태 be + 과거분사 — 누가 하는지보다 무슨 일이 일어나는지에 초점could be processed · are processed · are called
현재완료 have/has + 과거분사 — 과거의 일이 지금도 관련이 있을 때I've successfully processed · have built · has been
관계절 who / which + 절 — 사람이나 사물에 대한 추가 정보message, which is · acknowledge, which is

주의할 발음

화자는 don't, I'm, you're 같은 축약형과 약화된 형태를 53번 사용합니다. 들리는 대로 짧게 발음하세요.

  • “th” 소리: math /mæθ/, algorithm /ˈælɡəɹɪðm̩/, throughput /ˈθɹuː.pʊt/
  • “sh”와 “zh” 소리: partition /pɑɹˈtɪ.ʃən/, submission /səbˈmɪʃən/, retention /ɹɪˈtɛnʃən/, coordination /kəʊˌɔːdɪˈneɪʃən/, crucial /ˈkɹuː.ʃəl/
  • 긴 단어 — 강세 위치에 주의: fundamental /ˌfʌn.dəˈmɛn.təl/, successfully /səkˈsɛs.fə.li/, architecture /ˈɑː.kɪˌtɛk.t͡ʃə/, communicate /kəˈmjuːnɪkeɪt/, importantly /ɪmˈpɔɹ.tənt.li/

한국어 화자가 어려워하는 소리:

  • /f/ — ㅍ(/p/)으로 바꾸지 말고 윗니를 아랫입술에 대기: fundamental /ˌfʌn.dəˈmɛn.təl/, default /dɪˈfɔlt/, successfully /səkˈsɛs.fə.li/, helpful /ˈhɛlp.fl̩/, format /ˈfɔːɹ.mæt/
  • /v/ — ㅂ(/b/)과 구별하기: server /ˈsɝvɚ/, viewer /ˈvju.ɚ/, impressive /ɪmˈpɹɛsɪv/, comprehensive /ˌkɑm.pɹəˈhɛn.sɪv/, actively /ˈæktɪvli/
  • /z/ — ㅈ이 아니라 성대를 울리는 /s/: nowadays /ˈnaʊ.ə.deɪz/, organize /ˈɔɹɡənaɪz/, hose /hoʊz/, emphasize /ˈɛm.fə.saɪz/

이 영상으로 연습하는 방법

  1. 먼저 말하지 않고 영상을 끝까지 듣고 모르는 단어를 적어 둡니다.
  2. 0.75배속으로 한 문장씩 섀도잉을 시작하고, 익숙해지면 보통 속도로 돌아갑니다.
  3. 자신의 목소리를 녹음해 원본과 비교하고, topic, queue, server 같은 단어에 특히 주의합니다.

쉐도잉이란? 영어 실력을 빠르게 키우는 과학적 방법

쉐도잉(Shadowing)은 원래 전문 통역사 훈련을 위해 개발된 언어 학습 기법으로, 다언어 학자인 Dr. Alexander Arguelles에 의해 대중화된 방법입니다. 핵심 원리는 간단하지만 매우 강력합니다: 원어민의 영어를 들으면서 1~2초의 짧은 지연으로 즉시 소리 내어 따라 말하는 것——마치 '그림자(shadow)'처럼 화자를 따라가는 것입니다. 문법 공부나 수동적인 청취와 달리, 쉐도잉은 뇌와 입 근육이 동시에 실시간으로 영어를 처리하고 재현하도록 훈련합니다. 연구에 따르면 이 방법은 발음 정확도, 억양, 리듬, 연음, 청취력, 말하기 유창성을 크게 향상시킵니다. IELTS 스피킹 준비와 자연스러운 영어 소통을 원하는 분들에게 특히 효과적입니다.

섀도잉 방법: 단계별 전체 가이드 읽기 →