Shadowing Practice: System Design: Why is Kafka fast? - Learn English Speaking with Video

Les maken...
1
Why is Kafka fast?
2
What is the secret?
3
We will talk about it in this video.
4
Let's dive right in.
5
We'll first start by acknowledging that the term fast is ambiguous.
6
What does it even mean that Kafka is fast?
7
Are we talking latency?
8
Are we talking throughput?
9
Is fast compared to what?
10
Kafka is optimized for high throughput.
11
It is designed to move a large number of records in a short amount of time.
12
Think of it as a very large pipe moving liquid.
13
The bigger the diameter of the pipe, the larger the volume of liquid that can move through it.
14
So when someone says Kafka is fast, they usually refer to Kafka's ability to move a lot of data efficiently.
15
What are some of the design decisions that help Kafka move a lot of data quickly?
16
There are many design decisions that contributed to Kafka's performance.
17
In this video, we'll focus on two.
18
We think these two carry the most weight.
19
The first one is Kafka's reliance on sequential I.O.
20
What is sequential I.O.?
21
Let's dig deeper into that.
22
There's a common misconception that disk access is slow compared to memory access.
23
But this largely depends on data access patterns.
24
There are two types of disk access patterns, random and sequential.
25
For hard drives, it takes time to physically move the arm to different locations on the magnetic disks.
26
This is what makes random access slow.
27
For sequential access though, since the arm doesn't need to jump around it is much faster to read
28
and write blocks of data one after the other kafka takes
29
advantage of this by using an append only log as its primary data structure
30
and the pen only log adds new data to the end
31
of the file this access pattern is sequential now let's bring
32
this idea home with some numbers on modern hardware with an
33
array of this hard disks sequential will write reach hundreds of megabytes per second,
34
while random writes are measured in hundreds of kilobytes per second.
35
Sequential access is several order of magnitude faster.
36
Using hard disks has its cost advantage too.
37
Compared to SSD, hard disks come as one-third of the price, but with about three times the capacity.
38
Giving Kafka a large pool of cheap disk space without any performance penalty, means that Kafka can cost-effectively retain messages for a long period of time,
39
a feature that was uncommon to messaging systems before Kafka.
40
The second design choice that gives Kafka its performance advantage is its focus on efficiency.
41
Kafka has moved a lot of data from network to disk and then from disk to network.
42
It is critically important to eliminate excess copy when moving pages and pages of data between the disk and the network.
43
This is where zero copy principle comes into the picture.
44
Modern Unix operating systems are highly optimized to transfer data from disk to network without copying data excessively.
45
Let's dive deeper to see how this is done.
46
First, we look at how Kafka sends a page of data on disk to the consumer
47
when zero copy is not used at all.
48
First, the data is loaded from disk to the OS cache.
49
Second, the data is copied from the OS cache into the Kafka application.
50
Third, the data is copied from Kafka to the socket buffer.
51
And fourth, the data is copied from the socket buffer to the network interface card buffer.
52
And finally, the data is sent over the network to the consumer.
53
Now this is clearly inefficient.
54
There are four copies and two system calls.
55
Now let's compare this to zero copy.
56
The first step is the same.
57
The data page is loaded from the disk to the OS cache.
58
With zero copy, the Kafka application uses a system call called send file to
59
tell the operating system to directly copy the data from the OS cache to the network interface card buffer.
60
In this optimized path, the only copy is from the OS cache into the network card buffer.
61
With a modern network card, this copying is done with DMA.
62
DMA stands for Direct Memory Access.
63
When DMA is used, the CPU is not involved, making it even more efficient.
64
To recap, sequential I.O and zero copy principle are the cornerstone to Kafka's high performance.
65
Kafka uses other techniques to squeeze every ounce of performance of modern hardware, but these two are the most important in our view.
66
If you'd like to learn more about system design, check out our books and weekly newsletter.
67
Please subscribe if you learned something new.
68
Thank you so much and we'll see you next time.
📺 Zelfde Kanaal
The Secret Sauce Behind NoSQL: LSM TreeC1
1

The Secret Sauce Behind NoSQL: LSM Tree

Business EnglishUniversity StudentsWorking Adults
System Design Was HARD - Until You Knew the Trade-OffsC1
3

System Design Was HARD - Until You Knew the Trade-Offs

Business EnglishUniversity StudentsWorking Adults
8 Most Important System Design Concepts You Should KnowC1
7

8 Most Important System Design Concepts You Should Know

Business EnglishUniversity StudentsWorking Adults
Everything You Need to Know About DNS: Crash Course System Design #4C1
3

Everything You Need to Know About DNS: Crash Course System Design #4

Business EnglishUniversity StudentsWorking Adults
CAP Theorem SimplifiedC1
2

CAP Theorem Simplified

Business EnglishUniversity StudentsWorking Adults
What Is REST API? Examples And How To Use It: Crash Course System Design #3C1
2

What Is REST API? Examples And How To Use It: Crash Course System Design #3

Business EnglishUniversity StudentsWorking Adults
System Design: Why is Kafka Popular?C1
2

System Design: Why is Kafka Popular?

Business EnglishUniversity StudentsWorking Adults
What is API Gateway?C1
2

What is API Gateway?

Business EnglishUniversity StudentsWorking Adults
10+ Key Memory & Storage Systems: Crash Course System Design #5C1
2

10+ Key Memory & Storage Systems: Crash Course System Design #5

University StudentsWorking Adults
What Is Redis Really About? Why Is It So Popular?C2
4

What Is Redis Really About? Why Is It So Popular?

Business EnglishUniversity StudentsWorking Adults
Kubernetes Explained in 6 Minutes | k8s ArchitectureC1
48

Kubernetes Explained in 6 Minutes | k8s Architecture

Business EnglishUniversity StudentsWorking Adults
What Are Microservices Really All About? (And When Not To Use It)C1
2

What Are Microservices Really All About? (And When Not To Use It)

Business EnglishUniversity StudentsWorking Adults
How Does a URL Shortener Work?C1
3

How Does a URL Shortener Work?

Business EnglishInterview EnglishUniversity Students
How Git Works: Explained in 4 MinutesB2

How Git Works: Explained in 4 Minutes

Business EnglishUniversity StudentsWorking Adults
✨ Aanbevolen Video
Psychology of Intelligence: Learn Faster Than Everyone Around YouC1
60
50 zinnen

Psychology of Intelligence: Learn Faster Than Everyone Around You

Business EnglishUniversity StudentsWorking Adults
Linus Torvalds Explains What Makes Linux So FastC1
32 zinnen

Linus Torvalds Explains What Makes Linux So Fast

Business EnglishUniversity StudentsWorking Adults
Trust the Process: How to Design Teams That Actually WorkC1
1
346 zinnen

Trust the Process: How to Design Teams That Actually Work

Business EnglishUniversity StudentsWorking Adults
Database Sharding! Designing Data-Intensive Applications chapter 6C1
1
689 zinnen

Database Sharding! Designing Data-Intensive Applications chapter 6

Business EnglishUniversity StudentsWorking Adults
Design a File Upload Service Like a Senior EngineerC1
1
114 zinnen

Design a File Upload Service Like a Senior Engineer

Business EnglishUniversity StudentsWorking Adults
🔥 Java Multithreading Rapid Fire | Can You Answer These Interview Questions? #shortsC1
3
26 zinnen

🔥 Java Multithreading Rapid Fire | Can You Answer These Interview Questions? #shorts

Business EnglishInterview EnglishUniversity Students
Amazon Athena explained in 120 seconds💡C1
15
29 zinnen

Amazon Athena explained in 120 seconds💡

Business EnglishUniversity StudentsWorking Adults
Crypto Agility Explained: Protect Data from Quantum Computing ThreatsC1
2
162 zinnen

Crypto Agility Explained: Protect Data from Quantum Computing Threats

Business EnglishUniversity StudentsWorking Adults
Unit Conversion & Significant Figures: Crash Course Chemistry #2C1
2
131 zinnen

Unit Conversion & Significant Figures: Crash Course Chemistry #2

Business EnglishUniversity StudentsWorking Adults
Data Ingestion Best Practices! How to Ingest Data Efficiently at Enterprise Scale!C1
1
133 zinnen

Data Ingestion Best Practices! How to Ingest Data Efficiently at Enterprise Scale!

Business EnglishUniversity StudentsWorking Adults

Who Is This Video For?

This video is perfect for intermediate to advanced English learners aiming to boost their technical vocabulary and listening skills. If you want to practice understanding complex explanations while improving your ability to discuss technology, this is a great fit. It’s also ideal for those using shadow speak techniques—repeating phrases aloud to mimic fluency—since the clear, deliberate pacing makes it easy to follow and replicate.

Words & Idioms Worth Stealing

  • Ambiguous: Unclear or having multiple meanings. Example: "The term 'fast' is ambiguous in tech discussions."
  • Throughput: The amount of data processed in a given time. The video explains Kafka’s focus on high throughput.
  • Append-only log: A data structure where new information is added to the end. Kafka uses this for speed.
  • Order of magnitude: A large difference in size or scale. "Sequential access is several orders of magnitude faster than random."
  • Zero copy: A method to transfer data without extra copying. "Kafka uses zero copy to move data efficiently."

How to Get the Accent Right

The speaker uses a neutral, technical tone with clear stress on key terms. Practice shadowing by pausing after phrases like, "Let's dive right in" and repeating them to match the rhythm. Notice how "latency" (LAY-tuhn-see) and "throughput" (THROO-poot) are stressed on the first syllable. For longer sentences, focus on linking words: "It is designed to move a large number of records" flows smoothly by connecting "designed to" and "a large number." Use a shadowing site to record yourself and compare—this helps refine your pacing and intonation. Remember, technical English relies on clarity, so mimic the speaker’s deliberate delivery to sound confident.

By focusing on these elements, you’ll not only learn tech vocabulary but also master the art of explaining complex ideas in English. Whether you’re using shadowspeak or practicing alone, this video is a powerful tool for improving your speaking and listening skills.

Wat is de Shadowing-techniek?

Shadowing is een wetenschappelijk onderbouwde taalleermethode die oorspronkelijk is ontwikkeld voor professionele tolkentraining en gepopulariseerd door polyglot Dr. Alexander Arguelles. De methode is eenvoudig maar krachtig: je luistert naar native Engelse audio en herhaalt het onmiddellijk hardop — als een schaduw die de spreker volgt met slechts 1–2 seconden vertraging. In tegenstelling tot passief luisteren of grammaticadrills, dwingt shadowing je hersenen en mondspieren om echte spraakpatronen tegelijkertijd te verwerken en te reproduceren. Onderzoek toont aan dat het de uitspraaknauwkeurigheid, intonatie, ritme, verbonden spraak, luisterbegrip en spreekvaardigheid aanzienlijk verbetert — waardoor het een van de meest effectieve methoden is voor IELTS Speaking-voorbereiding en echte Engelse communicatie.

Shadowing-techniek: lees de volledige stap-voor-stap-gids →