Pratique du Shadowing: System Design: Why is Kafka fast? - Apprendre l'anglais à l'oral avec la vidéo
Choisissez un mode d'entraînement
Chaque mode ouvre une salle d'entraînement en plein écran.
68 phrases
Si les phrases sont trop courtes ou trop longues, cliquez sur Edit pour les ajuster.
1
Why is Kafka fast?
2
What is the secret?
3
We will talk about it in this video.
4
Let's dive right in.
5
We'll first start by acknowledging that the term fast is ambiguous.
6
What does it even mean that Kafka is fast?
7
Are we talking latency?
8
Are we talking throughput?
9
Is fast compared to what?
10
Kafka is optimized for high throughput.
11
It is designed to move a large number of records in a short amount of time.
12
Think of it as a very large pipe moving liquid.
13
The bigger the diameter of the pipe, the larger the volume of liquid that can move through it.
14
So when someone says Kafka is fast, they usually refer to Kafka's ability to move a lot of data efficiently.
15
What are some of the design decisions that help Kafka move a lot of data quickly?
16
There are many design decisions that contributed to Kafka's performance.
17
In this video, we'll focus on two.
18
We think these two carry the most weight.
19
The first one is Kafka's reliance on sequential I.O.
20
What is sequential I.O.?
21
Let's dig deeper into that.
22
There's a common misconception that disk access is slow compared to memory access.
23
But this largely depends on data access patterns.
24
There are two types of disk access patterns, random and sequential.
25
For hard drives, it takes time to physically move the arm to different locations on the magnetic disks.
26
This is what makes random access slow.
27
For sequential access though, since the arm doesn't need to jump around it is much faster to read
28
and write blocks of data one after the other kafka takes
29
advantage of this by using an append only log as its primary data structure
30
and the pen only log adds new data to the end
31
of the file this access pattern is sequential now let's bring
32
this idea home with some numbers on modern hardware with an
33
array of this hard disks sequential will write reach hundreds of megabytes per second,
34
while random writes are measured in hundreds of kilobytes per second.
35
Sequential access is several order of magnitude faster.
36
Using hard disks has its cost advantage too.
37
Compared to SSD, hard disks come as one-third of the price, but with about three times the capacity.
38
Giving Kafka a large pool of cheap disk space without any performance penalty, means that Kafka can cost-effectively retain messages for a long period of time,
39
a feature that was uncommon to messaging systems before Kafka.
40
The second design choice that gives Kafka its performance advantage is its focus on efficiency.
41
Kafka has moved a lot of data from network to disk and then from disk to network.
42
It is critically important to eliminate excess copy when moving pages and pages of data between the disk and the network.
43
This is where zero copy principle comes into the picture.
44
Modern Unix operating systems are highly optimized to transfer data from disk to network without copying data excessively.
45
Let's dive deeper to see how this is done.
46
First, we look at how Kafka sends a page of data on disk to the consumer
47
when zero copy is not used at all.
48
First, the data is loaded from disk to the OS cache.
49
Second, the data is copied from the OS cache into the Kafka application.
50
Third, the data is copied from Kafka to the socket buffer.
51
And fourth, the data is copied from the socket buffer to the network interface card buffer.
52
And finally, the data is sent over the network to the consumer.
53
Now this is clearly inefficient.
54
There are four copies and two system calls.
55
Now let's compare this to zero copy.
56
The first step is the same.
57
The data page is loaded from the disk to the OS cache.
58
With zero copy, the Kafka application uses a system call called send file to
59
tell the operating system to directly copy the data from the OS cache to the network interface card buffer.
60
In this optimized path, the only copy is from the OS cache into the network card buffer.
61
With a modern network card, this copying is done with DMA.
62
DMA stands for Direct Memory Access.
63
When DMA is used, the CPU is not involved, making it even more efficient.
64
To recap, sequential I.O and zero copy principle are the cornerstone to Kafka's high performance.
65
Kafka uses other techniques to squeeze every ounce of performance of modern hardware, but these two are the most important in our view.
66
If you'd like to learn more about system design, check out our books and weekly newsletter.
67
Please subscribe if you learned something new.
68
Thank you so much and we'll see you next time.
Télécharger l'application
Tout pour parler couramment
Prononciation IAChaque phrase notée
Phonétique (IPA)Maîtrisez chaque son
VocabulaireEnrichissez votre lexique
Jeu de vocabulaireApprenez en jouant
Même chaîne
C11
The Secret Sauce Behind NoSQL: LSM Tree
Daily ConversationUniversity StudentsWorking Adults
C13
Proxy vs Reverse Proxy (Real-world Examples)
Daily ConversationUniversity StudentsWorking Adults
C12
Cache Systems Every Developer Should Know
Daily ConversationUniversity StudentsWorking Adults
C12
Top 5 Redis Use Cases
Daily ConversationUniversity Students
C13
System Design Was HARD - Until You Knew the Trade-Offs
Daily ConversationUniversity StudentsWorking Adults
C17
8 Most Important System Design Concepts You Should Know
Daily ConversationUniversity StudentsWorking Adults
C13
Everything You Need to Know About DNS: Crash Course System Design #4
Daily ConversationUniversity StudentsWorking Adults
C12
CAP Theorem Simplified
Daily ConversationUniversity StudentsWorking Adults
C12
What Is REST API? Examples And How To Use It: Crash Course System Design #3
Daily ConversationUniversity StudentsWorking Adults
C12
System Design: Why is Kafka Popular?
Business EnglishUniversity StudentsWorking Adults
C16
What Are Microservices Really All About? (And When Not To Use It)
Business EnglishUniversity StudentsWorking Adults
C150
Kubernetes Explained in 6 Minutes | k8s Architecture
Business EnglishUniversity StudentsWorking Adults
C14
What is API Gateway?
Business EnglishUniversity StudentsWorking Adults
C12
10+ Key Memory & Storage Systems: Crash Course System Design #5
University StudentsWorking Adults
C24
What Is Redis Really About? Why Is It So Popular?
Business EnglishUniversity StudentsWorking Adults
C13
How Does a URL Shortener Work?
Business EnglishInterview EnglishUniversity Students
B2How Git Works: Explained in 4 Minutes
Business EnglishUniversity StudentsWorking Adults
FOR YOU
Vidéo recommandée
C11
120 phrases
Learn Data Modeling in 8 minutes: Dimensional Data Modeling, Data Vault, and One Big Table
Business EnglishUniversity StudentsWorking Adults
C110
227 phrases
Psychology of Intelligence: Learn Anything So Fast It's Almost Unfair
Business EnglishUniversity StudentsWorking Adults
C12
32 phrases
Rust in 100 Seconds
American AccentBusiness EnglishUniversity StudentsWorking Adults
C12
42 phrases
System Design: How to Build an API Rate Limiter
Business EnglishUniversity StudentsWorking Adults
C12
243 phrases
Proximity Search & Geospatial Indexes Explained
Business EnglishUniversity StudentsWorking Adults
C12
11 phrases
What are Design Patterns? | Introduction to Design Patterns and Principles | Geekific
Business EnglishUniversity StudentsWorking Adults
C11
80 phrases
The Most Confused Concepts in Engineering
Business EnglishUniversity StudentsWorking Adults
C13
96 phrases
Build a Real-Time Leaderboard on AWS with DynamoDB
Business EnglishUniversity StudentsWorking Adults
C12
356 phrases
Concurrency in Low-level Design Interviews w/ Staff Engineer
Business EnglishInterview EnglishUniversity Students
C12
195 phrases
JavaScript Visualized - Event Loop, Web APIs, (Micro)task Queue
Business EnglishUniversity StudentsWorking Adults
Vocabulaire et conseils d’expression pour cette leçon
Cette vidéo contient 68 phrases et 782 mots à répéter en shadowing. La partie parlée dure 4:56. Le locuteur parle à un rythme naturel d’environ 159 mots par minute, proche d’une conversation courante. Seuls 80 % des mots font partie des 3 000 mots les plus courants en anglais, le vocabulaire est donc exigeant.
Vocabulaire clé de cette vidéo
15 mots de la vidéo à retenir, avec leur prononciation et leur sens :
| Mot | Prononciation | Sens |
|---|---|---|
| disk nom | /dɪsk/ | disque |
| buffer nom | /ˈbʌfəɹ/ | solution tampon |
| cache nom | /kæʃ/ | cache, mémoire cache |
| compare verbe | /kəmˈpɛɚ/ | comparer |
| liquid nom | /ˈlɪk.wɪd/ | liquide |
| principle nom | /ˈpɹɪn.sɪ.pəl/ | principe |
| hardware nom | /ˈhɑɹdˌwɛɹ/ | équipement |
| pipe nom | /ˈpaɪp/ | cornemuse |
| dive verbe | /daɪv/ | nager, plonger |
| interface nom | /ˈɪn(t)ɚˌfeɪs/ | interface |
| socket nom | /ˈsɑkɪt/ | prise, douille |
| technique nom | /tɛkˈniːk/ | technique |
| penalty nom | /ˈpɛn.əl.ti/ | penalisation |
| refer verbe | /ɹɪˈfɜː/ | référer |
| efficient adjectif | /ɪˈfɪʃənt/ | efficace |
Les verbes à particule que vous entendrez
| Mot | Sens |
|---|---|
| check out verbe | mater, regarder |
Prononciation à surveiller
Le locuteur utilise 5 contractions et formes réduites, comme we'll, doesn't, you'd. Prononcez-les sous leur forme courte, telles que vous les entendez.
- Les sons « sh » et « zh »: cache /kæʃ/, efficient /ɪˈfɪʃənt/, efficiency /ɪˈfɪʃ.ən.si/, inefficient /ˌɪn.ɪˈfɪʃ.ənt/, misconception /ˌmɪs.kənˈsɛp.ʃən/
- Mots longs — placez bien l’accent: efficiency /ɪˈfɪʃ.ən.si/, eliminate /ɪˈlɪm.ɪ.neɪt/, diameter /daɪˈæmɪdɚ/, critically /ˈkɹɪt.ɪ.kə.li/, ambiguous /æmˈbɪɡ.ju.əs/
Comment s’entraîner avec cette vidéo
- Écoutez la vidéo en entier une fois sans parler et notez les mots que vous ne connaissez pas.
- Commencez à la vitesse 0,75×, répétez phrase par phrase, puis revenez à la vitesse normale quand cela devient facile.
- Enregistrez-vous et comparez avec l’original, en faisant attention à des mots comme disk, buffer, cache.
Qu'est-ce que la technique du Shadowing ?
Le Shadowing est une technique d'apprentissage des langues fondée sur la science, développée à l'origine pour la formation des interprètes professionnels. Le principe est simple mais puissant : vous écoutez de l'anglais natif et le répétez immédiatement à voix haute — comme une ombre suivant le locuteur avec un décalage de 1 à 2 secondes. Les recherches montrent une amélioration significative de la précision de la prononciation, de l'intonation, du rythme, des liaisons, de la compréhension orale et de la fluidité.