Shadowing Practice: The Secret Sauce Behind NoSQL: LSM Tree - Learn English Speaking with Video
בחר מצב תרגול
כל מצב פותח חדר תרגול במסך מלא.
77 משפטים
אם המשפטים קצרים או ארוכים מדי, לחץ על ערוך כדי להתאים אותם.
1
NoSQL databases like Cassandra have exploded in popularity in recent years.
2
One driver is the insatiable demand to ingest huge amounts of
3
data from an ever-growing number of sources like mobile application and IoT devices.
4
The secret sauce behind many of these NoSQL databases is a data structure called Log Structure Merge Tree.
5
An LSM tree is optimized for fast writes.
6
To understand how an LSM tree works, let's first take a look at how data is typically stored in a relational database.
7
A relational database is commonly backed by a data structure called a B-tree.
8
A B-tree is optimized for reads.
9
Updating the B-tree is relatively expensive as it involves random I.O and might include updating multiple pages on disk.
10
This limits how fast a B tree can ingest data.
11
An LSM tree works differently.
12
Writes are bashed in memory as they arrive in a structure called a mem table.
13
A mem table is ordered by object key and is usually implemented as a balanced binary tree.
14
As a mem table reaches a certain size, it is flushed to disk as an immutable sorted string table.
15
An ss table stores the key value pairs in a sorted sequence.
16
These writes are all sequential I.O.
17
They are fast on any storage media.
18
The new ss table becomes the most recent segment of the LSM tree.
19
As more data comes in, more and more of these immutable SS tables are created and added to the LSM tree,
20
with each one representing a small chronological subset of the incoming changes.
21
Since SS tables are immutable, an update to an existing object key does not override an existing table.
22
Instead, a new entry is added to the most recent SS table, which supersedes any entries in the old SS tables for the object key.
23
Deleting an object requires special handling too, since we cannot mark anything in the SS table as deleted.
24
To perform a delete, it adds a marker called a tombstone to the most recent SS table for the object key.
25
When we encounter the tombstone on read, we know the object has been deleted.
26
And yes, it is a bit unintuitive that a delete takes up extra space.
27
To serve a read request, we first try to find the key in the mem table, then in the most recent SS table in the LSM tree,
28
then the next SS table, and so on.
29
Since an SS table is sorted, the lookup can be done efficiently.
30
The accumulation of SS tables presents two issues.
31
As the number of SSTables grows, it will take an increasingly long time to look up a key.
32
And as the SSTables accumulate, there are more and more outdated entries as keys are updated and tombstones are added.
33
These take up precious disk space.
34
To fix these issues, there is a periodic merging and compaction process running in the background
35
to merge SSTables and discard outdated or deleted values.
36
This reclaims this space and caps the number of SSTables a read has to look through.
37
Since the tables are sorted, this merging and compaction process is simple and efficient.
38
The approach is similar to the one used in the merge phase of the merge sort algorithm.
39
This, in a nutshell, is how an LSM tree provides fast writes.
40
Let's recap.
41
An LSM tree buffers incoming writes in memory.
42
When the buffer fills up, we sort and flush it to disk as an immutable SS table.
43
The number of SS tables increases as more buffers get flushed to disk.
44
This creates a problem for reads, as each read has to search through these SS tables to perform a lookup.
45
To cap the number of SS tables it has to search through for each read, the LSM tree merges the SSTables and compacts them in the background.
46
Let's look at the compaction process more closely.
47
When the SSTables are merged, they are organized into levels.
48
This is where the tree part of the LSM tree comes into the picture.
49
There are different strategies to determine when and how the SSTables are merged and compacted.
50
There are two broad strategies, size-tiered compaction and leveled compaction.
51
Size tier compaction is optimized for write throughput.
52
Level compaction is more read optimized.
53
We won't go into too much detail on how these compaction strategies work.
54
There are a few key points to remember though.
55
Compaction keeps the number of SS tables manageable.
56
The SS tables are organized into levels, with each level gets exponentially larger as SS tables from the level above emerged into it.
57
Compaction consumes a lot of I.O.
58
A mistuned compaction could starve the system and slow down both read and write.
59
Finally, let's visit some common optimizations for the LSM trees in production systems.
60
There are many optimization strategies that try to provide read performance closer to that of the B tree.
61
What we explain below are just the common ones, and they are by no means exhaustive.
62
To look up a key, it performs searches on the SS tables at every level.
63
Even though search is fast on sorted data, going through all the on-disk SS tables consumes a lot of I.O.
64
Many systems keep a summary table in memory that contains the min-max range of each disk block of every level.
65
It allows the system to skip searches on those disk blocks where the key does not fall within the range.
66
This saves a lot of random I.O.
67
Another issue that could be potentially expensive is to look up a key that does not exist.
68
This would require looking through all the eligible blocks at all the levels.
69
Most systems keep a bloom filter at each level.
70
A bloom filter is a space-efficient data structure that returns a firm no if a key does not exist, or a probably yes if a key might exist.
71
This allows the system to skip a level entirely if the key does not exist there, which dramatically reduces the number of random I.O required.
72
In conclusion, NoSQL databases backed by an LSM tree could be tuned to support a very high write rate.
73
As with any database, proper tuning is the key.
74
For an LSM tree, compaction tuning is the most critical.
75
If you would like to learn more about system design, check out our books and weekly newsletter.
76
Please subscribe if you learn something new.
77
Thank you so much and we'll see you next time.
הורד אפליקציה
כל מה שאתה צריך כדי לדבר בשטף
הגיית AIציון לכל משפט
תרגול IPAשלוט בכל צליל
אוצר מיליםבנה את מאגר המילים שלך
משחק אוצר מיליםלמד תוך כדי משחק
אותו ערוץ
C13
Proxy vs Reverse Proxy (Real-world Examples)
Daily ConversationUniversity StudentsWorking Adults
C12
Cache Systems Every Developer Should Know
Daily ConversationUniversity StudentsWorking Adults
C12
Top 5 Redis Use Cases
Daily ConversationUniversity Students
C13
Everything You Need to Know About DNS: Crash Course System Design #4
Daily ConversationUniversity StudentsWorking Adults
C12
CAP Theorem Simplified
Daily ConversationUniversity StudentsWorking Adults
C12
What Is REST API? Examples And How To Use It: Crash Course System Design #3
Daily ConversationUniversity StudentsWorking Adults
C18
8 Most Important System Design Concepts You Should Know
Daily ConversationUniversity StudentsWorking Adults
C11
System Design: Why is Kafka fast?
Business EnglishUniversity StudentsWorking Adults
C16
What Are Microservices Really All About? (And When Not To Use It)
Business EnglishUniversity StudentsWorking Adults
C13
System Design Was HARD - Until You Knew the Trade-Offs
Daily ConversationUniversity StudentsWorking Adults
C13
How Does a URL Shortener Work?
Business EnglishInterview EnglishUniversity Students
C13
System Design: Why is Kafka Popular?
Business EnglishUniversity StudentsWorking Adults
C14
What is API Gateway?
Business EnglishUniversity StudentsWorking Adults
C12
10+ Key Memory & Storage Systems: Crash Course System Design #5
University StudentsWorking Adults
C150
Kubernetes Explained in 6 Minutes | k8s Architecture
Business EnglishUniversity StudentsWorking Adults
C24
What Is Redis Really About? Why Is It So Popular?
Business EnglishUniversity StudentsWorking Adults
B2How Git Works: Explained in 4 Minutes
Business EnglishUniversity StudentsWorking Adults
FOR YOU
המלץ על סרטון
C11
398 משפטים
Sec Growth DataScience staff meeting Sep 14 2022
Daily ConversationUniversity StudentsWorking Adults
C11
65 משפטים
Did Anthropic Kill Git?
Daily ConversationUniversity StudentsWorking Adults
C13
68 משפטים
what is a database index?
Daily ConversationUniversity StudentsWorking Adults
C11
65 משפטים
How do microSD Cards Store 2 TB in such a TINY space?
Daily ConversationUniversity StudentsWorking Adults
B21
42 משפטים
Database vs Data Warehouse vs Data Lake | Explaining the Differences
Daily ConversationUniversity StudentsWorking Adults
C1231 משפטים
Redis Crash Course - the What, Why and How to use Redis as your primary database
Daily ConversationUniversity StudentsWorking Adults
B21
205 משפטים
The Man Who Stole MySpace | Samy
Daily ConversationUniversity StudentsWorking Adults
C11
27 משפטים
You are wasting memory
Daily ConversationUniversity StudentsWorking Adults
C113
202 משפטים
SAC | Soft Actor Critic (SAC) architecture | SAC Explained
Daily ConversationUniversity StudentsWorking Adults
C11
689 משפטים
Database Sharding! Designing Data-Intensive Applications chapter 6
Daily ConversationUniversity StudentsWorking Adults
C11
74 משפטים
LangChain in 5 Minutes (Explained Clearly)
Daily ConversationUniversity StudentsWorking Adults
אוצר מילים והערות הגייה לשיעור הזה
שיעור הדיבור הזה ברמה C1 מבוסס על הסרטון “The Secret Sauce Behind NoSQL: LSM Tree”. המילים שחוזרות הכי הרבה: table, tree, key, Lsm, compaction. בסרטון הזה יש 77 משפטים ו-1089 מילים לתרגול Shadowing. הדיבור נמשך 7:26. הדובר שומר על קצב יציב של כ-147 מילים בדקה, קצב נוח לתרגול Shadowing. רק 72% מהמילים נמצאות בין 3,000 המילים הנפוצות ביותר באנגלית, כך שאוצר המילים מאתגר.
אוצר מילים מרכזי בסרטון הזה
15 המילים המתקדמות ביותר בסרטון, עם ההגייה והמשמעות שלהן:
| מילה | הגייה | משמעות |
|---|---|---|
| merge פועל | /mɝd͡ʒ/ | מיזג, מירג׳ג׳ |
| disk שם עצם | /dɪsk/ | דִּיסְק |
| tombstone שם עצם | /ˈtuːmstoʊn/ | מַצֵּבָה |
| consume פועל | /kənˈsum/ | צרך |
| organize פועל | /ˈɔɹɡənaɪz/ | ארגן |
| compact שם תואר | /kəmˈpækt/ | דחוס |
| accumulate פועל | /əˈkjuːmjʊˌleɪt/ | צָבַר, אָגַר |
| chronological שם תואר | /ˌkɹɑnəˈlɑd͡ʒɪkəl/ | כּרוֹנוֹלוֹגִי |
| reclaim פועל | /ɹɪˈkleɪm/ | תָּבַע בַּחֲזָרָה |
| starve פועל | /stɑɹv/ | גָּוַע |
| subset שם עצם | /ˈsʌbˌsɛt/ | תת קבוצה |
| accumulation שם עצם | /əˌkjuːm.jəˈleɪ.ʃən/ | אגירה, צבירה |
| periodic שם תואר | /ˌpɪɹiˈɑdɪk/ | מחזורי |
| explode פועל | /ɪkˈsploʊd/ | הִתְפּוֹצֵץ |
| emerge פועל | /ɪˈmɝd͡ʒ/ | צץ, התגלה |
Phrasal verbs שתשמעו
| מילה | משמעות |
|---|---|
| check out פועל | בדק |
דקדוק בסרטון הזה
המבנים שהדובר משתמש בהם הכי הרבה, עם המילים המדויקות מהסרטון:
| מבנה | בסרטון |
|---|---|
| סביל (Passive voice) be + past participle — הדגש הוא על מה שקורה, לא על מי שעושה את הפעולה | is optimized · are bashed · is ordered |
| הווה מושלם (Present perfect) have/has + past participle — פעולה שקרתה בעבר ועדיין יש לה חשיבות או השפעה עכשיו | have exploded · has been deleted |
הגייה ששווה לשים לב אליה
- הצלילים של “th”: throughput /ˈθɹuː.pʊt/, algorithm /ˈælɡəɹɪðm̩/
- הצלילים של “sh” ו-“zh”: compaction /kəmˈpækʃən/, flush /ˈflʌʃ/, optimization /ˌɑptəmaɪˈzeɪʃən/, exponentially /ɛkspə(ʊ)ˈnɛnʃəli/, insatiable /ɪnˈseɪʃ(j)əbəl/
- מילים ארוכות — שימו לב להטעמה: optimization /ˌɑptəmaɪˈzeɪʃən/, accumulate /əˈkjuːmjʊˌleɪt/, chronological /ˌkɹɑnəˈlɑd͡ʒɪkəl/, exponentially /ɛkspə(ʊ)ˈnɛnʃəli/, insatiable /ɪnˈseɪʃ(j)əbəl/
צלילים שדוברי עברית מתקשים בהם:
- /æ/ — פתוח יותר מה-“אֶ” העברית, לא כמו “a” ב-“cup”: compaction /kəmˈpækʃən/, compact /kəmˈpækt/, manageable /ˈmænəd͡ʒəb(ə)l/, bash /bæʃ/, algorithm /ˈælɡəɹɪðm̩/
- /r/ — הלשון לא נוגעת בשום מקום, לא ה-“ר” הגרונית: buffer /ˈbʌfəɹ/, organize /ˈɔɹɡənaɪz/, chronological /ˌkɹɑnəˈlɑd͡ʒɪkəl/, discard /dɪsˈkɑɹd/, override /oʊ.vɚˈɹaɪd/
- עיצור קולי בסוף מילה — /b/, /d/, /g/, /z/, /v/ בלי להפוך אותם לאטומים: merge /mɝd͡ʒ/, optimize /ˈɑptɪmaɪz/, outdated /aʊtˈdeɪtɪd/, organize /ˈɔɹɡənaɪz/, discard /dɪsˈkɑɹd/
איך לתרגל עם הסרטון הזה
- הקשיבו לכל הסרטון פעם אחת בלי לדבר, ורשמו את המילים שאתם לא מכירים.
- תרגלו Shadowing משפט אחר משפט במהירות רגילה, וחזרו על כל משפט עד שהקצב שלכם תואם את הדובר.
- הקליטו את עצמכם והשוו למקור, תוך תשומת לב למילים כמו merge, disk, tombstone.
מהי טכניקת השאדואינג?
שאדואינג היא טכניקת למידת שפה מבוססת מדע שפותחה במקור להכשרת מתורגמנים מקצועיים ופופולרית על ידי הפוליגלוט ד"ר אלכסנדר ארגואלס. השיטה פשוטה אך עוצמתית: אתה מקשיב לאודיו באנגלית ילידית וחוזר עליו בקול רם מיד — כמו צל העוקב אחרי הדובר עם עיכוב של 1–2 שניות בלבד. בניגוד להאזנה פסיבית או תרגילי דקדוק, שאדואינג מכריח את המוח והשרירים בפה לעבד ולשחזר דפוסי דיבור אמיתיים בו זמנית. מחקרים מראים שזה משפר משמעותית את דיוק ההגייה, האינטונציה, הקצב, הדיבור המחובר, הבנת השמיעה ושטף הדיבור — מה שהופך אותו לאחת השיטות היעילות ביותר להכנה לIELTS Speaking ולתקשורת באנגלית בעולם האמיתי.