शैडोइंग अभ्यास: Learn Data Modeling in 8 minutes: Dimensional Data Modeling, Data Vault, and One Big Table - वीडियो के साथ अंग्रेजी बोलना सीखें
पाठ बनाया जा रहा है...
1
Data modeling is one of the most important skills that data engineers have to learn.
2
So how is it so important?
3
Well, at the end of the day, data modeling is the structure of the data that we create for businesses.
4
Because the asset that we produce for businesses is data to make better decisions.
5
And data modeling has a bunch of different components to it.
6
And we're going to talk about each of them.
7
There's the highest level, it's called conceptual data modeling.
8
Then we're going to be going over the next layer down, which is logical data modeling.
9
And then the final layer that we're going to talk about is physical data modeling.
10
In this video, we should be covering all of those.
11
So at the end of this, you should know all the ins and outs of all three types of these data modeling.
12
So let's first talk about the highest level conceptual data modeling, which is the highest level way of thinking about it is like, what data do we need?
13
What data does the business desire?
14
What data do we have access to?
15
What are the sources?
16
Where can we find this data?
17
Can we like think about all the relationships of this data, like at a very high level? because sometimes you might want data in a business, but it's actually unfeasible to get it,
18
or it's going to take months and months and months.
19
And it's not ROI positive
20
because the time it would take to get the data is more than the value you can get out of it.
21
So this area of data modeling is very, very, very important because if you get this wrong,
22
you can end up spending a lot of extra time on the data model for things that don't matter that much.
23
Like there might be an extra column or an extra table
24
that like some data scientists ask for that they're only going to use one time.
25
And if you push back on that, it makes your job as a data engineer a lot easier because you remove that requirement because it's a low value requirement.
26
So you need to be able to like walk through all the requirements of the data model at this highest level.
27
And really having a deep understanding of the business is where you can really, really shine as a data engineer with conceptual data modeling.
28
So after you have mastered the conceptual data model, you skip into the next layer, which is the logical data model.
29
So the logical data model is where you start talking about like facts and dimensions and how they're related.
30
Like, so facts are events.
31
You can think of that as like I click the login button.
32
I purchase a product.
33
I send a message to a friend.
34
Like all of these are like events and actions that you can take on a website.
35
And these are going to be your facts because they can't really change.
36
After I've logged in on Facebook at 6.03 p.m., like I can't go back into a time machine and change that.
37
So then you have dimensions, which are going to be like your nouns or like your actors in this space, right?
38
So you have things like users or listings or posts or, you know, devices.
39
These are all like things that bring context.
40
Cause like I could be like, I'm a user in China who clicks an ad, right?
41
And then the user in China bit, that part of my user object and then clicking the ad is part of the fact.
42
And so those are the ways to be thinking about all of these things.
43
So the logical data model is all about how do you find all of these entities and how are they related.
44
So like, for example, when you click on an ad on Facebook, the entities that are involved, the user who clicked the ad, you also have the device that clicked the ad,
45
you also have the browser potentially that clicked the ad.
46
And so those are three dimensions
47
that you can bring in to your fact data to make your fact data richer and more powerful.
48
So after you have gotten the logical data layer nailed down, then you move into the final layer, which is the physical layer.
49
And this is the layer where people oftentimes, like they think the physical layer is the only part of data modeling when it's actually not,
50
it's actually like probably the least important in some regards, but it's also where it becomes real.
51
It becomes reality.
52
It's physical data now.
53
So this is where you start looking at the schema.
54
What are the columns of your data?
55
Like what are the data types of your data?
56
How are you storing this data?
57
How can you compress this data to make it smaller?
58
So in this space, you have a couple different data modeling techniques
59
that you can look into
60
that can really help understand all the way up from like physical to logical to conceptual
61
and you can like understand there's a couple different paradigms here
62
so one is called one big table
63
so one big table essentially says you don't need facts
64
and dimensions you can essentially have a fact
65
that has all of its context all the context is already in the fact
66
so you don't need to do joins you can just do aggregations you don't need to join at all
67
because when data gets very big joins become very expensive like actually joining in the the nouns into your data.
68
But the trade here, right, is that you're duplicating data, right?
69
Because every now my user data is now in every single fact, right?
70
And say I do 300 events on a data set.
71
This is what's called the compute storage trade off, right?
72
Because it is duplicated in storage.
73
But why is this potentially worth it is
74
because you can now do aggregations without doing join
75
and aggregations are faster than join because you don't have to like pull data from all these different locations, you can just aggregate right down and group very quickly.
76
And so this is what is the advantage of one big table, even though one big table also duplicates a lot of data and is super painful.
77
So then you have the more traditional approach of facts and dimensions, where like you separate your nouns and you separate your verbs and your events, and then you join them together.
78
That's called dimensional data modeling.
79
Dimensional data modeling is the oldest technique of all the techniques that we have talked about.
80
And it's the one that most data engineers should probably pick.
81
It's the most foundational and the most fundamental.
82
So definitely, if you haven't learned about dimensional data modeling, I'll put a link in the description below.
83
I have many hours of content on dimensional data modeling on my YouTube channel.
84
And then last but not least here is called a data vault.
85
So data vault is very interesting because data vault is all about preserving the rawness of the data.
86
So you want to bring in data in its most untransformed state and kind of stack it onto things.
87
And this allows you to essentially know exactly what the data was in its rawest form.
88
So you don't end up filtering something out that you didn't mean to or transform something that you made a mistake on.
89
And Data Vault allows you to back up into like the most raw form of the data.
90
It's kind of related to ELT, you know, extract load transform instead of extract transform load.
91
That's it's kind of related to that.
92
But it's also about how you actually model the data.
93
And it has a very strict kind of thing in it and like a strict kind of ideology to it.
94
That's why I don't really I don't really subscribe to Data Vault as much.
95
I really like dimensional data modeling and one big table data modeling.
96
But at the end of the day, like it's still a very useful technique.
97
And you can combine all of these techniques.
98
I'm going to give an example.
99
So when I worked at Airbnb, I actually built a data model that was a mix of one big table and data vault and dimensional data modeling.
100
So how it worked was we use data vault to get all of the inputs.
101
So the inputs in this case were for availability and price
102
because the host sets the price rules like what is the daily price?
103
Are there any discounts?
104
Are there any like any length of stay requirements?
105
Maybe you have to book for three days or seven days or you know, you can only book 48 hours in advanced, there's a lot of these different rules that hosts can set in Airbnb.
106
And so I took all those rules in their raw form and combine them into one data set called inputs.
107
And that was the data vault kind of technique that I use.
108
And then from the inputs table, we process it through to a new table called listing pricing.
109
So now we actually know what the actual prices would have been given those rules.
110
And that table was one big table because it had all of the nights in one column.
111
So one of the giveaways of like a one big table data set is that it uses complex data types, like it uses things like struct and array and like map
112
and those kind of like more complicated data types more than just like, you know, string and integer and decimal.
113
It's like one big table very often is a table within a table.
114
And that was a big thing I did here at Airbnb.
115
Keeping in mind that like you don't have to adhere to any one of these three religions
116
when you are working with with data modeling, you can also kind of do whatever you want, right?
117
And mix and match.
118
It's more of an art than a science.
119
And it's really about matching business requirements to the physical data modeling needs that you, or the physical data modeling tools that you have.
120
So yeah, I hope you found this content interesting and make sure to like, follow, and subscribe.
✨ अनुशंसित वीडियो
B21
220 वाक्य
How Modern Data Teams Work (Engineer, Analyst, Scientist, Architect, ML)
Business EnglishUniversity StudentsWorking Adults
C14
123 वाक्य
Introduction to Diminsionality Reduction (a.k.a. Manifold Learning)
Business EnglishUniversity StudentsWorking Adults
C1288 वाक्य
Using Selection & Scan Processing tools for reverse engineering | Geomagic Design X
Business EnglishUniversity StudentsWorking Adults
B2155 वाक्य
80% of Interior Design Basics in 10 minutes
Business EnglishUniversity StudentsWorking Adults
C15
34 वाक्य
Marketing Research Essentials Intro, Models & Intelligence
Business EnglishUniversity StudentsWorking Adults
C13
53 वाक्य
System Design Was HARD - Until You Knew the Trade-Offs
Business EnglishUniversity StudentsWorking Adults
C18
102 वाक्य
World Models explained in 10min..
Business EnglishUniversity StudentsWorking Adults
C11
392 वाक्य
Uber Data Engineering Mock Interview - Ride-Sharing Data Warehouse Schema
Business EnglishInterview EnglishUniversity Students
C11
50 वाक्य
0.0.1 How to become a CAE Engineer CAE | What are the ways to become a CAE Engineer as a fresher
Business EnglishUniversity StudentsWorking Adults
C13
84 वाक्य
What is the 8D report? (problem solving tools)
Business EnglishUniversity StudentsWorking Adults
C12
51 वाक्य
Cloud Models - CompTIA Network+ N10-009 - 1.3
Business EnglishUniversity StudentsWorking Adults
C11
133 वाक्य
Data Ingestion Best Practices! How to Ingest Data Efficiently at Enterprise Scale!
Business EnglishUniversity StudentsWorking Adults
इस पाठ की शब्दावली और बोलने के सुझाव
इस वीडियो में शैडोइंग के लिए 120 वाक्य और 1736 शब्द हैं। बोला गया हिस्सा 8:06 लंबा है। वक्ता तेज़ बोलता है, लगभग 214 शब्द प्रति मिनट, इसलिए जुड़ी हुई और कमज़ोर ध्वनियाँ ज़्यादा मिलेंगी। 87% शब्द अंग्रेज़ी के 3,000 सबसे आम शब्दों में से हैं, इसलिए शब्दावली आसान है।
इस वीडियो की मुख्य शब्दावली
वीडियो के 15 कम प्रचलित शब्द, उच्चारण और अर्थ के साथ:
- layer /ˈleɪ̯ɚ/ (संज्ञा) — परत, तह. A single thickness of some material covering a surface.
- technique /tɛkˈniːk/ (संज्ञा) — तकनीक. The practical aspects of a given art, occupation etc.; formal requirements.
- logical /ˈlɑd͡ʒɪkəl/ (विशेषण) — तार्किक. In agreement with the principles of logic.
- dimensional /daɪˈmɛn.ʃə.nəl/ (विशेषण) — विमीय. Of or pertaining to dimensions.
- requirement /ɹɪˈkwaɪɹmənt/ (संज्ञा) — आवश्यकता, अनिवार्यता. A necessity or prerequisite; something required or obligatory. Its adpositions are generally of in relation to who or what has given it, on in relation to whom…
- dimension /daɪˈmɛn.ʃən/ (संज्ञा) — पहलू. A single aspect of a given thing.
- noun /naʊn/ (संज्ञा) — संज्ञा, नाम. A word that functions as the name of a specific object or set of objects, such as person, animal, place, word, thing, phenomenon, substance, quality, or idea…
- duplicate /ˈd(j)uː.plɪˌkeɪt/ (क्रिया) — अनुकृति करना, अनुकरण करना. To make a copy of.
- column /ˈkɒləm/ (संज्ञा) — स्तंभ, सुतून. A solid upright structure designed usually to support a larger structure above it, such as a roof or horizontal beam, but sometimes for decoration.
- combine /kəmˈbaɪn/ (क्रिया) — मिलाना. To bring (two or more things or activities) together; to unite.
- entity /ˈɛn.tə.ti/ [-tʰi] (संज्ञा) — अस्तित्वी. That which has a distinct existence as an individual unit, often used for organizations which have no physical form.
- complicated /ˈkɑm.plɪˌkeɪ.tɪd/ (विशेषण) — जटिल, पेचीदा. Difficult or convoluted.
- skill /skɪl/ (संज्ञा) — कुशलता, कौशल. A capacity to do something well; a technique, an ability, usually acquired or learned, as opposed to abilities that are regarded as innate.
- string /stɹɪŋ/ (संज्ञा) — तार, रस्सी. A long, thin and flexible structure made from threads twisted together.
- log /lɔɡ/ (संज्ञा) — कुंदा, लट्ठा. The trunk of a dead tree, cleared of branches.
वीडियो में आने वाली फ़्रेज़ल वर्ब
- think about /ˈθɪŋk əˌbaʊt/ (क्रिया) — चिंतन करना. To ponder.
- look at (क्रिया) — हेरना. To observe or watch.
ध्यान देने योग्य उच्चारण
वक्ता 23 बार संक्षिप्त रूपों का प्रयोग करता है, जैसे don't, we're, can't। इन्हें वैसे ही छोटा बोलें जैसा आप सुनते हैं।
- “sh” और “zh” की ध्वनियाँ: dimensional /daɪˈmɛn.ʃə.nəl/, dimension /daɪˈmɛn.ʃən/, essentially /ɪˈsɛnʃəli/, potentially /pəˈtɛnʃ(ə)li/, shine /ʃaɪn/
- लंबे शब्द — बलाघात सही रखें: dimensional /daɪˈmɛn.ʃə.nəl/, essentially /ɪˈsɛnʃəli/, potentially /pəˈtɛnʃ(ə)li/, entity /ˈɛn.tə.ti/ [-tʰi], complicated /ˈkɑm.plɪˌkeɪ.tɪd/
इस वीडियो से अभ्यास कैसे करें
- पहले बिना बोले पूरा वीडियो एक बार सुनें और जो शब्द नहीं आते उन्हें लिख लें।
- 0.75× गति से शुरू करें, एक-एक वाक्य की शैडोइंग करें, और सहज लगने पर सामान्य गति पर लौटें।
- अपनी आवाज़ रिकॉर्ड करें और मूल से तुलना करें, ख़ासकर layer, technique, logical जैसे शब्दों पर ध्यान दें।
इस वीडियो का व्याकरण
वक्ता जिन संरचनाओं का सबसे अधिक प्रयोग करता है, वीडियो के असली शब्दों के साथ:
| संरचना | वीडियो में |
|---|---|
| Present perfect have/has + past participle — बीता हुआ काम जिसका असर अभी भी है | have mastered · I've logged · have gotten |
| Passive voice be + past participle — ज़ोर इस पर कि क्या होता है, न कि कौन करता है | are involved · is called · is duplicated |
| Relative clauses who / which + उपवाक्य — व्यक्ति या वस्तु के बारे में अतिरिक्त जानकारी | down, which is · modeling, which is · layer, which is |
शैडोइंग तकनीक क्या है?
शैडोइंग (Shadowing) एक विज्ञान-समर्थित भाषा सीखने की तकनीक है जो मूल रूप से पेशेवर दुभाषिया प्रशिक्षण के लिए विकसित की गई थी। विधि सरल लेकिन शक्तिशाली है: आप मूल अंग्रेज़ी ऑडियो सुनते हैं और तुरंत इसे ज़ोर से दोहराते हैं — जैसे वक्ता की छाया 1-2 सेकंड की देरी से। शोध से पता चलता है कि यह उच्चारण सटीकता, स्वर, लय, जुड़ी हुई ध्वनियाँ, सुनने की समझ और बोलने की प्रवाहशीलता में काफ़ी सुधार करता है।