Практика Shadowing: Password Storage Tier List: encryption, hashing, salting, bcrypt, and beyond - Изучайте разговорный английский по видео

If you're building a product that requires logging in, you probably have to deal with storing passwords.

But storing passwords opens you up to security risks, like someone breaking into your database, stealing all your passwords, and dealing immense reputational damage to your product.

So how do you store passwords in a secure fashion?

Let's start from the worst way, and steadily work our way up to something a little more secure.

I'm going to assume you have a little bit of software engineering knowledge.

Let's go.

The F tier way to store your passwords is like normal data.

a normal column perhaps with the rest of your user data.

Sometimes this is called storing passwords in plain text.

This is pretty bad because if a hacker gains access to your database, they can steal all the passwords effortlessly.

Security is all about multiple lines of defense, and storing passwords in plain text is not a defense.

If your database is breached, you're done, and database breaches happen quite often.

That's why this is F tier.

But how can we do better?

If you watched my encryption video, you might think that we should encrypt these passwords.

That's definitely an improvement, because if they break into your database, they get gibberish instead of useful information.

However, if they do get access to the key, then they can decrypt all the passwords, and you're unfortunately right back to F tier.

It's definitely nicer that the passwords aren't just sitting out there in the open, but if a hacker has access to your database already,

it may not be much more effort for them to steal the decryption key too from an adjacent server or config file.

So storing passwords in a way that makes it possible to retrieve the password increases the risk of an inside job,

where an employee with privileged access to the key decrypts people's passwords for nefarious purposes.

The Xword protection certainly helps, but this is D tier.

How do we get to C?

The key insight is that we only need to know

when the user types in the same password as when they signed up.

We don't actually need to know the password itself.

You might think the two are one and the same, but they actually aren't.

It's possible to take a password and generate a fingerprint from it, which will let us know if we encounter the same password in the future, but still be unable to recover the password directly from the fingerprint.

This is a technique known as hashing.

How do we use hashing?

When the user signs up, we take the password they provide, generate a small fingerprint or hash, and store it instead of the actual password.

In the future, when the user tries to log in, we take the password they typed in, we're the same hashing process and see if the output hashes compare.

If the hashes are the same, it's overwhelmingly likely that they typed in the correct password.

Thanks to the power of hashing, we're able to verify someone's password without storing the actual password.

Hashing might sound a little magical at the moment.

How does hashing prevent someone from recovering their original password from the output hash?

And how does it prevent two strings from hashing to the same thing

and allowing someone to log in with the wrong password?

For the first one, how to prevent someone from recovering their original password, the process of hashing involves a lot of aggressive mixing of the data in the original input to produce the output.

It's like taking three paints of different colors and mixing them together which probably results in some weird shade of brown.

With just the brown, you'd probably be unable to guess the original three colors that made the brown.

In the same way, you can't figure out the original password from the hash

because the bits in the data have all been thoroughly mixed and scrambled.

This property is known as being a one-way function, like a one-way street.

Once you hash, you can't unhash.

For the second, which is how hashing makes it hard for two strings to hash to the same value known as a hash collision, the answer is similar to what I just said.

That hash function mixes the data in such a way that even small tweaks in the input result in totally different hashes.

As a result, for good hash functions, there is no publicly known way to generate two strings whose hashes collide without just trying many many many strings.

No luck with these ten!

This property is called collision resistance.

Picking the right hash function is important.

We used to think that older hash functions that you may have heard about, like md5 and SHA-1 had collision resistance,

but cryptographic progress marches on and they've been proved insecure.

For example, about a decade ago, researchers published a method for generating a collision, two strings that have the same MD5 hash that you can run in about a second on your laptop.

So we've moved on to stronger hash functions like SHA-2, which is the one I've been using in this video.

With our new hashing technique under our belt, let's hash all the passwords in our database instead of encrypting them to arrive at C tier password storage.

Now, we don't have a key that effortlessly unlocks the password table like before, which is definitely an improvement.

However, we need to talk about dictionary attacks.

As it turns out, many people use extremely weak passwords like password or 123456.

If a hacker has access to your database, they can just run your hash function on password or 123456 to get the hash, and then find all the hashes that match in your database.

Voila, they've broken those people's passwords.

This is called a dictionary attack because you could run through the dictionary, hashing words as you go and seeing if any of them match the hash.

Remember that it's not possible to directly turn a hash back into a password, in general.

However, because the hacker knows the hash came from a password, they can use their knowledge of human nature to narrow down the possible guesses, which makes guessing feasible.

Furthermore, they can use lists of the most common passwords to pre-generate a huge database of hashes.

So all they need to do is to compare their huge database to your database to find matches.

These databases are known as rainbow tables.

How can we defend against this?

Let's go to B tier for a technique that protects against rainbow tables but not against dictionary attacks.

It's a technique called salting.

When the user signs up, rather than directly hashing each password, we can first generate a short random string called the salt.

Then we prepenn it to the string before running the hash.

We then store the salt next to the hashed password.

When the user logs in, we can prepenn the remembered salt to the password they entered and hash to see if it matches the stored password.

It's important to use the same salt at login that was generated during signup, which is why we want to store it in the database.

Otherwise the user wouldn't be able to log in even with the right password because the hashes wouldn't match.

This makes Rainbow Tables useless because their databases only contain non-salted hashes.

For example, these users have the password QWERTY but the table doesn't match anymore.

This has the added benefit of decorrelating users who have the same password.

Previously if two users used the same password their hashes would be the same, which is not ideal because you can see

that they have the same password even if you don't know what the password actually is.

Now, even though their passwords are the same, their salts are different, so the hashes are different.

But why is this still B-tier?

Remember the other issue we talked about, which is dictionary attacks.

An attacker with a list of common passwords can still try all of them against a hash using the salt.

They don't get the benefit of pre-computation, but they can still start from scratch.

Furthermore, specialized hardware like GPUs have made it possible for people to compute billions of hashes per second, which translates into billions of guesses per second.

How on earth do we stop this one?

Let's see what A-tier has to say.

An A-tier is using a specialized password hashing function that is deliberately slow.

The previous hash functions we discussed are designed to be fast because they're used for other applications besides passwords.

Password hashing functions like bcrypt, scrypt, and argon2 come with salting for free and more importantly are designed to be really really really slow,

to consume lots of power and to take lots of memory.

This sounds weird, but this is actually on purpose, to defend against the overwhelming power of hardware.

The billions of hashes per second we saw a moment ago can be slowed down to mere thousands per second, if not even slower, because you can actually choose the level of slowness you want.

This is known as the work factor.

With only thousands of guesses per second, that's still enough to easily go through the most common passwords, but it's not enough to break tougher or more obscure passwords.

With a high enough work factor, hackers might be able to break some of the passwords in your system, but not all of them.

After all, the goal of security is not to be immune to attacks which is impossible, but to hinder attackers enough that they turn their attention to other places.

100

As a concrete example, let's look at bcrypt.

101

The output of bcrypt looks like this.

102

It has a prefix that identifies the output as a bcrypted thing, a work factor, a 22 character salt, and a 31 character hash.

103

The most interesting thing about this is that the work factor is exponential.

104

When you increase the work factor by 1, the function becomes twice as slow.

105

Here's a graph of bcrypt hash time by work factor on my laptop.

106

The exponential increase is quite clear.

107

Most folks recommend setting it to around 15 for real use cases, which takes 1.3 seconds on my laptop.

108

Fun fact, when it was first published, the recommended work factor was just 6, which would take just 2 milliseconds on my modern laptop.

109

So is that it?

110

Are we done?

111

Is there an even higher tier?

112

S tier?

113

It's kind of a trick answer, but there is.

114

S tier is not storing passwords at all.

115

As we've learned in this video, storing passwords is quite tricky, so consider ways to avoid doing it at all.

116

For example, you can use other authentication services like sign in with Google

117

or sign in with Facebook and others so that users can log into your product using an established authentication platform.

118

Not only is it more convenient for users, you get to sleep better at night knowing that your system doesn't have any passwords at all in it.

119

That's all I have for today.

120

Let's recap.

121

First, we looked at storing passwords in plain text which means anyone can easily access them.

122

So we considered encrypting them, which is not ideal because their original form can still be recovered.

123

We introduced hashing to make that impossible, but we saw that people can pre-compute huge rainbow tables to quickly break passwords.

124

To counteract that, we introduced salting, which individualizes each password with a random string called the salt, so that pre-computed databases don't work anymore.

125

Unfortunately, hardware is so fast that even without pre-computed databases, attackers can still try billions of guesses per second,

126

so we need to switch to deliberately slow password hashing functions that greatly decrease the rate of guesses.

127

Finally, we bypassed the whole problem by considering how to not store passwords at all.

128

I hope this was helpful.

129

If you enjoyed this video, please consider sharing it with somebody else who might also enjoy it.

130

Thanks again!

Что такое техника Shadowing?

Shadowing — это научно обоснованная техника изучения языка, изначально разработанная для подготовки профессиональных переводчиков и популяризированная полиглотом доктором Александром Аргуэльесом. Метод прост, но эффективен: вы слушаете аудио на английском от носителей языка и немедленно повторяете вслух — как тень, следующая за говорящим с задержкой в 1–2 секунды. В отличие от пассивного прослушивания или грамматических упражнений, Shadowing заставляет мозг и мышцы рта одновременно обрабатывать и воспроизводить реальные речевые паттерны. Исследования показывают, что это значительно улучшает точность произношения, интонацию, ритм, связную речь, понимание на слух и беглость речи — что делает его одним из самых эффективных методов для подготовки к IELTS Speaking и реального общения на английском.

Практика Shadowing: Password Storage Tier List: encryption, hashing, salting, bcrypt, and beyond - Изучайте разговорный английский по видео

Об этом уроке

Ключевая лексика и фразы

Советы по практике

Что такое техника Shadowing?