跟讀練習: Large Language Models explained briefly - 通過影片學習英語口語

選擇練習模式

每種模式都會開啟全屏練習室。

81 句

如果句子過短或過長，請點選 Edit 進行調整。

Imagine you happen across a short movie script that describes a scene between a person and their AI assistant.

想像你偶然看到一份短篇電影劇本，內容描述一個人和他的AI助理之間的對話場景。

The script has what the person asks the AI.

這份指令稿包含使用者向AI提出的要求。

but the AI's response has been torn off.

但AI的回應被截斷了。

Suppose you also have this powerful magical machine that can take any text

and provide a sensible prediction of what word comes next.

You could then finish the script by feeding in what you have to the machine, seeing what it would predict to start the AI's answer,

and then repeating this over and over with a growing script completing the dialogue.

When you interact with a chatbot, this is exactly what's happening.

A large language model is a sophisticated mathematical function that predicts what word comes next for any piece of text.

Instead of predicting one word with certainty, though, what it does is assign a probability to all possible next words.

to build a chatbot.

What you do is lay out some text that describes an interaction between a user and a hypothetical AI assistant,

You add on whatever the user types in as the first part of that interaction,

And then you have the model repeatedly predict the next word that such a hypothetical AI assistant would say in response, and that's what's presented to the user.

In doing this, the output tends to look a lot more natural

if you allow it to select less likely words along the way at random.

So what this means is even though the model itself is deterministic, A given prompt typically gives a different answer each time it's run.

Models learn how to make these predictions by processing an enormous amount of text, typically pulled from the internet.

For a standard human to read the amount of text that was used to train GPT -3, for example, if they read "non -stop, 24 /7," it would take over 2600 years.

Larger models since then train on much, much more.

You can think of training a little bit like tuning the dials on a big machine.

The way that a language model behaves is entirely determined by these many different continuous values, usually called parameters or weights.

Changing those parameters will change the probabilities that the model gives for the next word on a given input.

What puts the "large" in large language model is how they can have hundreds of billions of these parameters.

you No human ever deliberately sets those parameters.

Instead, they begin at random, meaning the model just outputs gibberish, but they are repeatedly refined based on many example pieces of text.

One of these training examples could be just a handful of words, or it could be thousands, but in either case, the way this works is to pass in all

but the last word from that example into the model, and compare the prediction that it makes with the true last word from the example.

An algorithm called backpropagation is used to tweak all of the parameters in such a way

that it makes the model a little more likely to choose the true last word

and a little less likely to choose all the others.

When you do this for many, many trillions of examples, Not only does the model start to give more accurate predictions on the training data,

but it also starts to make more reasonable predictions on text that it's never seen before.

Given the huge number of parameters and the enormous amount of training data, The scale of computation involved in training a large language model is mind -boggling.

To illustrate, imagine that you could perform 1 billion additions and multiplications every single second.

How long do you think it would take for you to

do all of the operations involved in training the largest language models?

Do you think it would take a year?

Maybe something like 10 ,000 years?

The answer is actually much more than that.

It's well over 100 million years.

you This is only part of the story, though.

This whole process is called pre -training.

The goal of auto -completing a random passage of text from the internet

is very different from the goal of being a good AI assistant.

To address this, chatbots undergo another type of training, just as important, called reinforcement learning with human feedback.

Workers flag unhelpful or problematic predictions, and their corrections further change the model's parameters,

making them more likely to give predictions that users prefer.

Looking back at the pre -training though, this staggering amount of computation is only made possible by using special computer chips

that are optimized for running many many operations in parallel, known as GPUs.

However, not all language models can be easily parallelized.

Prior to 2017, most language models would process text one word at a time.

But then, a team of researchers at Google introduced a new model known as the Transformer.

Transformers don't read text from the start to the finish.

they soak it all in at once, in parallel.

The very first step inside a transformer, and most other language models for that matter, is to associate each word with a long list of numbers.

The reason for this is that the training process only works with continuous values, so you have to somehow encode language using numbers.

and each of these list of numbers may somehow encode the meaning of the corresponding word.

What makes Transformers unique is their reliance on a special operation known as attention.

This operation gives all of these lists of numbers a chance to talk to one another, and refine the meanings that they encode based on the context around, all done in parallel.

For example, the numbers encoding the word "bank" might be changed based on the context surrounding it, to somehow encode the more specific notion of a riverbank.

Transformers typically also include a second type of operation, known as a feed -forward neural network,

and this gives the model extra capacity to store more patterns about language learned during training.

All of this data repeatedly flows through many different iterations of these two fundamental operations.

And as it does so, the hope is

that each list of numbers is enriched to encode whatever information

might be needed to make an accurate prediction of what word follows in the passage.

At the end, one final function is performed on the last vector in this sequence, which now has had a chance to be influenced by all the other context from the input text,

as well as everything the model learned during training, to produce a prediction of the next word.

Again, the model's prediction looks like a probability for every possible next word.

Although researchers design the framework for how each of these steps work, it's important to understand

that the specific behavior is an emergent phenomenon based on how those hundreds of billions of parameters are tuned during training.

This makes it incredibly challenging to determine why the model makes the exact predictions that it does.

What you can see is that when you use large language model predictions to autocomplete a prompt,

the words that it generates are uncannily fluent, fascinating, and even useful.

If you're a new viewer and you're curious about more details on how transformers and attention work, boy do I have some material for you.

One option is to jump into a series I made about deep learning, where we visualize and motivate the details of attention and all the other steps in a transformer.

But also, on my second channel I just posted a talk

that I gave a couple months ago about this topic for the company TNG in Munich.

Sometimes I actually prefer the content that I make as a casual talk rather than a produced video, but I leave it up to you which one of these feels like the better follow on.

you you

下載應用

流利口語所需的一切

AI 發音評分逐句打分

音標練習掌握每個音

詞彙學習建立你的詞庫

詞彙遊戲邊玩邊學

掃碼

Download on theApp Store Get it onGoogle Play

同頻道

Differential equations, a tourist's guide | DE1

Daily ConversationUniversity Students

Transformers, the tech behind LLMs | Deep Learning Chapter 5

Business EnglishUniversity StudentsWorking Adults

Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra

Cambridge Exam PrepUniversity StudentsWorking Adults

FOR YOU

關於本課

在本課中，學習者將通過分析大型語言模型的工作原理來提升英語理解能力。這段影片將揭示如何通過與人工智慧助手的對話來預測後續單詞。這種技術展示了語言模型如何處理文本並提供自然且流暢的響應。通過觀看影片並跟讀，學習者不僅可以擴充套件詞彙量，還能提升發音和語調的準確性，從而提高口語能力。您可以嘗試使用大語言模型的示例來進行英語影子跟讀，一邊學習一邊練習。

關鍵詞彙與短語

大型語言模型 (Large Language Model)
預測 (Predict)
引數 (Parameters)
上下文 (Context)
訓練 (Training)
線上文本 (Text from the internet)
注意力機制 (Attention Mechanism)
影子跟讀 (Shadow Speak)

練習建議

觀看這個影片時，建議您嘗試 shadow speak 的方法。首先，觀看影片的同時注意影片的節奏和語調。然後，暫停影片並重復模仿每一句話，注意發音和音調的變化。由於影片中的對話通常富有層次感，您可以將先前演示的段落分解成更小的部分，逐步練習。使用看YouTube學英語的方法，結合英語影子跟讀，不僅能提高口語流利度，還能幫助您更好地理解和運用複雜句子的結構。在每次練習後，可以回放影片，以便對比您自己的發音與原聲之間的細微差異。在日常學習中，堅持利用這些方法，將有助於大大提升您的英語表達能力。

影片中的語法

說話人最常用的結構，並附上影片中的原話：

結構	影片中的用法
被動語態 be + 過去分詞 — 強調發生了什麼，而不是誰做的	has been torn · is called · is only made
現在完成時 have/has + 過去分詞 — 過去發生但與現在仍有關聯的事	has been torn · has had

什麼是跟讀法？

跟讀法 (Shadowing) 是一種有科學依據的語言學習技巧，最初開發用於專業口譯員的培訓，並由多語言者Alexander Arguelles博士普及。這個方法簡單而強大：您在聽英語母語原聲的同時立即大聲重複——就像是一個延遲1-2秒緊跟說話者的影子。與被動聽力或語法練習不同，跟讀法強迫您的大腦和口腔肌肉同時處理並模模擬實的講話模式。研究表明它能顯著提高發音準確性，語調，節奏，連讀，聽力理解和口語流利度——使其成為雅思口語備考和真實英語交流最有效的方法之一。

影子跟讀法: 閱讀完整分步指南 →

跟讀練習: Large Language Models explained briefly - 通過影片學習英語口語

選擇練習模式

下載應用

同頻道

Differential equations, a tourist's guide | DE1

The Newton art puzzle

Transformers, the tech behind LLMs | Deep Learning Chapter 5

Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra

Fourier series

Euler's Formula Poem

Vectors | Chapter 1, Essence of linear algebra

How might LLMs store facts | Deep Learning Chapter 7

Essence of linear algebra preview

The inscribed square problem

Oh, wait, actually the best Wordle opener is not “crane”…

Hilbert's Curve: Is infinite math useful?

But what is a neural network? | Deep learning chapter 1

Divergence and curl: The language of Maxwell's equations, fluid flow, and more

But what is quantum computing? (Grover's Algorithm)

Tie random ends: How many loops?

推薦影片

World Models explained in 10min..

An ex-OpenAI researcher just deleted language from the LLM...

AI Agents, Clearly Explained

PyTorch in 100 Seconds

Computational Linguistics: Crash Course Linguistics #15

Prompt engineering - Introduction

Google’s AI Course for Beginners (in 10 minutes)!

You're Living Inside a Prediction

How to go from B1 to C2 in English (what actually works)

Introduction to statistical prediction models

關於本課

關鍵詞彙與短語

練習建議

影片中的語法

什麼是跟讀法？

跟讀練習: Large Language Models explained briefly - 通過影片學習英語口語

選擇練習模式

下載應用

Differential equations, a tourist's guide | DE1

The Newton art puzzle

Transformers, the tech behind LLMs | Deep Learning Chapter 5

Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra

Fourier series

Euler's Formula Poem

Vectors | Chapter 1, Essence of linear algebra

How might LLMs store facts | Deep Learning Chapter 7

Essence of linear algebra preview

The inscribed square problem

Oh, wait, actually the best Wordle opener is not “crane”…

Hilbert's Curve: Is infinite math useful?

But what is a neural network? | Deep learning chapter 1

Divergence and curl: The language of Maxwell's equations, fluid flow, and more

But what is quantum computing? (Grover's Algorithm)

Tie random ends: How many loops?

World Models explained in 10min..

An ex-OpenAI researcher just deleted language from the LLM...

AI Agents, Clearly Explained

PyTorch in 100 Seconds

Computational Linguistics: Crash Course Linguistics #15

Prompt engineering - Introduction

Google’s AI Course for Beginners (in 10 minutes)!

You're Living Inside a Prediction

How to go from B1 to C2 in English (what *actually* works)

Introduction to statistical prediction models

關於本課

關鍵詞彙與短語

練習建議

影片中的語法

什麼是跟讀法？

How to go from B1 to C2 in English (what actually works)