シャドーイング練習: Bebop: Accelerating LLM RL Training via MTP - 動画で英語スピーキングを学ぶ
練習モードを選ぶ
各モードは全画面の練習ルームで開きます。
46 文
文が短すぎたり長すぎる場合は、Editをタップして調整してください。
1
Welcome to the AI Research Roundup.
2
I'm Alex.
3
A fascinating paper trending on X this week, published just three days ago on June 10, 2026, tackles one of the biggest bottlenecks in training modern AI.
4
As reinforcement learning becomes central to aligning large language models, accelerating the computationally expensive generation phase remains a critical open challenge.
5
To solve this, this paper introduces a method
6
that achieves up to a 1.8 times end-to-end training acceleration by changing how we handle multi-token prediction,
7
which is a speed-up technique where an AI model predicts several future words at once instead of just one.
8
The paper is titled Breaking Entropy Bounds, Accelerating Reinforcement Learning Training via Multi-Token Prediction with Rejection Sampling.
9
And as we will see, their approach completely bypasses the need for expensive online updates, making it highly practical for large-scale systems.
10
Figure 1 illustrates this core discovery by plotting how the number of accepted tokens changes as model entropy shifts.
11
In Figure 1 Panel A, we see that standard target-only sampling leads to a sharp linear drop in acceptance as entropy increases.
12
Our total variation loss, which measures the difference between probability distributions, completely flattens this curve, keeping the accept length consistently high.
13
Then, figure 1 panel B explains this visually, showing that the total variation draft model matches the sharp target distribution much better than the wider cross-entropy draft,
14
achieving an 85% overlap.
15
While figure 1 showed the theoretical impact of entropy, Figure 2 tracks actual acceptance rates during training on a software engineering benchmark.
16
In the first step of multi-token prediction, the acceptance rate declines by only about 1%.
17
However, this degradation accelerates in subsequent steps, because the third step drops by over 3% over time.
18
This compounding drop reveals why standard multi-token prediction struggles to maintain speed-ups during reinforcement learning.
19
Table 1 details why standard training objectives fall short.
20
The forward callback-Leibler divergence, shown in the second column, lacks tail suppression, which is a mechanism that prevents wasting training effort on extremely rare words.
21
Because of this, optimization is spread too thin across the entire vocabulary.
22
In contrast, both reverse callback-Leibler divergence
23
and the proposed total variation loss focus updates on relevant tokens by using gradients proportional to the draft probability.
24
Table 2 presents the actual acceptance rates across different task domains using QN 3.5,
25
where the end-to-end total variation loss consistently outperforms all other objectives.
26
On the software engineering tasks, our method increases the acceptance rate by 8% percent over the cross entropy baseline.
27
Even on out-of-distribution evaluation tasks, it provides a solid two percent improvement.
28
Figure 6 demonstrates how these improvements translate to reinforcement learning training.
29
In panel A, which shows reasoning workloads, the rejection sampling with total variation loss maintains the highest accept length throughout training.
30
Panel B and panel circa, representing software engineering workloads on larger models, confirm this trend because the total variation loss remains completely stable,
31
while target-only sampling steadily degrades.
32
Alright, figure 7 details how these stable except links translate to concrete speed-ups in wall clock training time.
33
Panel A shows that on reasoning tasks, the proposed rejection sampling with total variation loss reduces the latency
34
per training step by nearly half compared to the baseline without multi-token prediction.
35
Moving to panel B and panel C, similar latency reductions of up to 40% occur for software engineering and agent workloads,
36
which demonstrates consistent computational efficiency throughout the reinforcement learning phase.
37
Figure 8 illustrates how the choice of training loss affects the relationship between model entropy and token acceptance.
38
In panel A, which focuses on reasoning.
39
Standard target-only sampling and rejection sampling with cross-entropy show a steep decline in accept length as entropy rises.
40
In panel B and panel C, similar trends hold for software engineering.
41
Across all these tests, rejection sampling with total variation loss yields a nearly flat line, which confirms that total variation training successfully decouples the acceptance rate from entropy.
42
Decoupling token acceptance from model entropy via rejection sampling and total variation loss represents a major leap in accelerating reinforcement learning.
43
Bypassing expensive online updates makes speculative decoding highly practical for training large language models.
44
And that is a wrap.
45
I am Alex from the AI Research Roundup.
46
Thanks for tuning in.
アプリをダウンロード
流暢に話すために必要なすべて
AI発音採点文ごとに採点
IPA練習すべての音をマスター
単語学習自分の単語帳を作る
単語ゲーム遊びながら覚える
同じチャンネル
FOR YOU
おすすめ動画
C12
182 文
We Let $200 GPT-Pro Think for 7 Days on Unsolved Math
Daily ConversationUniversity StudentsWorking Adults
C113
202 文
SAC | Soft Actor Critic (SAC) architecture | SAC Explained
Daily ConversationUniversity StudentsWorking Adults
C12
11 文
Why Is There So Much AI Slop?
Daily ConversationUniversity StudentsWorking Adults
C11
119 文
Claude Code Is Wasting Tokens... Microsoft Built a Fix
Daily ConversationUniversity StudentsWorking Adults
C11
114 文
Harnessing generative AI in architecture, engineering & construction
Daily ConversationUniversity StudentsWorking Adults
C11
74 文
LangChain in 5 Minutes (Explained Clearly)
Daily ConversationUniversity StudentsWorking Adults
C12
272 文
How I'd Learn n8n if I had to Start Over in 2026
American AccentDaily ConversationUniversity StudentsWorking Adults
C11
407 文
Why AI Tokens are so Expensive - Computerphile
Daily ConversationUniversity StudentsWorking Adults
C12
171 文
Why the Arab World never united
Daily ConversationUniversity StudentsWorking Adults
C11
995 文
Wharton’s Top AI Professor: Stop Starting Software Companies — The Real AI Money Is Here
Daily ConversationUniversity StudentsWorking Adults
C11
321 文
Mixture of Experts Explained Visually: How Trillion-Parameter Models Actually Work
Daily ConversationUniversity Students
C12
82 文
China just dropped a new frontier model... and nobody can explain how
American AccentDaily ConversationUniversity StudentsWorking Adults
コンテキストと背景
このビデオでは、AI研究の最新のトピックが紹介されています。話者は、モダンAIのトレーニングにおける大きなボトルネックの解消方法について論じています。特に、強化学習が大規模言語モデルの整合性を確保するために重要である一方、その計算コストが高い生成フェーズを加速する方法に焦点を当てているのが特徴です。このような背景を理解することで、英語を学ぶ私たちも最新の技術について情報を得ることができるため、日常会話の一環として役立ちます。
日常コミュニケーションのための5つのフレーズ
- この論文は、AIトレーニングのボトルネックに挑むものです。
- 我々の手法は、エンドツーエンドでのトレーニング加速を実現します。
- 標準的なターゲットのサンプリングと比較して、パフォーマンスが向上します。
- トレーニングステップごとに、レイテンシをほぼ半分に削減できます。
- 新しい手法により、受け入れ率を安定させることができます。
ステップバイステップ・シャドウイングガイド
このビデオの内容を効果的に理解し、英語スピーキング練習を行うために、以下の手順に従いましょう。
- ビデオを視聴し、自然なリズムを捉えます。話者の声のトーンや発音に耳を傾け、どのように単語を強調しているかを学びます。
- 短いフレーズを選びます。上の「日常コミュニケーションのための5つのフレーズ」から1つを選び、繰り返し声に出してみましょう。
- 逐次的に繰り返し練習します。最初はビデオの音声に合わせて、次に自分だけで言ってみます。
- 自分の発音を録音してみます。シャドウスピーキングの練習を通じて、自分の発音を確認し、修正点を見つけることができます。
- 練習を続け、他のフレーズにも挑戦します。自信を持って話せるようになるまで、リピートを続けましょう。
このプロセスを通じて、あなたの shadow speech（シャドウスピーチ）能力が向上し、英語スピーキング練習においても自信が持てるようになります。日々の練習を積み重ねながら、shadowspeakの技術を磨いていきましょう。
シャドーイングとは？英語上達に効果的な理由
シャドーイング（Shadowing）は、もともとプロの通訳者養成プログラムで開発された言語学習法で、多言語習得者として知られるDr. Alexander Arguelles によって広く普及されました。方法はシンプルですが非常に効果的：ネイティブスピーカーの英語を聞きながら、1〜2秒の遅延で声に出してすぐに繰り返す——まるで「影（shadow）」のように話者を追いかけます。文法ドリルや受動的なリスニングと異なり、シャドーイングは脳と口の筋肉が同時にリアルタイムで英語を処理・再現することを強制します。研究により、発音精度、抑揚、リズム、連音、リスニング力、そして会話の流暢さが大幅に向上することが確認されています。IELTSスピーキング対策や自然な英語コミュニケーションを目指す方に特におすすめです。