Pratique du Shadowing: Bebop: Accelerating LLM RL Training via MTP - Apprendre l'anglais à l'oral avec la vidéo

Choisissez un mode d'entraînement

Chaque mode ouvre une salle d'entraînement en plein écran.

46 phrases

Si les phrases sont trop courtes ou trop longues, cliquez sur Edit pour les ajuster.

Welcome to the AI Research Roundup.

I'm Alex.

A fascinating paper trending on X this week, published just three days ago on June 10, 2026, tackles one of the biggest bottlenecks in training modern AI.

As reinforcement learning becomes central to aligning large language models, accelerating the computationally expensive generation phase remains a critical open challenge.

To solve this, this paper introduces a method

that achieves up to a 1.8 times end-to-end training acceleration by changing how we handle multi-token prediction,

which is a speed-up technique where an AI model predicts several future words at once instead of just one.

The paper is titled Breaking Entropy Bounds, Accelerating Reinforcement Learning Training via Multi-Token Prediction with Rejection Sampling.

And as we will see, their approach completely bypasses the need for expensive online updates, making it highly practical for large-scale systems.

Figure 1 illustrates this core discovery by plotting how the number of accepted tokens changes as model entropy shifts.

In Figure 1 Panel A, we see that standard target-only sampling leads to a sharp linear drop in acceptance as entropy increases.

Our total variation loss, which measures the difference between probability distributions, completely flattens this curve, keeping the accept length consistently high.

Then, figure 1 panel B explains this visually, showing that the total variation draft model matches the sharp target distribution much better than the wider cross-entropy draft,

achieving an 85% overlap.

While figure 1 showed the theoretical impact of entropy, Figure 2 tracks actual acceptance rates during training on a software engineering benchmark.

In the first step of multi-token prediction, the acceptance rate declines by only about 1%.

However, this degradation accelerates in subsequent steps, because the third step drops by over 3% over time.

This compounding drop reveals why standard multi-token prediction struggles to maintain speed-ups during reinforcement learning.

Table 1 details why standard training objectives fall short.

The forward callback-Leibler divergence, shown in the second column, lacks tail suppression, which is a mechanism that prevents wasting training effort on extremely rare words.

Because of this, optimization is spread too thin across the entire vocabulary.

In contrast, both reverse callback-Leibler divergence

and the proposed total variation loss focus updates on relevant tokens by using gradients proportional to the draft probability.

Table 2 presents the actual acceptance rates across different task domains using QN 3.5,

where the end-to-end total variation loss consistently outperforms all other objectives.

On the software engineering tasks, our method increases the acceptance rate by 8% percent over the cross entropy baseline.

Even on out-of-distribution evaluation tasks, it provides a solid two percent improvement.

Figure 6 demonstrates how these improvements translate to reinforcement learning training.

In panel A, which shows reasoning workloads, the rejection sampling with total variation loss maintains the highest accept length throughout training.

Panel B and panel circa, representing software engineering workloads on larger models, confirm this trend because the total variation loss remains completely stable,

while target-only sampling steadily degrades.

Alright, figure 7 details how these stable except links translate to concrete speed-ups in wall clock training time.

Panel A shows that on reasoning tasks, the proposed rejection sampling with total variation loss reduces the latency

per training step by nearly half compared to the baseline without multi-token prediction.

Moving to panel B and panel C, similar latency reductions of up to 40% occur for software engineering and agent workloads,

which demonstrates consistent computational efficiency throughout the reinforcement learning phase.

Figure 8 illustrates how the choice of training loss affects the relationship between model entropy and token acceptance.

In panel A, which focuses on reasoning.

Standard target-only sampling and rejection sampling with cross-entropy show a steep decline in accept length as entropy rises.

In panel B and panel C, similar trends hold for software engineering.

Across all these tests, rejection sampling with total variation loss yields a nearly flat line, which confirms that total variation training successfully decouples the acceptance rate from entropy.

Decoupling token acceptance from model entropy via rejection sampling and total variation loss represents a major leap in accelerating reinforcement learning.

Bypassing expensive online updates makes speculative decoding highly practical for training large language models.

And that is a wrap.

I am Alex from the AI Research Roundup.

Thanks for tuning in.

Télécharger l'application

Tout pour parler couramment

Prononciation IAChaque phrase notée

Phonétique (IPA)Maîtrisez chaque son

VocabulaireEnrichissez votre lexique

Jeu de vocabulaireApprenez en jouant

Scanner le QR

Download on theApp Store Get it onGoogle Play

Même chaîne

Evidence Markets: Upgrading Prediction Markets

IELTS PrepTOEFL PrepUniversity Students

FOR YOU

Vidéo recommandée

182 phrases

Wharton’s Top AI Professor: Stop Starting Software Companies — The Real AI Money Is Here

Daily ConversationUniversity StudentsWorking Adults

321 phrases

Mixture of Experts Explained Visually: How Trillion-Parameter Models Actually Work

Daily ConversationUniversity Students

82 phrases

China just dropped a new frontier model... and nobody can explain how

Daily ConversationUniversity StudentsWorking Adults

Vocabulaire et conseils d’expression pour cette leçon

Cette leçon d’expression orale de niveau C1 s’appuie sur la vidéo « Bebop: Accelerating LLM RL Training via MTP ». Les mots qui reviennent le plus souvent : training, entropy, panel, variation, total. Cette vidéo contient 46 phrases et 706 mots à répéter en shadowing. La partie parlée dure 4:38. Le locuteur parle à un rythme naturel d’environ 152 mots par minute, proche d’une conversation courante. Seuls 67 % des mots font partie des 3 000 mots les plus courants en anglais, le vocabulaire est donc exigeant.

Vocabulaire clé de cette vidéo

Les 15 mots les plus avancés de la vidéo, avec leur prononciation et leur sens :

Mot	Prononciation	Sens
entropy nom	/ˈɛntɹəpi/	entropie
variation nom	/ˌvɛəɹiˈeɪʃn̩/	variation
token nom	/ˈtoʊkən/	symbole
reinforcement nom	/ˌɹiːɪnˈfɔːsmənt/	renfort
rejection nom	/ɹɪˈd͡ʒɛkʃən/	refus, rejet
prediction nom	/pɹɪˈdɪkʃən/	prédiction
accelerate verbe	/ɪkˈsɛl.əˌɹeɪt/	accélérer
workload nom		charge de travail
decouple verbe	[diːˈkʌpəɫ]	découpler
divergence nom	/daɪˈvɜː(ɹ)d͡ʒəns/	divergence
latency nom	/ˈleɪ.tən.si/	latence
baseline nom	/ˈbeɪslaɪn/	référence
illustrate verbe	/ˈɪl.əˌstɹeɪt/	illustrer
bypass nom	/ˈbaɪpæs/	contournement, déviation
translate verbe	/tɹænzˈleɪt/	traduire

Les verbes à particule que vous entendrez

Mot	Prononciation	Sens
speed up verbe	/spiːdˈʌp/	accélérer

Prononciation à surveiller

Les sons « sh » et « zh »: variation /ˌvɛəɹiˈeɪʃn̩/, rejection /ɹɪˈd͡ʒɛkʃən/, prediction /pɹɪˈdɪkʃən/, computationally /ˌkɑmpjʊˈteɪʃənəli/, degradation /ˌdɛɡɹəˈdeɪʃən/
Mots longs — placez bien l’accent: reinforcement /ˌɹiːɪnˈfɔːsmənt/, accelerate /ɪkˈsɛl.əˌɹeɪt/, probability /ˌpɹɑ.bəˈbɪl.ə.ti/, computationally /ˌkɑmpjʊˈteɪʃənəli/, speculative /ˈspɛkjuləˌtɪv/

Les sons difficiles pour les francophones :

/tʃ/ et /dʒ/ — à ne pas adoucir en « ch » et « j »: rejection /ɹɪˈd͡ʒɛkʃən/, divergence /daɪˈvɜː(ɹ)d͡ʒəns/
/r/ anglais — langue recourbée, sans frotter la gorge: entropy /ˈɛntɹəpi/, variation /ˌvɛəɹiˈeɪʃn̩/, reinforcement /ˌɹiːɪnˈfɔːsmənt/, rejection /ɹɪˈd͡ʒɛkʃən/, prediction /pɹɪˈdɪkʃən/

Comment s’entraîner avec cette vidéo

Écoutez la vidéo en entier une fois sans parler et notez les mots que vous ne connaissez pas.
Commencez à la vitesse 0,75×, répétez phrase par phrase, puis revenez à la vitesse normale quand cela devient facile.
Enregistrez-vous et comparez avec l’original, en faisant attention à des mots comme entropy, variation, token.

Qu'est-ce que la technique du Shadowing ?

Le Shadowing est une technique d'apprentissage des langues fondée sur la science, développée à l'origine pour la formation des interprètes professionnels. Le principe est simple mais puissant : vous écoutez de l'anglais natif et le répétez immédiatement à voix haute — comme une ombre suivant le locuteur avec un décalage de 1 à 2 secondes. Les recherches montrent une amélioration significative de la précision de la prononciation, de l'intonation, du rythme, des liaisons, de la compréhension orale et de la fluidité.

Technique du shadowing : lire le guide complet étape par étape →

Pratique du Shadowing: Bebop: Accelerating LLM RL Training via MTP - Apprendre l'anglais à l'oral avec la vidéo

Choisissez un mode d'entraînement

Télécharger l'application

Evidence Markets: Upgrading Prediction Markets

We Let $200 GPT-Pro Think for 7 Days on Unsolved Math

SAC | Soft Actor Critic (SAC) architecture | SAC Explained

Why Is There So Much AI Slop?

Claude Code Is Wasting Tokens... Microsoft Built a Fix

Harnessing generative AI in architecture, engineering & construction

LangChain in 5 Minutes (Explained Clearly)

How I'd Learn n8n if I had to Start Over in 2026

Why AI Tokens are so Expensive - Computerphile

Why the Arab World never united