Papers
arxiv:2611.18550

Restoring Diversity in Distilled Image and Video Generators by Probing Activations

Published on Nov 13
· Submitted by
Ozgur Kara
on Nov 16
Authors:
,
,
,
,
,

Abstract

Distilled few-step diffusion models exhibit diversity collapse because activation variance is concentrated into fewer directions in an intermediate block at the initial denoising step, and ReDiverse restores sample diversity by reintroducing teacher variance directions through adapter fine-tuning or training-free activation steering while preserving generation quality.

Distilling diffusion models into few-step student models substantially reduces sampling latency, but induces severe diversity collapse: samples generated from different random seeds show minimal variation. In this work, we investigate the root cause of this failure by probing the internal representations of student models relative to their teachers. We find that distilled students maintain total activation variance comparable to their teachers, but concentrate it into substantially fewer directions. This directional collapse is localized to an identifiable intermediate block at the initial denoising step. Consequently, diversity collapse stems from a misallocation of activation variance rather than an inherent loss of generative capacity. Guided by this finding, we propose ReDiverse to reinstate the missing variance directions. At training time, ReDiverse-FT fine-tunes a lightweight adapter that regresses student activations onto the teacher's, reviving the teacher's feature variance characteristic. At inference time, ReDiverse-TF introduces a training-free mechanism that steers activations along precomputed teacher principal directions. Across image and video distillation benchmarks, ReDiverse consistently restores sample diversity while preserving generation quality, further supported by the user study.

Community

Similar quality, higher diversity!

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Could this be done with weight space programming?

Hi authors, thanks for the work. Two small questions on v1 (issues are disabled on the repo, so asking here):

  1. Sec. 5.2 says ReDiverse surpasses Noise Optimization on all diversity metrics except Vendi for SD3. In Table 4a this holds for SD3, but on SDXL Noise Opt. is ahead on all five (DINO 67.0 vs 65.0, DS 37.9 vs 33.3, LP 71.2 vs 67.9, TP 25.1 vs 23.0, Vendi 7.83 vs 7.03).
  2. Table 5a, Regress at all blocks: TPIPS_avg 26.6 equals that row's overall TPIPS in Table 8, while every other row's TPIPS_avg is 6 to 9 points below its Table 8 value (e.g. default 15.4 vs 22.4, teacher 21.7 vs 30.2). It is also the only row without a delta. Could that cell be the overall value?

Short check script: https://github.com/checkpaper/audits/tree/main/2026/2611.18550-rediverse

For context, more and more papers are hard to review carefully, so we are building tools that help readers check the numbers. Full write-up: https://checkpaper.app/r/82em7qft78?utm_source=hf

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.18550
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 6

Browse 6 models citing this paper

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.18550 in a dataset README.md to link it from this page.

Spaces citing this paper 1

Collections including this paper 1