Summary
FreeLong extends pretrained short-video diffusion models without additional training by blending low-frequency global features with high-frequency local-subsequence features during denoising, targeting global consistency and local spatiotemporal detail at once.
Research paths
How this paper contributes to the site's broader research map.
- Video Generation and World ModelsCore work
FreeLong extends pretrained short-video diffusion models through frequency-aware temporal feature blending, addressing duration without retraining the base generator.
Research question
Directly running a short-video diffusion model on a much longer sequence degrades quality; the paper associates this with reduced spatial high-frequency content and excessive temporal high-frequency content.
What the paper contributes
- Diagnoses a frequency-distribution distortion when short-video models are extended to long sequences.
- Introduces a training-free blend of low-frequency global and high-frequency local video features during denoising.
- Supports coherent long and multi-prompt generation without retraining the base model.
Evidence and evaluation scope
The paper evaluates FreeLong on multiple base video diffusion models and reports improved consistency and fidelity, including multi-prompt transitions. Exact frame lengths, base models, and metrics are experiment-specific.
Scope and limitations
The training-free method depends on the capabilities of the pretrained short-video model and the chosen global/local feature decomposition. It extends temporal range but does not remove all long-horizon semantic or physical-consistency limitations.
Positioning for related work
FreeLong is an inference-time, training-free approach to long-video generation. It differs from retraining or distillation methods by modifying temporal feature mixing during denoising.
Related-work context
Lu et al. propose FreeLong, a training-free long-video generation method that blends low-frequency global features with high-frequency local features during denoising to extend short-video diffusion models.
A concise, neutral description of how this paper can be situated in related work.
Official abstract
Video diffusion models have made substantial progress in various video generation applications. However, training models for long video generation tasks require significant computational and data resources, posing a challenge to developing long video diffusion models. This paper investigates a straightforward and training-free approach to extend an existing short video diffusion model (e.g. pre-trained on 16-frame videos) for consistent long video generation (e.g. 128 frames). Our preliminary observation has found that directly applying the short video diffusion model to generate long videos can lead to severe video quality degradation. Further investigation reveals that this degradation is primarily due to the distortion of high-frequency components in long videos, characterized by a decrease in spatial high-frequency components and an increase in temporal high-frequency components. Motivated by this, we propose a novel solution named FreeLong to balance the frequency distribution of long video features during the denoising process. FreeLong blends the low-frequency components of global video features, which encapsulate the entire video sequence, with the high-frequency components of local video features that focus on shorter subsequences of frames. This approach maintains global consistency while incorporating diverse and high-quality spatiotemporal details from local videos, enhancing both the consistency and fidelity of long video generation. We evaluated FreeLong on multiple base video diffusion models and observed significant improvements. Additionally, our method supports coherent multi-prompt generation, ensuring both visual coherence and seamless transitions between scenes.
The abstract is reproduced for scholarly identification and remains under the paper publisher/authors’ original copyright; it is not covered by this page’s CC BY license.
Evidence references
| What to verify | Location in the paper |
|---|---|
| Problem statement | Abstract |
| Method and contributions | Abstract; SpectralBlend Temporal Attention method section |
| Evaluation statement | Abstract; experiments on multiple base video diffusion models |
Primary source: NeurIPS proceedings record (NeurIPS 2024 proceedings version; arXiv:2407.19918v1 cross-checked).
How to cite
Cite the paper—not this explainer—for scientific claims. Cite this page only when reusing its original commentary.
Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. “FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention.” Advances in Neural Information Processing Systems (2024), 37, 131434-131455. https://doi.org/10.52202/079017-4177.
Reuse policy
Original explanatory text on this page is licensed under CC BY 4.0 with attribution and a link to this page. Paper title, abstract, figures, and bibliographic metadata are excluded and retain their original rights. Creative Commons Attribution 4.0 International.