Summary
ViPO turns one scalar reward per generated sample into spatially and temporally structured, pixel-level advantage maps, using pretrained vision features to focus GRPO updates on perceptually important regions while retaining the standard training pipeline.
Research paths
How this paper contributes to the site's broader research map.
- Trustworthy Visual Generation Post-TrainingCore work
ViPO uses perceptual features to convert a sample-level GRPO advantage into spatial or spatiotemporal advantages, changing where the optimization update is applied.
Research question
A scalar reward treats an image or video as one indivisible unit, so GRPO cannot explicitly direct optimization toward localized artifacts or fine-grained spatial and temporal cues.
What the paper contributes
- Introduces a GRPO variant that lifts scalar feedback into pixel-level structured advantages.
- Uses a Perceptual Structuring Module with pretrained vision backbones to build spatially and temporally aware advantage maps.
- Keeps the method lightweight, architecture-agnostic, and compatible with existing GRPO pipelines.
Evidence and evaluation scope
The paper reports improvements over vanilla GRPO on both image and video benchmarks, including in-domain preference alignment and out-of-domain generalization. Consult the official tables for exact rewards, datasets, backbones, and effect sizes.
Scope and limitations
The structured map is induced by pretrained visual features and therefore depends on what those backbones encode. Pixel-level optimization does not by itself guarantee that every reward model is reliable or causally localized.
Positioning for related work
ViPO belongs to work that increases reward granularity for generative-model RL. Its defining step is spatial–temporal redistribution of advantage rather than changing the base generator or replacing GRPO's overall optimization structure.
Related-work context
Ni et al. propose Visual Preference Policy Optimization (ViPO), which uses a perceptual structuring module to transform scalar rewards into spatially and temporally aware pixel-level advantages for GRPO-based image and video generation.
A concise, neutral description of how this paper can be situated in related work.
Official abstract
Reinforcement learning (RL) has become a powerful tool for post-training visual generative models, with Group Relative Policy Optimization (GRPO) increasingly used to align generators with human preferences. However, existing GRPO pipelines rely on a single scalar reward per sample, treating each image or video as a holistic entity and ignoring the rich spatial and temporal structure of visual content. This coarse supervision hinders the correction of localized artifacts and the modeling of fine-grained perceptual cues. We introduce Visual Preference Policy Optimization (ViPO), a GRPO variant that lifts scalar feedback into structured, pixel-level advantages. ViPO employs a Perceptual Structuring Module that uses pretrained vision backbones to construct spatially and temporally aware advantage maps, redistributing optimization pressure toward perceptually important regions while preserving the stability of standard GRPO. Across both image and video benchmarks, ViPO consistently outperforms vanilla GRPO, improving in-domain alignment with human-preference rewards and enhancing generalization on out-of-domain evaluations. The method is architecture-agnostic, lightweight, and fully compatible with existing GRPO training pipelines, providing a more expressive and informative learning signal for visual generation.
The abstract is reproduced for scholarly identification and remains under the paper publisher/authors’ original copyright; it is not covered by this page’s CC BY license.
Evidence references
| What to verify | Location in the paper |
|---|---|
| Problem statement | Abstract |
| Method and contributions | Abstract; Perceptual Structuring Module section |
| Evaluation statement | Abstract; image and video benchmark sections |
Primary source: CVF open-access paper (CVPR 2026 open-access version; arXiv:2511.18719v4 checked for revision date).
How to cite
Cite the paper—not this explainer—for scientific claims. Cite this page only when reusing its original commentary.
Ziqi Ni, Yuanzhi Liang, Rui Li, Yi Zhou, Haibin Huang, Chi Zhang, and Xuelong Li. “Seeing What Matters: Visual Preference Policy Optimization for Visual Generation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2026), 27260-27269.
Reuse policy
Original explanatory text on this page is licensed under CC BY 4.0 with attribution and a link to this page. Paper title, abstract, figures, and bibliographic metadata are excluded and retain their original rights. Creative Commons Attribution 4.0 International.