We systematically study attention sinks in bidirectional diffusion and causal autoregressive text-to-image transformers, revealing architecture- and stage-dependent patterns through targeted interventions and register-token experiments.

We present Diff-Aid, a plug-in module for rectified text-to-image diffusion transformers that adaptively adjusts block-wise interactions between token-level textual conditions across denoising trajectories.

We introduce the Weighted Diversity Score to quantify block-wise representation diversity and propose DiverseDiT++, which improves diffusion transformer training across image generation, protein inverse folding, and 3D molecule generation.

We propose TexTailor, an inference-time method that tailors block-wise textual guidance in MMDiTs to improve text-image alignment while enabling precise editing and accelerated inference.

We propose SIPO, a timestep-aware preference optimization framework that stabilizes diffusion alignment through gradient clipping and masking with importance reweighting, consistently improving image and video generation models.

We propose DiverseDiT, an efficient framework that enhances representation diversity with long residual connections and a representation diversity loss, without relying on external guidance.

We present Omni-Video 2, a scalable and computationally efficient model that connects pretrained multimodal large-language models (MLLMs) with video diffusion models for unified video generation and editing. Our key idea is to exploit the understanding and reasoning capabilities of MLLMs to produce explicit target captions to interpret user instructions.

We present Omni-Video, an efficient and effective unified framework for video understanding, generation, as well as instruction-based editing. Our key insight is to teach existing multimodal large language models (MLLMs) to produce continuous visual clues that are used as the input of diffusion decoders, which produce high-quality videos conditioned on these visual clues.

We introduce Uni-CoT, a unified reasoning framework that extends CoT principles to the multimodal domain, empowering Multimodal Large Language Models (MLLMs) to perform interpretable, step-by-step reasoning across both text and vision.

We introduce Dual-Iterative Optimiza tion (Dual-IPO), an iterative paradigm that sequentially optimizes both the reward model and the video generation model for improved synthesis quality and human preference alignment. For the reward model, our framework ensures reliable and robust reward signals via CoT-guided reasoning, voting-based self-consistency, and preference certainty estimation.

We propose FreeCus, a training-free framework for zero-shot subject-driven image generation using diffusion transformers (DiTs). The method introduces three innovations: (1) pivotal attention sharing (PAS) to preserve subject layout while maintaining editability, (2) adjusted noise shifting (ANS) to enhance fine-grained detail extraction, and (3) semantic feature compensation (SFC) via multimodal LLMs to address missing attributes.

We develop Cockatiel, a three stage training pipeline which assembles the advantages of multiple video-detailed caption (VDC) models and human preferences by conduct ing ensembling synthetic and human preference training on VDC models based on our caption quality scorer.

We propose an effective approach for incorporating LLMs into text-to-image diffusion models, improving the awareness of LLMs towards the CLIP visual and textual space, thus facilitating more expressive language understanding. Moreover, we devise an efficient three-stage training pipeline that accomplish fast adaptation of LLM textual features with a small amount of resources, serving as an strong baseline of integrating LLMs into diffusion models and paving the way of this important topic.

We propose an attention calibration mechanism to improve the concept-level understanding of the T2I model. Specifically, we first introduce new learnable modifiers bound with classes to capture attributes of multiple concepts. Then, the classes are separated and strengthened following the activation of the cross-attention operation, ensuring comprehensive and self-contained concepts. Additionally, we suppress the attention activation of different classes to mitigate mutual influence among concepts.

We make in-depth analyses on how to represent a data point in the feature space, how to calculate a fair distance using selected samples, and how many instances to use from each set. Together with these analysis, we build a comprehensive system for synthesis comparison, providing reliable and consistent ranks for unsupervised image generation models including GANs and Diffusion Models.

We provide a comprehensive survey on image synthesis under limited data, including data-efficient generative modeling, few-shot generative adaptation, few-shot and one-shot image synthesis.

We propose textural modulation (TexMod) and strctural discriminator (StructD) for improving the performance of few-shot image generaion.

We propose semantic-aware generator (SAG) and low-level feature augmentation (LFA) for improving the performance of few-shot image generaion.

we propose ProtoGAN, a GAN that incorporates the metric-learning-based prototype mechanism into adversarial learning by aligning the prototypes and features of synthesized distribution and the real distribution.

we propose DFSGAN for few-shot image generation, which takes dynamic Gaussian mixture (DGM) latent codes as the generator’s input.

We propose a frequency-aware model for training GANs under limited data, facilitating high-quality few-shot image syntheisi.

We propose a frequency-aware model for few-shot image generation, enabling high-fidelity synthesis for downstream tasks.

We develop Better Embedding and More Shots to address the distorted embedding of target data in few-shot learning.