Publications

We introduce an LLM-driven motion planner that translates natural-language descriptions into explicit 3D object and camera trajectories for controllable video generation.

We introduce the YT360-EyeTracking dataset and spherical geometry-aware vision transformers that combine visual and spatial-audio cues for saliency prediction in 360-degree video.

We combine 3D Gaussian splatting, Neural ODE camera modeling, and hierarchical spatiotemporal learning for fast, memory-efficient, and temporally consistent video representation.

We present a unified framework that leverages HyperNetworks and CLIP for domain adaptation, image synthesis, and manipulation tasks using StyleGAN.

We propose CLIPAway, a method that harmonizes focused embeddings from CLIP to guide diffusion models for seamless object removal from images.

We investigate sequential compositional generalization capabilities in multimodal models, introducing the CompAct benchmark.

We improve event-based video reconstruction by employing hypernetworks that dynamically generate network parameters conditioned on event statistics.

We introduce ViLMA, a zero-shot benchmark designed to evaluate linguistic and temporal grounding capabilities of video-language models.

We leverage dataset cartography to improve compositional generalization in Transformer models.

We propose VidStyleODE, a method for disentangled video editing that combines StyleGAN with Neural ODEs for temporally consistent video manipulation.

We present CLIPInverter, a method that uses CLIP guidance for StyleGAN inversion, enabling text-driven editing of real images.

We propose a burst photography approach to enhance extremely dark images using learned multi-frame processing.

We introduce SLAMP, a stochastic video prediction model that disentangles appearance and motion in a latent space.

We propose mustGAN, a multi-stream GAN architecture for synthesizing MR images across different contrast types.

We introduce a cross-lingual visual pre-training approach to improve multimodal machine translation.

We propose belief-regulated dual propagation networks for learning the effects of actions on articulated multi-part objects.

We present an approach for manipulating attributes of natural scene images by hallucinating new visual content.

We introduce Procedural Reasoning Networks for understanding multimodal procedural content such as cooking recipes.

We propose a conditional GAN approach for synthesizing multi-contrast MRI images.

We introduce RecipeQA, a challenge dataset for evaluating multimodal comprehension of cooking recipes.

We propose spatio-temporal saliency networks for predicting dynamic visual saliency in videos.

We systematically re-evaluate automatic evaluation metrics for image captioning and analyze their correlation with human judgments.

We introduce TasvirEt, a benchmark dataset for automatic Turkish image description generation.

We propose a method to generate images of outdoor scenes conditioned on attributes and semantic layouts.

We present a deformable part-based tracking method using coupled global and local correlation filters.

We propose an objective quality metric for evaluating deghosting algorithms in HDR imaging.

We propose a sparse sampling approach for image matting based on KL-divergence.

We propose a distributed representation based query expansion approach for generating image captions.

We predict image memorability using attention-driven spatial pooling combined with image semantics.

We provide a comprehensive survey and evaluation of the state of the art in HDR deghosting.

We propose a top-down saliency estimation method using superpixel-based discriminative dictionaries.

We present a structure-preserving image smoothing method based on region covariances.

We propose a visual saliency estimation method that nonlinearly integrates features using region covariances.

We formulate graph transduction as a non-cooperative game and derive efficient algorithms for semi-supervised learning.