PhD Proposal: Towards Effective Temporal Modeling for Video Understanding and Beyond

ideo understanding is a fundamental research topic in computer vision, which requires extracting and analyzing information from videos automatically. Compared to the image modality, the video modality significantly differs from it with additional temporal dependencies, which provide crucial clues to help understand what happens across time. Therefore, how to effectively model temporal relationships for videos is of vital importance for video understanding.This thesis is divided into two parts. In the first part, we mainly concentrate on how to model the temporal dependencies for different video understanding tasks, including action recognition, temporal action localization, and video summarization. In the second part, beyond the pure understanding of video content, we focus on how to represent large-scale videos compactly and efficiently. We propose to encode diverse videos into a single neural network and explore its superior advantages in various video downstream tasks.

Examining Committee

Chair:

Dr. Abhinav Shrivastava

Department Representative:

Dr. Ramani Duraiswami

Members:

Dr. Furong Huang

Dr. Chen Sun