In a little over a decade, billions of video streams captured worldwide (by wearables, vehicles, webcams, or urban cameras) will generate over an exapixel per second of information. This "big visual data" stream offers the promise of enabling new applications of computing that span many critical fields: autonomous robotics, transportation, medicine, public health and safety, retail, smart-city management, and entertainment. However, architecting systems that scale to analyze the world's visual data, and developing algorithms for intelligently interpreting the contents of often-on video while respecting privacy, stands as a significant computing challenge. The Intel Science and Technology Center for Visual Cloud Systems (ISTC-VCS) based at CMU was created in response to this challenge. The center brings together experts in hardware architecture, compilers, databases, distributed systems, and computer vision/machine learning to design new systems that will serve as the compute engines for new applications that efficiently analyze large repositories (or continuous streams) of video data at scale. The ISTC-VCS is pursuing research in the following areas:

Platforms for Large-Scale Video Analysis

The ISTC-VCS will create new, open platforms that make it simple for non-expert programmers to author large-scale video analysis applications. This includes offline analysis of large video databases and online analysis of video streams from thousands of cameras.

Performance-Motivated Algorithms for Video Analysis

The ISTC-VCS will develop new algorithms for processing video data more intelligently and more efficiently, while also respecting privacy. These efforts will exploit the structure of large-scale video datasets to avoid unnecessary computation, employ specialized video compression schemes, and continue to advance the state-of-the-art in video understanding.

Systems for Querying and Analyzing Video Datasets

"What is SQL for video?" This work pursues the design of new systems and programming abstractions for developing video analytics applications that operate on large datasets.

Specializing Hardware for Video Processing

The high compute cost of video-processing encourages the use of specialized hardware solutions. The ISTC-VCS will explore new hardware designs and new tool chains for mapping image analysis algorithms to the world's increasingly diverse array of accelerated computing solutions.