Logo: University of Southern California

Janus Will Monitor Multiple Data Flows to Look for Realtime Trouble

Viterbi scientists are developing a new technology, "event detection"

April 26, 2012 —

Janus: Roman god of gates, doors, doorways, beginnings, endings and time
A team from the USC Viterbi School or Engineering is working on improving a new safety technology called event detection, which attempts to catch crime as it occurs.

The goal is to make surveillance systems now widely used in many businesses, buildings and other public areas smarter: more able to recognize suspicious incidents immediately and automatically when they occur, and then go both forward and back in time to trace and to follow them. That is, the team wants to create an online tool that can integrate alarms and clues from multiple media rapidly into a form that helps police in real time.

Cyrus Shahabi, director of the USC Viterbi School of Engineering Integrated Media Systems Center (IMSC) is leading a $1 million two-year National Institute of Justice (NIJ) funded effort named Janus after the two-faced Roman god of gates, doors, doorways, beginnings, endings and time, portrayed looking simultaneously into the future and the past. The official title is “Multi Source Event Collection System for Effective Surveillance of Criminal Activity.” The plan is to build on and combine a variety of technologies that IMSC investigators, including Ram Nevatia, Gerard Medioni, Farnoush Banaei-Kashani and SeonHo Kim, have been working on for years. Shahabi, for example, has been involved for years on a geospatial decision-making system called GeoDec.

Cyrus Shahabi

“The core idea behind this proposal,” says the plan, “is to utilize state-of-the-art content analysis techniques for extracting events from streams of incidents, while cross referencing these events in space and time (rather than content) to present a holistic scheme to a human operator.” The Viterbi team is collaborating with USC’s Department of Public Safety (DPS) to evaluate the performance of the tool. IMSC has been working for more than a year on its GeoImmersion project, an effort to apply sophisticated IT techniques to delineated geographical areas. USC is its designated area;

Gerard Medioni
the Janus system will be tested there.

Janus’ focus is event detection, or pulling evidence of a specific kind of occurrence – in this case, a crime out of a welter of data, mostly images, but also some text streaming through multiple sensor inputs. One stimulus is unambiguous and relatively easy to set and automate: a gunshot, in a Janus category by itself. Other events, like a "purse snatch + fleeing fugitive" are much more difficult to characterize for automatic capture, particularly when one aim is to capture a recognizable facial image of a fleeing fugitive.

Ram Nevatia
The proposal for the grant sets forth a strategy for both stages of these harder categories event recognition. An early, relatively simple example is monitoring traffic and interpreting cars' motion to infer specific violations. From there, the USC team is attempting to move on to inferring human actions.

“Our approach is to decompose complex actions into simpler actions in a hierarchical way. We call the simplest actions to be primitive actions (e.g., walking); these are inferred directly from the observables such as position, velocity and appearance of objects. More complex actions are composed of the primitive ones, or other, less complex, composite actions. This representation leads naturally to recognition by use of statistical graphical models.”

Farnoush Banaei-Kashan
The proposal presents a demonstration of this architecture being used to detect and infer on common human activity: people meeting in a terminal, starting from the basic observables and then using probabilistic elimination of false leads.

Once an event is identified, the facial identification component depends on another new technology that the Viterbi team has been working on, which is automatic computer control of a pan-zoom-tilt (PZT) video camera to follow a target. The laboratory of one of the researchers (Medioni) has been working on this problem successfully for years; the Janus system will incorporate elements of this technology.

Janus will also use text streams as a source of events, but with a special kind of search engine: “We intend to develop efficient text data indexing techniques that allow for identifying the search criteria textually (i.e., by giving a set of keywords), spatially (i.e., by identifying a spatial region or point) and temporally (i.e.. by identifying a time interval), and receive text data items that are most relevant in all three dimensions to the search criteria.”

Seanho Kim
Shahabi’s InfoLab has been working on these kinds of advanced text search incident identifications.

Finally, “Janus deploys novel algorithms that use advanced spatial and temporal reasoning approaches to connect-the-dots, i.e., to recognize activities out of the incidents that happen in large areas and over long timeframes,” says Banaei-Kashani, another investigator who also serves as the project manager.

In addition to security applications, Janus technology may also be able to “advance several decision-making applications, such as disaster response, geo-realistic simulations, games and training, urban planning, urban public health, that would otherwise fail due to the overwhelming amount of information from several independent streams of data,” according to the project proposal.

Finally, the team members promised they will try to get USC students involved. “We will introduce Janus-specific research projects into the USC’s Research Experiences for Undergraduates (REU) site, which supports research participation by undergraduate students in CS research and the USC Women in Science and Engineering program.”