Process Query Systems -- Projects

Project Descriptions

• Autonomic Computing [HTML] | [PDF]
• Network Security [HTML] | [PDF]
• Airborne Plume Tracking [HTML] | [PDF]
• Social Networks [HTML] | [PDF]
• Visual Tracking [HTML] | [PDF]

The PQSNet display provides a graphical interface to the TRAFEN engine as it is applied to network attack data. Tracks and their respective scores are displayed for each of the hypotheses applied to each experiment listed on the lefthand side of the page. The experiments vary, ranging from observations of everything from Snort alerts to the movements of fish in an aquarium. PQSNet's tracking models generate predictive analyses of patterns in the data collected by sensors of various types.

The PQS fish-tracking system is a demonstration of the power and flexibility of the Process Query Systems engine. The same codebase that processes network attack data, also analyzes and predicts the movements of fish; the only difference between the two systems is the nature of the sensor input.

Correlated positions leave a trail with a history of 5 past positions. Positions that are correlated together are assigned that same color. Color assignments are arbitrary. Tracking is succesful if the trail behind the fish is all of the same color. If the trail switches color frequently, then the tracker is having difficulty tracking a fish.

The models predict the next position of the fish, although this is not shown due to the low resolution. This allows us to correctly keep track of the fish when they dissapear from view briefly. The predictions also help us disambiguate the fish when they cross paths.

(Live video feed coming soon.)

Fish tracking sensor video footage (QuickTime format):

Notes about the videos:

• Environment is a five-gallon tank with two Red Platys named Genghis and Napoleon.
• Tracks are superimposed over video in real-time.
• Each circle represents the current position of the target.
• Each light-blue cross represents a past observation.
• Each target has a different color circle.
• When the targets disappear behind an obstacle, a different-colored cross remains briefly at the point of the last observation, until the target is re-acquired or a specified timeout is reached.
• Sometimes the targets cross and their colors switch.
• Sometimes things in the room behind the tank are momentarily "seen" and tracked because of similarity in color to the real targets.

One of the major problems in visually tracking objects is the need for human interpretation of the data. For instance, consider a person being tracked using video images. When this person hides behind an obstruction, and is therefore no longer visible for the camera, conventional tracking systems quickly lose track of the person and are no longer able to indicate where the person is. A human interpreter is then needed to conclude that the person is hiding, and probably (with certain probability) is still there.

A Process Query System (PQS) is able to track and predict the path of arbitrary objects, based only on a description of their dynamic behavior, thus eliminating the need for precise identification of each object in every frame. The PQS is therefore able to draw human-like conclusions, allowing the system to track the person even when he/she is out of view. Additionally, using dynamic descriptions of tracked objects allows for low quality video signals, or even infra-red video, to be used for tracking.

Our data source for infrared tracking and classification experiments is a Thermal-Eye TSCss Long Wave Infrared camera produced by L3 Communications Infrared Products. It comes in many configurations; the camera used for these experiments has a fixed focus lens with a 50field-of-view and an operating range of +6to +40C. The output is a grayscale image of the scene which is output in NTSC-compatible format and captured by a Linux PC with an off-the-shelf framegrabber.

For the purposes of our experiments, we assume that the objects to be tracked will be emitting significantly more IR energy than their surroundings; because the camera's rendition of a scene depends upon contrast, we count on the fact that interesting objects will be rendered in bright white. We process the video stream frame-by-frame, and generate observations which the PQS tracks.

In processing a frame, we thus classify each pixel as being "in" or "out" -- that is, part of an object worth tracking, or not -- based on its brightness. Upon classifying pixels as in or out, we process the frame with a standard Union Find algorithm that locates connected groups of in pixels, and computes the centroid (in screen coordinates) and approximate mass of the group (in pixels). It is these two pieces of information that comprise the observation that will be processed by the PQS.

The PQS reports the top hypotheses (the tracks most likely to actually correspond with real, moving objects), along with a snapshot of their state (including their momentum and mass-momentum) back to the video processing system, which then annotates the video image with tracking information as well as indicators for whether a tracked object is growing in size (a large, positive mass momentum) and whether it seems to have a momentum that will put it on a collision course with another point. From the perspective of a camera pointed at a 3D scene, this can tell us whether an object is merely milling around aimlessly, or seems to be purposefully moving toward either the camera -- if it is growing in mass -- or towards some other sensitive zone (as indicated by its momentum).

More footage: (all .wmv format; you may have to right-click and save to disk before playing)

This project concerns the querying of networks of physical sensors and real time anomaly detection. Networks of sensing devices embedded in our world (weather, traffic, environmental, seismic, temperature) will soon produce massive amounts of real-time information and require new search methods.

The analysis of social networks puts the focus on discovering what roles people, or groups of people, play in a social structure by analyzing their actions over time. This means that it is possible, for instance, to discover what parts of a business are running efficiently, and what parts are running inefficiently, or to discover independently operating terrorist cells. Because a PQS uses models that describe changes over time, it is ideally suited to discover, and predict, the dynamics of a constantly changing social environment.

(More info coming soon.)

At least half the cost of an IT infrastructure is in the manpower required for setup and maintenance. Configuring a network with workstations and servers, and keeping it running, usually requires lots of man-hours and expertise; something that can be quite expensive. As networks are getting larger, and systems more complex, this cost factor is expected to increase drastically, especially considering that fewer and fewer numbers of new and skilled administrators are joining the workforce each year.

(More info coming soon.)