Modeling Guide

Object Tracking

The Object Tracking graph consists of various image processing algorithms and techniques divided into disparate components, which together create a real-time multiple object tracking system.

The algorithm is able to detect, segment, recognize, and track multiple objects in previously unknown conditions without any prior knowledge about the objects.

The algorithm can be divided into four main parts: frame preprocessing, object detection, object segmentation, and object tracking.

Frame Preprocessing

First, the Capture Video operator reads a new frame and sends it to the Multiplexer operator, which duplicates the received frame and resends it to two different operators. Then, the Mixture of Gaussians 2 Background Subtractor operator extracts the foreground area and outputs a binary image that represents the pixels that have the highest chance of being changed from previous frames to the current one. Because the result can have some additional noise, a sequence of components is applied to remove it.

Object Detection

After the denoised mask of a foreground is received, the Find Contours operator finds a set of different contours that describe the potential objects for the given frame. For each contour, the convex hull is detected in the Convex Hulls operator, and then for each convex hull, the bounding rectangles are found in the Bounding Rectangles operator. If there are any rectangles that overlap each other, they are merged in the Merge Overlapping operator. The extracted list of rectangles that describe the potential objects is then sent to the Shape Segmentation operator.

Object Segmentation

If there is any information about the objects detected in the past, the segmentation algorithm can be applied. Its role is to find the overlapping objects and estimate the best possible position based on the foreground mask, as well as information about the objects gathered in the previous frames. The output of the Shape Segmentation operator is a list of all of the objects detected and segmented in the current frame.

Object Tracking

After the objects are found, the last step is to recognize them based on the information from the previous frames, and to build an estimate of where the objects could be in the future. Then, the Draw Objects operator draws a rectangle of the specific color that represents each object, and the Draw Paths operator draws the path that the object made from the past to the current frame. Last, the results are displayed through the Motion JPG Stream operator and can be viewed with the Stream Viewer UI.


Operators in this graph use the following libraries: