Frame Preprocessing

First, the Capture Video operator reads a new frame and sends it to the Multiplexer operator, which duplicates the received frame and resends it to two different operators. Then, the Mixture of Gaussians 2 Background Subtractor operator extracts the foreground area and outputs a binary image that represents the pixels that have the highest chance of being changed from previous frames to the current one. Because the result can have some additional noise, a sequence of components is applied to remove it.

Object Detection

After the denoised mask of a foreground is received, the Find Contours operator finds a set of different contours that describe the potential objects for the given frame. For each contour, the convex hull is detected in the Convex Hulls operator, and then for each convex hull, the bounding rectangles are found in the Bounding Rectangles operator. If there are any rectangles that overlap each other, they are merged in the Merge Overlapping operator. The extracted list of rectangles that describe the potential objects is then sent to the Shape Segmentation operator.

Object Segmentation

If there is any information about the objects detected in the past, the segmentation algorithm can be applied. Its role is to find the overlapping objects and estimate the best possible position based on the foreground mask, as well as information about the objects gathered in the previous frames. The output of the Shape Segmentation operator is a list of all of the objects detected and segmented in the current frame.

Object Tracking

After the objects are found, the last step is to recognize them based on the information from the previous frames, and to build an estimate of where the objects could be in the future. Then, the Draw Objects operator draws a rectangle of the specific color that represents each object, and the Draw Paths operator draws the path that the object made from the past to the current frame. Last, the results are displayed through the Motion JPG Stream operator and can be viewed with the Stream Viewer UI.