Vision Sensing Techniques for Intelligent Surveillance System

With the development of intelligent surveillance systems, video analysis, and recognition technology have become the most important core techniques in this field. In order to construct a surveillance system with higher intelligence, this research proposes a number of advanced video recognition technologies, including the camera interference/tampering detection, pedestrian detection, abandoned luggage detection, pedestrian re-identification and intelligent interface for visualization. Video surveillance uses cameras as the primary input sensor to achieve automatic monitoring. Therefore, how to protect the camera has become the top priority. We propose real-time camera sabotage/tampering detection technology which quickly detects whether or not cameras are hindered by deliberate shelter, disorientation, out of focus, disconnection and other damage via the video analysis. We initially locate the key points whose appearances are relatively stable. Monitoring the changes of these key points and scene structure can detect the tampering events precisely and efficiently. Our method requires lower computational cost and obtains higher stability and accuracy rate in comparison to the existing methods. After protecting cameras, we propose a scene-specific pedestrian detection and object classification. Our approach is location-based, which cab discover scene-dependent discriminative features to identifying foreground objects of different categories (e.g., pedestrians, bicycles, and vehicles). We incorporate a similarity grouping procedure capable of gathering more consistent training examples from a considerably larger neighbor area and train the　specific pedestrian detectors for each grouped local area. Our approach gets significant improvement in detection and classification comparing the traditional generic object detector and classifier. Also, we propose an ensemble of invariant features (EIF), which can properly handle the color variations and human poses/viewpoints for matching pedestrian images observed in different cameras. Our proposed method belongs the direct method, which requires no domain learning. The novel features combined both the holistic and region-based features. The holistic features are extracted by using a publicly available pre-trained deep convolutional neural network (DCNN) used in generic object classification. In contrast, the region-based features are extracted based on our proposed two-way Gaussian Mixture Model fitting (2WGMMF), which overcomes the self-occlusion and poses variations. In addition to the appearance feature, the face information is undoubtedly the indispensable vital in video surveillance. We propose a 3D face alignment algorithm in the 2D image based on Active Shape Model. We off-line train a 3D shape model with different view-based local texture models from a 3D database, and then on-line fit a face in a 2D image by these models. This method mainly leverages additional depth information on the traditional 2D image alignment problem and gets a promising improvement compared to the existing model-based and regression-based approaches. Since the human poses, and their gaze directions are especially valuable information to the surveillance system, the head poses can be directly estimated by the alignment result of the proposed 3D model subsequently. Based on the robust pedestrian detection and re-identification algorithm, we also focus the problem of event detection in surveillance cameras. We take the abandoned luggage detection as an example since it is one of the most critical and challenge problems in video surveillance. We propose the complementary background model which combines short- and long-term background models to classify each pixel as 2-bit code where each bit represents a foreground or background. Subsequently, we introduce a finite-state machine framework to identify static foreground regions based on the temporal transition of code patterns and to determine whether the selected area contain abandoned objects by analyzing the back-traced trajectories of luggage owners. The experimental results obtained based on video images from 2006 Performance Evaluation of Tracking and Surveillance (PETS2006), 2007 Advanced Video, Signal-based Surveillance (AVSS2007) databases and NTU data set collected by ourselves. We show that the proposed approach is useful for detecting abandoned luggage and that it outperforms previous methods. Finally, based on the above core technologies, we also propose two advanced visualization interface, which facilitates people to observe quickly and search incidents of pedestrians within a camera network.