Deep Neural Networks on Video Sensor Networks:Quantized and Distributed Approaches
Date Issued
2015
Date
2015
Author(s)
Hung, Pei-Hen
Abstract
As better performance is achieved by deep convolutional networks with more and more layers, the increasing number of computational workload and weighting parameters makes it only possible to be implemented on servers in cyber space but infeasible to be deployed in physical-world embedded systems because of huge storage and memory bandwidth requirements. In this thesis, we proposed two methods to bridge the gap between deep learning and physical-world. First, we introduced an efficient method to quantize the model parameters. Instead of taking the quantization process as a negative effect on precision, we regarded it as a regularization problem to prevent overfitting, and a two-stage quantization technique including soft- and hard-quantization is developed. With the help of our quantization method, not only 93.75\% of the parameter memory size can be reduced by replacing the word length from 32-bit floating point to 2-bit fixed point, but also keep the testing accuracy after quantization with acceptable drop or even better than previous approaches in some dataset, and the additional training overhead is only 3\% of the ordinary one. Second, to fit the scenario of video sensor networks, a distribute model is proposed to distribute the computational effort into servers and sensor nodes. With the additional analysis engine on sensor nodes, redundant video sequence can be filtered out in early step, transmission bandwidth and data storage will be saved. At the same time, with the auto-encoder in deep neural networks model, we can further reduce more than 90\% transmission bandwidth between the distributed sensors and servers with less than 1\% in classify accuracy drop on the server side.
Subjects
Deep Neural Networks
Video Sensor Networks
Quantized
Distributed
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-104-R02943122-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):ce6b3f433521ed8da0f539cfc83f1619