Data compression and query for large scale sensor data on COTS DBMS
Journal
Proceedings of the 15th IEEE International Conference on Emerging Technologies and Factory Automation, ETFA 2010
Date Issued
2010
Author(s)
Abstract
Multi-dimensional temporal data set is the common format in sensor network applications to store sampled temporal data. As time goes on, the size of the core tables in the data set may increase to enormous size and the tables become not managable. In order to reduce storage space and allow on-line query, how to trade off data compression effectiveness for on-line query performance is a challenge issue. In this paper, we are concerned with an effective framework for temporal data set that does not scarify on-line query performance and is specifically designed for very large sensor network database. The sampled data are compressed using several candidate approaches including dictionary-base compress and lossless vector quantization. In the mean time, on-line queries are conducted without decompressing the compressed data set so as to enhance the query performance. Experiments are conducted on a power meter database and sonoma database to evaluate the proposed methodologies in terms of data compression rate and data query speed. The results show that the compression rate ranges from 70% for numerical data to 20% for character data. In the mean time, the increased overhead for online query is limited up to 2%. ©2010 IEEE.
Subjects
Data compression; Sensor network; Sensor network applications
Other Subjects
Common format; Compression rates; Data query; Data sets; Large-scale sensors; Lossless; Network database; Numerical data; Power meters; Query performance; Sampled data; Sensor network applications; Storage spaces; Temporal Data; Trade off; Database systems; Factory automation; Sensor networks; Vector quantization; Data compression
Type
conference paper