Streamlined Request Processing for an In-Memory DBMS
Date Issued
2016
Date
2016
Author(s)
Hsu, Kuo-Feng
Abstract
An in-memory database management system (DBMS) holds data in memory, and periodically flushes data to disk to protect them from power failures. Because data access is at main memory speed, the performance bottlenecks of an in-memory DBMS now shift to other system components that were previously considered non-critical, such as request/response packet processing, logging, and persistence. In this thesis, we describe the design, implementation and evaluation of a network-based in-memory DBMS that features a low-overhead network packet processing engine using Data Plane Development Kit (DPDK) and mTCP and allowing user-level applica-tions to manipulate TCP payloads, batched replication of in-memory memory logs, and a highly efficient database persistence mechanism called Batch mOdifications with Se-quential Commit (BOSC). To efficiently integrate these mechanisms into a coherent whole, we have experimented with multiple software architectures and eventually arrived at one that strikes a nice balance between request processing parallelism and threading overhead. A detailed performance study on a fully operational prototype shows that the resulting in-memory DBMS is able to sustain nearly one million requests per second when the memory is fully utilized and logging is turned on.
Subjects
in-memory data processing
internet of things
efficient TCP
remote logging
sequential disk access
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-105-R03921038-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):8b631d92cd8d42300a157420d34e0def