MulTa-HDC: A Multi-Task Learning Framework for Hyperdimensional Computing
Journal
IEEE Transactions on Computers
Journal Volume
70
Journal Issue
8
Pages
1269-1284
Date Issued
2021
Author(s)
DOI
ITCOB
Abstract
Brain-inspired Hyperdimensional computing (HDC) has shown its effectiveness in low-power/energy designs for edge computing in the Internet of Things (IoT). Due to limited resources available on edge devices, multi-task learning (MTL), which accommodates multiple cognitive tasks in one model, is considered a more efficient deployment of HDC. However, as the number of tasks increases, MTL-based HDC (MTL-HDC) suffers from the huge overhead of associative memory (AM) and performance degradation. This hinders MTL-HDC from the practical realization on edge devices. This article aims to establish an MTL framework for HDC to achieve a flexible and efficient trade-off between memory overhead and performance degradation. For the shared-AM approach, we propose Dimension Ranking for Effective AM Sharing (DREAMS) to effectively merge multiple AMs while preserving as much information of each task as possible. For the independent-AM approach, we propose Dimension Ranking for Independent MEmory Retrieval (DRIMER) to extract and concatenate informative components of AMs while mitigating interferences among tasks. By leveraging both mechanisms, we propose a hybrid framework of Multi-Tasking HDC, called MulTa-HDC. To adapt an MTL-HDC system to an edge device given a memory resource budget, MulTa-HDC utilizes three parameters to flexibly adjust the proportion of the shared AM and independent AMs. The proposed MulTa-HDC is widely evaluated across three common benchmarks under two standard task protocols. The simulation results of ISOLET, UCIHAR, and MNIST datasets demonstrate that the proposed MulTa-HDC outperforms other state-of-the-art compressed HD models, including SparseHD and CompHD, by up to 8.23% in terms of classification accuracy. © 1968-2012 IEEE.
Subjects
Brain-inspired computing; hyperdimensional computing; machine learning; multi-task learning
Other Subjects
Associative processing; Classification (of information); Economic and social effects; Internet of things; Learning systems; Linearization; Associative memory; Classification accuracy; Hybrid framework; Internet of thing (IOT); Memory overheads; Memory retrieval; Performance degradation; State of the art; Multi-task learning
Type
journal article