薛智文臺灣大學:資訊工程學研究所紀忠毅Chi, Chung-YiChung-YiChi2010-06-092018-07-052010-06-092018-07-052009U0001-2707200914144000http://ntur.lib.ntu.edu.tw//handle/246246/185439數位化的時代來臨， 伴隨網際網路普及的推波助瀾， 數位音樂呈現爆炸式的成長， 人們接觸音樂的方式也漸趨多元。 特別的是， 利用音樂情緒來組織與搜尋歌曲成為一種新興的趨勢。 然而， 目前針對音樂播放清單自動產生（Automatic Playlist Generation）的研究， 仍著重於利用傳統的音樂屬性與訊號分析提供推薦。 而且，大多數研究也將此問題視為一種靜態的推薦。本論文中，我們認為此類問題較適合採用連續性的最佳化問題來詮釋，並於文中提出適應性的使用者喜好模型，以提供個人化的音樂播放清單推薦服務。主要概念為透過蒐集使用者聆聽音樂的行為（如評分、略過與重播）當作即時回應， 從中學習其對於播放清單中，音樂情緒轉變的喜好。們採用強化式學習（Reinforcement Learning）來學習使用者的喜好， 並產生播放清單的推薦。 此外，透過兩組使用者模擬的案例最佳化學習參數。我們定義多個評估指標以衡量並比較不同推薦方法的優劣。 最後，透過兩個多月的使用者研究，實際觀察不同推薦方法的適用性。 研究結果顯示，本論文提出的推薦方法在大部份的評估指標中， 都擁有優於基準方法的表現。The digitization and online distribution of music content in the Internet era have led to an enormous volume of accessible digital music and diversified the ways with which consumers explore music. In particular, an emerging trend in music exploration is to organize and to search songs according to song emotions. However, research on Automatic Playlist Generation (APG) primarily focuses on leveraging traditional metadata and audio similarity for recommendation. Moreover, Mainstream solutions view APG as a static problem.his thesis argues that the APG problem is better modeled as a continuous optimization problem, and proposes an adaptive preference model for personalized APG based on emotions. The main idea is to collect a user''s behavior in music playing (e.g., rating, skipping and replaying) as immediate feedback in learning the user''s preferences for music emotion within a playlist.einforcement learning is adopted to learn the user''s current preferences, which are used to generate personalized playlists. Learning parameters are tuned via simulation of two hypothetical users. Several evaluation metrics are defined to measure the performance of our approach. A two-month user study is conducted to evaluate the APG solutions. The results show that in most of the evaluation metrices the proposed approach presents a superior performance in comparison with the baseline approach.Acknowledgments iiibstract vist of Figures xiiist of Tables xivhapter 1 Introduction 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4hapter 2 Related Work 5.1 Music Emotion Models . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Music Emotion Estimation . . . . . . . . . . . . . . . . . . . . . . . 7.3 Automatic Playlist Generation . . . . . . . . . . . . . . . . . . . . . 7.3.1 Related Products . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . 14hapter 3 Emotion-Based Personalized APG 17.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2 Proposed Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.1 MEonPlay Automatic Playlist Recommender . . . . . . . . . 21.2.2 APG as a Reinforcement Learning Problem . . . . . . . . . . 22hapter 4 Emotion-Based Adaptive Preference Model 25.1 Annotated Music Dataset . . . . . . . . . . . . . . . . . . . . . . . . 25.1.1 Songs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.1.2 Paritcipants . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.1.3 Music Emotion Model . . . . . . . . . . . . . . . . . . . . . 26.1.4 Annotation Process . . . . . . . . . . . . . . . . . . . . . . . 27.1.5 Usage of the POP500 Dataset . . . . . . . . . . . . . . . . . 28.2 Preference Modeling with Reinforcement Learning . . . . . . . . . . 28.2.1 Modeling States and Actions . . . . . . . . . . . . . . . . . . 29.2.2 Designing the Reward Function . . . . . . . . . . . . . . . . 29.2.3 Solving APG with temporal-difference learning . . . . . . . . 31.2.4 Parameter Selection with Simulation . . . . . . . . . . . . . . 35hapter 5 Experimental Evaluation 39.1 The Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39.2 Experiment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 39.3 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3.1 Miss Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3.2 Miss-to-Hit(k) . . . . . . . . . . . . . . . . . . . . . . . . . 42.3.3 Listening-Time Ratio . . . . . . . . . . . . . . . . . . . . . . 43.3.4 User Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.4 Evaluation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 43hapter 6 Conclusion 49.1 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . . 50.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51ibliography 522081273 bytesapplication/pdfen-US播放清單自動產生音樂推薦系統音樂情緒判斷強化式學習機器學習Automatic Playlist GenerationMusic Recommender SystemMusic Emotion EstimationReinforcement LearningMachine Learningthesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/185439/1/ntu-98-R96922052-1.pdf