鄭士康臺灣大學：資訊網路與多媒體研究所蔡函辰Tsai, Han-ChenHan-ChenTsai2007-11-272018-07-052007-11-272018-07-052007http://ntur.lib.ntu.edu.tw//handle/246246/58438在科技高度發達的數位時代，科技媒體成為藝術創作的新表現媒介。資訊技術與多媒體的結合帶來了具創新性的數位內容，以及高附加價值的影音互動多媒體產業。本論文介紹並且實作一個互動影音系統；「虛擬鋼琴系統」。研究動機起源於數位虛擬樂器與音樂玩具等相關研究；應用數位影像處理和MIDI技術，結合影音互動即時系統。藉由攝影機對使用者的手部運動作取像偵測，再進行動作的分析，以即時產生相對音高之鋼琴琴聲。本研究主要以虛擬鋼琴彈奏為實作，在技術層面，探討科技和媒體技術，在應用面上可以和數位內容產業之互動式遊戲結合，未來更可延伸至可攜式數位學習載具等產業。The information technology is highly developed nowadays. It is also used as a new tool for creating the contents. Combining information technology with multimedia can bring us innovative and value-added digital contents for multimedia industries. In this thesis, we will propose and implement an interactive multimedia system, virtual piano system, which integrates information technology with interactive multimedia contents. This research’s motivation is based on a concept originally derived from the digital musical instrument and music toys. In this thesis, we use a video camera to capture the movements of user’s fingertips. After detecting and analyzing the movement, the corresponding piano sound will be played in real-time. Digital image processing and MIDI technology are involved in this study. It has many potential capabilities for developing products for digital content industries such as interactive game and portable digital learning device.Abstract i 摘要 ii Contents iii List of Figures v List of tables vii Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Literature Survey 2 1.3 Approach 3 1.4 Organization of Thesis 4 Chapter 2 Background 5 2.1 Video and Interactive System 5 2.2 Color Model and Hue 6 2.3 Audio an MIDI 8 Chapter 3 Design of Interactive Virtual Piano System 13 3.1 Introduction 13 3.2 Image detection 14 3.2.1 Computing Hue from RGB 14 3.2.2 Design of Threshold 15 3.2.2.1 Threshold of Skin Hue Value 16 3.2.2.2 Threshold of Hue Variation 18 3.2.2.3 Discussion of these two methods 23 3.3 Audio Feedback 24 3.3.1 Pitch Shifting Based on Wavetable Synthesis 24 3.3.2 MIDI Note Events 29 3.4 Visual Feedback 33 3.4.1 Interactive Animation Design 33 3.4.2 Interactive User Interface 34 Chapter 4 Experiments and Results 36 4.1 Demonstration of Interactive Virtual Piano System 36 4.2 User Evaluation Experiments 40 Chapter 5 Further Discussions 43 5.1 Velocity Detection 43 5.2 Chroma Key Technique 44 Chapter 6 Conclusions 45 References 46 Appendix A: Questionnaires for User Satisfaction Survey 481138344 bytesapplication/pdfen-US虛擬樂器互動即時virtual instrumentinteractivereal-timethesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/58438/1/ntu-96-R94944010-1.pdf