指導教授：歐陽明臺灣大學：資訊網路與多媒體研究所蔡伊婷Tsai, Yi-Ting (Evelyn)Yi-Ting (Evelyn)Tsai2014-11-292018-07-052014-11-292018-07-052014http://ntur.lib.ntu.edu.tw//handle/246246/263462外國旅客來到亞洲地區國家時，第一重要的事情就是「吃」。外國旅客無法馬上辨認出食物也看不懂菜單上的文字，而這會使旅客感到困惑。近年來因健康意識的抬頭以及飲食與營養的密切關係，使得食物辨識技術受到越來越多的關注。而隨著個人手持電腦裝置像是智慧型手機或是平板電腦的普遍性增加，將食物辨識系統放入手機的應用程式中是個解決上述問題的方案。本論文設計與開發出一套可透過截取特徵點辨識食物及辨別中文菜單上文字的系統。食物辨識利用Gabor濾波器、顏色、尺度不變特徵轉換（SIFT）與局部二元模式（LBP）並結合稀疏編碼（Sparse Coding）四種特徵法用來描述一個特定的食物。 再對於每種特徵建立支持向量機分類器（SVM classifier），結合自適應增技術（Adaboost）去整合所有的弱分類，而形成強分類器。我們建立了一個含有67種食物的資料庫，每個食物有100張從網路與相簿收集的圖片。另外，我們利用Google 的Tesseract光學字元辨識（OCR）來整合菜單辨識。由於中文菜名主要以烹飪、主料、配料和刀法來命名，主要的成分通常都會放在後面。我們創立一個由繁體中文的食物名稱所構成的語義網絡之語言模型。為了提升準確性，我們將以中文菜單命名法來給予每道菜每個字元不一樣的評分方式，使得結果顯示相同類型的菜色。因此我們建立一個資料庫，這裡面含有123 種以上的菜名，及每個字元所擁有的比重與每個字元的關聯性。除此之外，會提供中英文有關此菜色的介紹，包括起源、食材、烹調方式並營養成分分析。使用者測試結果顯示，食物辨識系統運算比Google Image快上二倍的時間，而菜單辨別系統比Google Translate快上一倍的時間(t = 2.45)。食物辨識與菜單辨識的使用者滿意度分別為80.45%與 83.41%。最後，本文為了加速食物辨識部分的計算時間利用了雲端運算系統與平行運算。Foreigners in Chinese-speaking countries of Asia often face a problem with eating out. They neither recognize the foods nor can read the menus about them, making first-timers confused about what to eat and what they are eating. Food recognition is a topic of research that has received increasing attention due to the rising concern for health and its cause-and-effect relationship to nutrition and diet. A computer-aided tool for food recognition that can allow people to know what they are eating is a proper solution to this issue. Here we propose a system that identifies a food item by its characteristic features, and is also able to recognize Chinese words on menus. It utilizes SIFT and local binary pattern with sparse coding, Gabor and color features as descriptors for a particular food item. We use SVM classifier for training each feature. Adaboost algorithm is applied to perform evaluations of each feature or descriptor of a food item and to assign a weight to that feature. A data bank of pictures of 67 food items, at least 100 images for each of them and collected from internet search or direct photographing of our meals, is constructed for system testing. Another function of menu character recognition is achieved with Google’s Tesseract optical character recognition (OCR), which analyzes and extracts texts from images and compares them to the names of food items in the data bank to find the best match. Chinese dishes are often named according to the style of cooking, the primary and ingredients used, and other descriptive words; in addition; the main ingredient is often placed at the end of a name. Therefor we also incorporate a language model composed of a semantic network of names of foods in Traditional Chinese. To increase accuracy, we designate differing weights to each character in a dish name according to the naming pattern in Chinese dishes. Our databank contains more than 123 dish names with differing weights and semantic relations between each character in the names. The recognition results will show the results of recognition of food image or food name; in addition, we will also show the food’s ingredients, nutrition information (calories, vitamins, lipids), cooking style, in both English and Chinese languages. The results show that computation time for recognition in both food and menu is two to three times less using our system, compared to that required using Google Image and Google Translate (t = 2.45). Overall user satisfaction is 80.45% for our food recognition system and 83.41% for menu recognition system. Finally, this system uses cloud computing and parallel computing to accelerate computation on a mobile platform.致謝 3 中文摘要 4 Abstract 5 List of Figures 9 List of Tables 10 Chapter 1. Introduction 11 1.1 Background and Motivation 11 1.2 Proposed Solution 12 1.3 Contribution 13 1.4 Thesis Organization 14 Chapter 2.Related Work 15 2.1 Food Recognition 15 2.2 Character Recognition 18 2.2.1. Rules of Naming Chinese Menu 18 2.2.2. Optical Character Recognition 18 2.2.3. Tesseract OCR 19 Chapter 3.Framework 23 Chapter 4.Food Identification 25 4.1 Data Collection 25 4.2 Feature Extraction 27 4.2.1. SIFT with Sparse Coding 27 4.2.2. Local Binary Patterns with Multi-resolution Sparse Coding 29 4.2.3. Color Histograms 30 4.2.4. Gabor Texture 31 4.3 Multi-class Classification 32 Chapter 5.Supplementation 35 5.1 Chinese Character Recognition 35 5.1.1. Image Pre-Processing 35 5.1.2. Optical Character Recognition 36 5.2 Cloud Computing 37 5.2.1. System Architecture 37 Chapter 6.Experiment 38 6.1 User Interface 38 6.2 Food Recognition 42 User Study 1: Evaluation of System Performance 44 User Study 2: A Comparison of Food Search by Brute Force and by Our System 47 6.3 Menu Recognition 49 User Study 1: Evaluation of System Performance 50 User Study2: A Comparison of Food Search by Brute Force and by Our System 52 6.4 Parallel Computing 55 Chapter 7.Conclusion 56 Chapter 8.Bibliography 582642467 bytesapplication/pdf論文公開時間：2014/07/04論文使用權限：同意有償授權(權利金給回饋學校)食物辨識菜單辨識特徵描速分類器光學字元辨識平行運算thesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/263462/1/ntu-103-R01944047-1.pdf