陳信希臺灣大學：資訊工程學研究所梁玉婷Liang, Yu-TingYu-TingLiang2007-11-262018-07-052007-11-262018-07-052007http://ntur.lib.ntu.edu.tw//handle/246246/53936自動問答技術是自然語言處理領域中一個非常熱門的研究方向。使用者以自然語言輸入問題，自動問答系統綜合運用各種自然語言處理技術，來迅速且有效率地擷取相關答案以提供給使用者。使用者不單只對詢問事實有興趣，也會對詢問意見有興趣。本論文設計並實現一個意見問答系統，以供回答有關人們意見，感覺，或想法的問句。自動問答系統一般包括三個主要組成部分：問句分析、答案區段選取和答案抽取。我們的研究著重前兩部分，也就是問句分析和答案區段選取。 問句分析包含三個主要工作：問句類型，問句焦點，以及問句意見傾向的判定。我們定義六種意見問句類型，並提出雙層的問句分類器。第一層分類器分辨問句是問事實，亦或是意見。倘若是意見，第二層分類器進一步將此問句分類到定義的六種意見問句。這兩個分類器的效能分別可以達到87.8%和92.5%的Ｆ值。接著我們討論判斷問句焦點和問句意見傾向的方法。前者判斷出的焦點用來提交到資訊檢索系統中，以找出與問句相關的句子作為可能的答案句。後者判斷問句的意見傾向，並與可能的答案句比對，來保留具有相同意見傾向的答案句。 答案區段選取亦包含三個主要工作。在資訊檢索系統提供的相關句中，我們會判斷焦點（焦點領域識別）是否在意見範圍內（意見領域識別），如果是，再進一步判斷此範圍的意見傾向是否和問句相同（意見傾向偵測）。實驗包含了十八種不同的組合。其中最好的模型利用部分焦點比對，在意見範圍層次達到40.59%的Ｆ值。倘若去除相關性的影響，最好的模型之Ｆ值可以提升到87.18%。我們進一步將問句按照其主題分類並實驗，其結果顯示出不同主題的問句，有著不同的困難度。 最後我們總結目前的實驗結果，並提出一些相當有趣的議題，以供未來研究，進而實現一個完整的意見問答系統。Question answering (QA) systems provide an elegant way for people to access an underlying knowledge base. Humans are not only interested in factual questions but also interested in opinions. In this thesis, an opinion QA system dealing with opinion questions are proposed. We attempt to investigate technologies of question analysis and answer passages retrieval. For question analysis, six opinion question types are defined. A two-layered framework utilizing two question type classifiers is proposed. Algorithms for these two classifiers are discussed. The performance achieves 87.8% in general question classification and 92.5% in opinion question classification. The question’s focus and polarity are detected as well to form an IR query and sieve out relevant sentences which have the same polarity to the question. For answer passages retrieval, three components are introduced. Relevant sentences retrieved by the IR system are further identified whether the focus (Focus Scope Identification) is in a scope of opinion text spans (Opinion Scope Identification) or not, and if yes, whether the polarity of the scope matches with the polarity of the question (Polarity Detection). A total of 18 combinations are proposed and experimented. The best model achieves 40.59% of F-measure using partial match at boundary level. With relevance issues removed, the F-measure of the best model boosts up to 87.18%. We further divide the experiment results by topics. The results indicate difficulties of different topics. We conclude with some yet unsolvable but quite interesting problems to study in the future to build a complete opinion QA system.Table of Contents 口試委員會審定書 i 誌謝 ii 中文摘要 iii Abstract iv Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Related Work 2 1.3 Main Issues 5 1.4 Thesis Structure 5 Chapter 2 Opinion Question Answering System Framework 7 2.1 Overview 7 2.2 Part-of-Speech Tagger 8 2.3 Question Analysis 8 2.3.1 Two-Layered Question Classification 9 2.3.2 Question Focus Extraction and Polarity Detection 10 2.4 Answer Passages Retrieval 11 Chapter 3 Experimental Corpus 13 3.1 Sources 13 3.1.1 TREC 13 3.1.2 NTCIR 15 3.1.3 Internet Polls 16 3.1.3.1 Chinatimes 16 3.1.3.2 Era 16 3.1.3.3 TVBS 17 3.1.4 OPQ 17 3.2 Questions Collection 19 3.2.1 TREC 19 3.2.2 NTCIR 19 3.2.3 Internet Polls 20 3.2.3.1 Chinatimes 20 3.2.3.2 Era 21 3.2.3.3 TVBS 22 3.2.3.4 Statistics for Polls 22 3.2.4 OPQ 23 3.2.5 Overall Statistics 24 3.3 Answers Annotation 24 3.3.1 Questions Analysis 25 3.3.1.1 Three Challenges on Holders 25 3.3.1.2 Two Challenges on Opinions 26 3.3.1.3 Two Challenges on Concepts 27 3.3.2 Questions Selection Strategy 29 3.3.3 Answers Annotation of OPQ 31 Chapter 4 Two-Layered Question Classification 33 4.1 The Purpose of Two Layers 33 4.2 Opinion Questions Taxonomy 35 4.3 Q-Classifier 37 4.3.1 Algorithm 38 4.3.2 Experiment 42 4.3.2.1 Experiment Setup 42 4.3.2.2 Experiment Results and Discussions 42 4.4 OPQ-Classifier 46 4.4.1 Algorithm 46 4.4.1.1 Grammatical Forms 46 4.4.1.2 Scoring Function 47 4.4.1.2.1 Feature Extraction 48 4.4.1.2.2 Opinion Question Type Assignation 51 4.4.2 Experiment 51 4.4.2.1 Experiment Setup 51 4.4.2.2 Experiment Results and Discussions 52 Chapter 5 Answer Passages Retrieval 55 5.1 Question Focus Extraction 55 5.2 Question Polarity Detection 57 5.3 Focus Scope Identification 59 5.3.1 Exact Match 59 5.3.2 Partial Match 59 5.3.3 Lenient 60 5.4 Opinion Scope Identification 62 5.4.1 Sentence Level 62 5.4.2 Subsentence Level 62 5.4.3 Boundary Level 62 5.4.4 Determining Whether the Opinion is Toward the Focus 66 5.5 Polarity Detection 67 5.5.1 Opinion Score Approach 67 5.5.2 Action Approach 69 5.6 Combined Approaches 72 5.6.1 Combined Approach on Focus Scope Identification 72 5.6.2 Combined Approach on Polarity Detection 73 5.7 Experiments 73 5.7.1 Evaluation Metrics 74 5.7.2 Experiment on Focus Scope Identification 75 5.7.2.1 Experiment Setup 75 5.7.2.2 Experiment Results and Discussions 75 5.7.3 Experiment on Opinion Scope Identification 78 5.7.3.1 Experiment Setup 78 5.7.3.2 Experiment Results and Discussions 78 5.7.4 Experiment on Polarity Detection 81 5.7.4.1 Experiment Setup 81 5.7.4.2 Experiment Results and Discussions 82 5.7.5 Experiment on Corpus of Different Relevance Degrees 84 5.7.5.1 Experiment Setup 85 5.7.5.2 Experiment Results and Discussions 85 5.7.6 Experiment on Combined Approaches 87 5.7.6.1 Experiment Setup 88 5.7.6.2 Experiment Results and Discussions 88 5.7.7 Experiment on Questions of Different Topics 89 5.7.7.1 Experiment Setup 90 5.7.7.2 Experiment Results and Discussions 90 5.7.8 Overall Error Analysis 92 Chapter 6 Conclusion and Future Works 97 6.1 Conclusion 97 6.2 Future Works 100 References 103 Appendix 107 І Details of the 60 Selected Opinion Questions 107 II A List of Linking Elements 109 Ш Action Seed Vocabulary of Do’s 111 IV Action Seed Vocabulary of Dont’s 113844525 bytesapplication/pdfen-USopinion question answering systemquestion answeringquestion analysisanswer passages retrievalopinion extraction意見問答系統自動問答系統問題分析答案區段選取意見擷取thesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/53936/1/ntu-96-R93922145-1.pdf