陳正剛臺灣大學：工業工程學研究所陳柏良Chen, Po-LiangPo-LiangChen2007-11-262018-06-292007-11-262018-06-292006http://ntur.lib.ntu.edu.tw//handle/246246/51240費雪線性區別法(FLD)和Fukunaga-Koontz線性區別法(FKLD)是具有不同特性的分類分析(classification analysis)方法，它們的目的在於找到一個「區別向量(discriminant vector) 」，使得訓練資料(training instances)投影於其上—稱之為「區別分數(discriminant score)」—可用來分類不同群的資料。 我們以FKLD為基礎，提出新的分類方法去改善傳統FKLD的表現。首先，我們結合了FKLD以及主成分分析(PCA)的概念使FKLD在分類上更有意義。接著擴展FKLD到兩個區別空間(discriminant space)中，來得到更佳的分類正確率。我們命名此分類方法為「使用二區別空間之Fukunaga-Koontz 線性區別法(Two-Space FKLD) 」。但在一些實例當中，Two-Space FKLD仍有不足之處，因此我們進一步結合Two-Space FKLD，以及費雪區別分析中的概念來改善它。由於Two-Space FKLD只能用來解決兩群資料的分類問題，所以我們利用一對一比較的方法(one-against-one approach)，來解決多群資料的分類問題。 由於上述方法都是線性分析的方法，為了能夠處理蘊含非線性特徵的資料，因此從費雪線性區別方法衍生出以核函數(kernel function)為根基的費雪區別方法(Kernel Fisher Discriminant)，此方法是把資料從原本的屬性空間(attribute space)投射到一個較高維度的特徵空間(feature space)。我們利用核函數為根基發展出Two-Space FKLD的非線性方法，來分類蘊含非線性特徵的資料。此非線性方法命名為「使用二區別空間之核心Fukunaga-Koontz區別法(Two-Space KFKD)」。此非線性的方法能有效的區分蘊含非線性特徵的資料。 最後，在案例學習中，透過三筆現實世界中的資料來驗證所提出的方法，同時根據這些資料，我們對多種分類方法進行比較。分析的結果指出，我們提出的方法其分類效能等同—甚至優於—FLD，KFD或SVM等方法。Fisher’s Linear Discriminant (FLD) and Fukunaga-Koontz Linear Discriminant (FKLD) have different characteristics in classification analysis. However, they share the same objective to find a “discriminant vector” on to which the projection of training instances, called “discriminant scores”, can be used to discriminate different classes. In this research, we propose novel approaches to address issues of using the FKLD approach and improve its performance. First, in order to make the FKLD approach more meaningful in classification, we combine the FKLD with the concept of Principal Component Analysis (PCA). Then, we extend the FKLD approach with two “discriminant spaces” to improve the classification accuracy, and name the novel linear classification method “Two-Space Fukunaga-Koontz Linear Discriminant (Two-Space FKLD)”. But the Two-Space FKLD method is still not sufficient in some cases. Therefore, we further combine the Two-Space FKLD with the concept of FLD analysis. Then we expend the novel classification methods to multi-class classification problems using one-against-one approach. Because linear methods are not sufficient to analyze the data with nonlinear characteristics, the nonlinear Kernel Fisher Discriminant (KFD) is extended approach from the FLD based on kernel function. KFD transforms the instances from the original attribute space to the feature space with higher dimension. In this research we will also develop a kernel-based nonlinear FK discriminant approach called “Two-Space Kernel Fukunaga-Koontz Discriminant (Two-Space KFKD)”. The Two-Space KFKD method can also effectively discriminate the data with nonlinear characteristics. Finally, we demonstrate our proposed methods through three real-world datasets in our case study. We also compare the performance of various classifiers based on these datasets. The results indicate that the performances of our methods are equal, sometimes superior, to FLD, KFD or SVM approaches.Abstract i 論文摘要 ii Contents iii Contents of Figures iv Contents of Tables vi Chapter 1 Introduction 1 1.1 Background 1 1.2 Multivariate Classification Approaches 3 1.2.1 Fisher’s Linear Discriminant (FLD) 3 1.2.2 Fukunaga - Koontz Linear Discriminant Approach (FKLD approach) 5 1.3 Problem Description and Research Objective 9 1.4 Thesis Organization 11 Chapter 2 Two-Space Fukunaga-Koontz Linear Discriminant Method 12 2.1 Combining the FKLD Approach with the PCA concept 12 2.2 Extending the FKLD Approach with Two Discriminant Spaces 17 2.3 Combining the FLD Approach with the Two-Space FKLD Method 25 2.3.1 P-transformed FLD (PFLD) 27 2.3.2 Combining the PFLD approach with the Two-Space Method 29 2.4 Extending to Multi-class Problem with One-Against-One 31 Chapter 3 Two-Space Kernel Fukunaga-Koontz Discriminant Method 32 3.1 Kernel Fisher Discriminant (KFD) 32 3.2 Using Kernel Function for the Two-Space FKLD Method 37 3.3 Combining the KFD approach with the Two-Space KFKD Method 44 3.3.1 Q-transformed KFD (QKFD) 44 3.3.2 Combining the QKFD approach with the Two-Space KFKD Method 46 Chapters4 Case study 50 4.1 Medline text Dataset 50 4.2 Balance Dataset 52 4.3 Hayes-Roth Dataset 59 Chapters5 Conclusions 63 Reference 64 Appendix A: Program User Interface 66 Appendix B: Computer Code - C# 67 B.1 FLD Code 67 B.2 FKLD Code 70 B.3 Two-Space FKLD Code 74 B.4 PFLD Code 78 B.5 KFD Code 79 B.6 Two-Space KFKD Code 82 B.7 QKFD Code 861351451 bytesapplication/pdfen-US費雪線性區別Fukunaga-Koontz 線性區別核心費雪區別Fisher’s Linear DiscriminantFukunaga-Koontz Linear DiscriminanKernel Fisher Discriminantthesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/51240/1/ntu-95-R93546003-1.pdf