2002-08-012024-05-18https://scholars.lib.ntu.edu.tw/handle/123456789/706202摘要：資訊檢索系統不論在設計、研發、運作等各階段，評估均是其中不可或缺的重要環節。透過此程序，研究者能藉以驗證系統效益、比較各種檢索技術的優劣，以作為改進之參考，使資訊檢索系統的運作及效能更臻完善。目前全球有三大評估會議：美洲地區的TREC，歐洲地區的CLEF，亞洲地區的NTCIR，本計畫申請人為亞洲地區NTCIR跨語言資訊檢索評估項目的主席，目前參與的國家有台灣，日本，以及韓國。中文一向是資訊檢索研究極為重視的書面語，如何有效檢索中文文件更是研究者努力的目標，但是中文資訊檢索測試集的缺乏，無法公平、有效地評估資訊檢索系統的績效。本人於三年前完成全世界第一套的中文資訊檢索測試集，運用該測試集舉辦了第二屆NTCIR中文與英文資訊檢索評估項目，第三屆更擴展為中、日、韓、英多語資訊檢索評估。在會議籌辦過程，與日本、韓國的學者持續不斷討論，我們發覺資訊檢索的評估仍然有許多研究課題，也需要透過舉辦評估會議與論壇，與世界各國研究者交流討論。因此，遂有本研究計畫申請案的產生，台灣、日本、韓國的學者決定進行一項國際性跨語文字資訊系統的量化評估計畫。分年的重點如下所示： 1. 第一年：大規模同質性資料之檢索<br> Abstract: The researches on cross-language information systems have been increasingly important while the Internet becomes the major channel of information access. The different users from different countries join the common information channel to find the needed information. To provide a user-friendly, cross-language, multi-language, and inter-language information systems is the major concern of researchers. Among all kinds of researches, evaluations of information systems perhaps catch the least attention of researchers. Most of researchers focus on the design of new systems using complicated technologies rather than the evaluation of the designed systems. However, evaluation is indispensable for a successful information system. Two possible roles the evaluation could play in the development of information systems. The first is the so-called “formative evaluation”; the second is “conclusive evaluation”. The formative evaluation is used to evaluate the developing information systems. The conclusive evaluation is used to evaluate the developed information systems. Both of evaluations are crucial. The proposed project is an international joint effort of Japan, Korea, and Taiwan. The Japanese partner will be supported by National Institute of Informatics; the Korean partner will find the funding support from Korean Science and Engineering Foundation; the Taiwan partner will find funding support from National Science Council. Each country will collect the resources for her language independently, construct the common evaluation platform cooperatively, and exchange the resources and technologies. The three countries will discuss by email and the regular meeting will be held periodically. Two evaluation workshops will be organized by three partners and opened to international participants. The core tasks of this project are shown as follows: 1. The first year: Retrieval Evaluation for Large Volume of Consistent Data （2002-08-01/2003-07-31）  Collect consistent documents (news articles)  Collect users’ information need of news articles  Construct XML schema or DTD of document and information need  Tag documents and information need using XML tags  Take part in the international academic cooperation  Develop metrics of information retrieval  Hold information retrieval evaluation workshop 2. The second year（2003-08-01/2004-07-31）  Enlarge the collection to magazines, abstract of academic paper, patent document, etc.  Construct heterogeneous users’ information need  Develop relevance judgment tool for information retrieval  Construct XML schema or DTD of document and information need  Tag documents and information need using XML tags  Organize forum for resources- and technology-sharing of information retrieval 3. The third year（2004-08-01/2005-07-31）  Collect spoken data and transcribe to written data  Collect users’ information need of spoken data  Construct test collection of multi-media information retrieval  Take part in th資訊檢索量化評估Information RetrievalQuantitative Evaluation