陳光華江玉婷2006-08-232018-05-302006-08-232018-05-302000http://ntur.lib.ntu.edu.tw//handle/246246/29210在國內資訊檢索研究已日趨受到重視，卻缺少合適的測試評估機制之背景下，本文參考國外 各測試集的結構、特性與建構經驗，設計中文資訊檢索測試集的方法與程序，發展華文世界第一 套中文資訊檢索測試集，實際地進行測試集的規劃建置工作。本研究所建立的測試集包括132,207 篇新聞文件、50 個查詢主題、以及文件與查詢主題間的相關判斷，平均每個查詢主題有16.32 篇相關文件。研究結果顯示，以統計抽樣的觀點，本測試集的文件數量具有一定的效度；查詢主 題呈現詳盡且多樣化的查詢需求，能反映真實的檢索情況；由三位判斷者進行的相關判斷具有顯 著的一致性，推斷它們是具有可信度的。本測試集不僅具有可應用性，並能作為進一步資訊檢索 與評估研究之基礎。The research and development of information retrieval has made considerable progress recently. However, there is not any applicable test mechanism for system evaluation in the Chinese research society. This paper reports our research on the design and implementation of the first Chinese information retrieval benchmark. According to the framework and contents of the existing foreign benchmarks, we develop a methodology to establish the Chinese IR benchmark. An IR benchmark consists of three parts: document set, topic set, and relevance judgments. Our document set contains 132,207 documents collected from news web sites, the topic set contains 50 topics transformed from real users’ information needs, and each topic has on the average16.34 related documents as a result of the relevance judgments. The results of our research show that the quantity of document set is valid from the viewpoint of sampling statistics. The topics reveal multiple kinds of information need, and they also reflect certain real retrieval environment. Besides, the judgments given by three judges have exhibited significant consistency, so we conclude their reliability. Although the benchmark is in its first edition, it possesses a complete structure and medium scale. On this basis, it is readily feasible to expand this benchmark's current scale to a proper large one in the near future.application/pdf446138 bytesapplication/pdfzh-TW國立臺灣大學圖書資訊學系資訊檢索測試集資訊檢索評估文件集查詢主題相關判斷IR Benchmark（IR Test Collection）IR EvaluationDocument SetTopicRelevance Judgmentjournal articlehttp://ntur.lib.ntu.edu.tw/bitstream/246246/29210/1/jicls1999.pdf