https://scholars.lib.ntu.edu.tw/handle/123456789/24378
Title: | 中文資訊檢索測試集之設計與製作 The Design and Implementation of the Chinese IR Benchmark |
Authors: | 陳光華 江玉婷 |
Keywords: | 資訊檢索測試集;資訊檢索評估;文件集;查詢主題;相關判斷;IR Benchmark(IR Test Collection);IR Evaluation;Document Set;Topic;Relevance Judgment | Issue Date: | 2000 | Publisher: | 臺北市:國立臺灣大學圖書資訊學系 | Journal Volume: | 6 | Journal Issue: | 3 | Start page/Pages: | 61-80 | Source: | 資訊傳播與圖書館學 | Abstract: | 在國內資訊檢索研究已日趨受到重視,卻缺少合適的測試評估機制之背景下,本文參考國外 各測試集的結構、特性與建構經驗,設計中文資訊檢索測試集的方法與程序,發展華文世界第一 套中文資訊檢索測試集,實際地進行測試集的規劃建置工作。本研究所建立的測試集包括132,207 篇新聞文件、50 個查詢主題、以及文件與查詢主題間的相關判斷,平均每個查詢主題有16.32 篇相關文件。研究結果顯示,以統計抽樣的觀點,本測試集的文件數量具有一定的效度;查詢主 題呈現詳盡且多樣化的查詢需求,能反映真實的檢索情況;由三位判斷者進行的相關判斷具有顯 著的一致性,推斷它們是具有可信度的。本測試集不僅具有可應用性,並能作為進一步資訊檢索 與評估研究之基礎。 The research and development of information retrieval has made considerable progress recently. However, there is not any applicable test mechanism for system evaluation in the Chinese research society. This paper reports our research on the design and implementation of the first Chinese information retrieval benchmark. According to the framework and contents of the existing foreign benchmarks, we develop a methodology to establish the Chinese IR benchmark. An IR benchmark consists of three parts: document set, topic set, and relevance judgments. Our document set contains 132,207 documents collected from news web sites, the topic set contains 50 topics transformed from real users’ information needs, and each topic has on the average16.34 related documents as a result of the relevance judgments. The results of our research show that the quantity of document set is valid from the viewpoint of sampling statistics. The topics reveal multiple kinds of information need, and they also reflect certain real retrieval environment. Besides, the judgments given by three judges have exhibited significant consistency, so we conclude their reliability. Although the benchmark is in its first edition, it possesses a complete structure and medium scale. On this basis, it is readily feasible to expand this benchmark's current scale to a proper large one in the near future. |
URI: | http://ntur.lib.ntu.edu.tw//handle/246246/29210 | Rights: | 國立臺灣大學圖書資訊學系 |
Appears in Collections: | 圖書資訊學系 |
File | Description | Size | Format | |
---|---|---|---|---|
jicls1999.pdf | 435.68 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.