Evaluating a Metadata-based Term Suggestion Interface for PubMed with Real Users with Real Requests
Resource
Proceedings of the Annual Meeting of the American Society for Information Science & Technology, 46(1), 1-18 .
Journal
Proceedings of the American Society for Information Science and Technology
Pages
1
Date Issued
2009
Date
2009
Author(s)
Abstract
This paper reports results of an evaluation study of MAP (Multi-faceted Access to PubMed), a metadata induced query suggestion interface for PubMed bibliographic search.
A novel evaluation methodology was used to address the challenges involved in evaluating an IIR (Interactive Information Retrieval) system such as the MAP interface. The most significant aspect of this methodology is that, instead of using assigned tasks common in traditional IR evaluation, it asks real users with real search requests to search with real systems in an experimental setting. Several performance measures were created based on which comparisons were made between MAP and PubMed baseline. MAP was shown to perform better in several of these measures, especially when the search requests had not been attempted before.
The finding pointed to search characteristics as an important intervening variable in IIR evaluation. The advantages of and potential threats to our methodology were also discussed.
Type
conference paper
File(s)![Thumbnail Image]()
Loading...
Name
index.html
Size
23.46 KB
Format
HTML
Checksum
(MD5):b8e5e1d954848d95fa2cee81eb2ebcf8