Page 102 - DLIS405_INFORMATION_STORAGE_AND_RETRIEVAL

Unit 10: Information Storage and Retrieval System

Characteristics of an ISRS include lack of centralization, graceful degradation in the event of hardware Notes
failure, and the ability to rapidly adapt to changing demands and resources. The lack of centralization
helps to ensure that catastrophic data loss does not occur because of hardware or programme failure,
or because of the activities of malicious hackers. Graceful degradation is provided by redundancy of
data and programming among multiple computers. The physical and electronic diversity of an ISRS,
along with the existence of multiple operating platforms, enhances robustness, flexibility, and
adaptability. (These characteristics can also result in a certain amount of chaos.) In addition to these
features, some ISRSs offer anonymity, at least in theory, to contributors and users of the information.
A significant difference between an ISRS and a database management system (DBMS) is the fact
that an ISRS is intended for general public use, while a DBMS is likely to be proprietary, with access
privileges restricted to authorized entities. In addition, an ISRS, having no centralized management,
is less well-organized than a DBMS.

What do you mean by ISRS. Define?

10.1 Information Retrieval System Evaluation

To measure ad hoc information retrieval effectiveness in the standard way, we need a test collection
consisting of three things:
• A document collection
• A test suite of information needs, expressible as queries
• A set of relevance judgments, standardly a binary assessment of either relevant or non-rel-
evant for each query-document pair.
The standard approach to information retrieval system evaluation revolves around the notion of
relevant and non-relevant documents. With respect to a user information need, a document in the
test collection is given a binary classification as either relevant or non-relevant. This decision is
referred to as the gold standard or ground truth judgment of relevance. The test document collection
and suite of information needs have to be of a reasonable size: you need to average performance
over fairly large test sets, as results are highly variable over different documents and information
needs. As a rule of thumb, 50 information needs has usually been found to be a sufficient minimum.
Relevance is assessed relative to and not a query. For example, an information need might be:
Information on whether drinking red wine is more effective at reducing your risk of heart attacks
than white wine.
This might be translated into a query such as: wine and red and white and heart and attack and
effective.
A document is relevant if it addresses the stated information need, not because it just happens to
contain all the words in the query. This distinction is often misunderstood in practice, because the
information need is not overt. But, nevertheless, an information need is present. If a user types
python into a web search engine, they might be wanting to know where they can purchase a pet
python. Or they might be wanting information on the programming language Python.
From a one word query, it is very difficult for a system to know what the information need is. But,
nevertheless, the user has one, and can judge the returned results on the basis of their relevance to
it. To evaluate a system, we require an overt expression of an information need, which can be used
for judging returned documents as relevant or non-relevant. At this point, we make a simplification:
relevance can reasonably be thought of as a scale, with some documents highly relevant and others
marginally so. But for the moment, we will use just a binary decision of relevance.

LOVELY PROFESSIONAL UNIVERSITY 97