Muddiest Point for Lecture 13

The classification and clustering is quite useful in information retrieval, since classification is based on supervised learning and clustering is based on unsupervised learning. Can we say that generally classification is more accurate than clustering?

Reading notes for Lecture 14

The main idea of the semantic web is machine-readable ontology standards. An "ontology" is a theory of the categories of things that comprise a given domain. Familiar examples of ontologies include taxonomies and controlled vocabularies. Every computer system uses an ontology. Computer systems that are built independently of one another, however, often cannot interoperate because they use different ontologies. Now that computer systems are heavily networked, numerous user groups have begun standardizing their ontologies. The semantic web consists of mechanisms for "marking up" ontologies and then processing them.
There are roughly four kinds of ontologies: document ontologies (e.g., the chapters of a book or the footnotes of a paper), metadata ontologies (e.g., the format of a file or the copyright status of a document), domain ontologies (e.g., the components of an automobile or the entries of a schedule), and service ontologies (e.g., the inputs of a software module, the steps of a transaction, or the formats of messages that are passed back and forth between a client and server). Because its topic is information retrieval and not web services generally, this course is mainly about document and metadata ontologies. In practice, however, document and metadata ontologies often include elements of domain ontologies, and many services use documents and metadata. For completeness, therefore, weeks 9 and 10 respectively will be on domain and service ontologies.
The semantic web includes a layered series of markup languages starting with XML (itself derived from SGML). The most distinctive aspect of XML is that user groups can use it to define specialized sublanguages to mark up the ontologies that are meaningful for their own work. "User-centered design" in a semantic web context means precisely the codification of a user group's ontologies. This is important because document collections that have been marked up within a standard XML-based language can be stored and retrieved in much more sophisticated ways than unstructured plain text documents, or documents whose structures have not been marked up in a standardized way. Although it is too soon to be certain, the result may be a revolution in the technology of information retrieval. In the past, this course has generally applied ideas from user interface design to more traditional information retrieval technologies.

Muddiest point for lecture 12

This lecuture is pretty interesting since it combine the adaptive system feature with information retrieval. adaptive IR is kind of based on relevance feedback and user profile. My question is:
Will the user profile and feedback be stored in the system, or just used when during the specific retrieval?

Reading notes for lecture 13

Text classification is one of the first applications of machine learning, that applies to the general problem of supervised inductive learning: given a set of training documents, classified to one or more predefined categories, learn to automatically classify new documents. Automated text classification has been used in a number of different applications: automatic indexing, content management, filtering, and routing, word sense disambiguation, and Yahoo!-style search space categorization. A plethora of techniques have been developed for text classification,including Nearest-Neighbours, Regression, Neural Networks
Naive Bayes, Decision Trees, and Support Vector Machines. In most cases, the classification algorithms require sufficient training data in order to generalize well on unseen documents.
However, the generalization using labeled examples is an extremely costly and time-consuming activity. The
need for classifiers that can learn from small training samples is imperative. This is an area of active research and several experiments have been conducted to boost conventional classifiers’ performance, by combining supervised learning with semi-supervised or unsupervised, using techniques such as co-training active learning and transductive SVMs .

Muddiest Point for lecture 10

Lecture 10 talks about some interesting topics on web retrieval. It also solved my question about the web ranking mechanism.

Reading Notes for lecture 11

Popular online communities and services such as Flickr, Youtube, Facebook or LinkedIn are spearheading an emerging type of information on the Web. This information is composed of classical textual and multimedia data, in concert with additional data (tags, annotations, comments, ratings). Perhaps most significantly, the information is overlaid on an explicit social network created by the participants of each of these communities. The result is a rich structure of inter-relationships between content items, participants and services. Although the size of such networks requires the use of advanced Information Retrieval techniques, classical IR models are not tailored for this type of content as they do not (in general) take advantage of the particular structure and unique aspects of this socially-driven content.
I'm interested in changes which such new media made on the IR system.

Muddiest Point for Exam

I'm quite confused about the last question. Can professor explain it on class?

Reading Notes for Lecture 9

Current information retrieval systems have addressed these inherent properties of information seeking and indexing in a variety of ways. Browsing has been employed to facilitate the iterative and ill-defined nature of information seeking, but can lead to a loss of direction and overly narrow searches. Queries provide a means to direct the search, but often rely on the user understanding a complex query language and proper vocabulary to be effective. An integration of these strategies is a promising approach that can solve some of these problems. Information visualization techniques such as the Perspective Wall at Xerox PARC can be used in interface design to improve both browsing and querying.
Good information retrieval system design combines a combination of support for information seeking strategies, such as browsing and direct querying, in an interface that provides effective cues to the location, use, and characteristics of the retrieved information. Feedback techniques are also crucial to support the iterative refinement of information needs.
Guidelines for the design of information retrieval systems must address not only issues of look-and-feel, but also of effective interaction. Dialog models based on relevance feedback and query reformulation explicitly address the ill-defined nature of information seeking by allowing users to learn from the repository and iteratively refine the information need. Systems need to support a number of interaction styles, such as querying and browsing, to accommodate the different kinds of search strategies users may need to use.
Design strategies for retrieval systems need to pay particular attention to the interaction between users and the texts they retrieve. People's information seeking behavior needs to be analyzed in the problem solving context in which their information needs arise. For example, what are some of the common associations people make? What information is closely related? What different perspectives can a piece of information be viewed from? The answers to these questions often differ, depending critically on the nature of the task and the individuals performing the task. Task analysis, user modeling and interaction modeling are some of the strategies that can be used to improve the design of retrieval systems.

Muddiest points for Lecture 6

Question:

In what level the performance of a IR system can be defined as 'bad'? Is there any standard? for example, when F- measure becomes ***, the system is considered as a bad IR system?

Reading Notes for Lecture 7

Task:
1. IIR Chapter 9.
2. Xu, J. and Croft, W. B. 2000. Improving the effectiveness of information retrieval with local context analysis. ACM Trans. Inf. Syst. 18, 1 (Jan. 2000), 79-112.
3. Wang, X., Fang, H., and Zhai, C. 2008. A study of methods for negative relevance feedback. In Proceedings of the 31st Annual international ACM SIGIR Conference on Research and Development in information Retrieval (Singapore, Singapore, July 20 - 24, 2008). SIGIR '08. ACM, New York, NY, 219-226.
4. Donna Harman, Relevance feedback revisited. Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval. Pages: 1 - 10. Copenhagen, Denmark. 1992.

The idea of relevance feedback is to involve the user in the retrieval process so as to improve the final result set. In particular, the user gives feedback on the relevance of documents in an initial set of results. The basic procedure is:

• The user issues a (short, simple) query.
• The system returns an initial set of retrieval results.
• The user marks some returned documents as relevant or nonrelevant.
• The system computes a better representation of the information need based on the user feedback.
• The system displays a revised set of retrieval results.

Relevance feedback can go through one or more iterations of this sort. The process exploits the idea that it may be difficult to formulate a good query when you don't know the collection well, but it is easy to judge particular documents, and so it makes sense to engage in iterative query refinement of this sort. In such a scenario, relevance feedback can also be effective in tracking a user's evolving information need: seeing some documents may lead users to refine their understanding of the information they are seeking.
The idea of relevance feedback ( ) is to involve the user in the retrieval process so as to improve the final result set. In particular, the user gives feedback on the relevance of documents in an initial set of results. The basic procedure is:

• The user issues a (short, simple) query.
• The system returns an initial set of retrieval results.
• The user marks some returned documents as relevant or nonrelevant.
• The system computes a better representation of the information need based on the user feedback.
• The system displays a revised set of retrieval results.

Relevance feedback can go through one or more iterations of this sort. The process exploits the idea that it may be difficult to formulate a good query when you don't know the collection well, but it is easy to judge particular documents, and so it makes sense to engage in iterative query refinement of this sort. In such a scenario, relevance feedback can also be effective in tracking a user's evolving information need: seeing some documents may lead users to refine their understanding of the information they are seeking.
The idea of relevance feedback ( ) is to involve the user in the retrieval process so as to improve the final result set. In particular, the user gives feedback on the relevance of documents in an initial set of results. The basic procedure is:

• The user issues a (short, simple) query.
• The system returns an initial set of retrieval results.
• The user marks some returned documents as relevant or nonrelevant.
• The system computes a better representation of the information need based on the user feedback.
• The system displays a revised set of retrieval results.

Relevance feedback can go through one or more iterations of this sort. The process exploits the idea that it may be difficult to formulate a good query when you don't know the collection well, but it is easy to judge particular documents, and so it makes sense to engage in iterative query refinement of this sort. In such a scenario, relevance feedback can also be effective in tracking a user's evolving information need: seeing some documents may lead users to refine their understanding of the information they are seeking.
Pseudo relevance feedback,also known as blind relevance feedback , provides a method for automatic local analysis. It automates the manual part of relevance feedback, so that the user gets improved retrieval performance without an extended interaction.
Query expansion(QE) is the process of reformulating a seed query to improve retrieval performance in information retrieval operations. In the context of web search engines, query expansion involves evaluating a user's input (what words were typed into the search query area, and sometimes other types of data) and expanding the search query to match additional documents. Query expansion involves techniques such as:

• Finding synonyms of words, and searching for the synonyms as well
• Finding all the various morphological forms of words by stemming each word in the search query
• Fixing spelling errors and automatically searching for the corrected form or suggesting it in the results
• Re-weighting the terms in the original query

Query expansion is a methodology studied in the field of computer science, particularly within the realm of natural language processing and information retrieval.
Q1:
I am curious about the realization of finding synonyms, if it is realized by creating synonyms list for a word? or by other ways?

Muddiest Point for Lecture 5

The Probabilistic ranking theroy is quite confusing, but I think the implementation should be much easier than I previously thought, because all the probability work is done by program.
My question is Can the probablity of relevance be biased if there is relevance between two terms, for example, synonym?

Reading Notes for lecture 6

Task:

1. IIR chapter 8.
2. Karen Sparck Jones, What's the value of TREC: is there a gap to jump or a chasm to bridge?
3. Kalervo Järvelin, Jaana Kekäläinen. Cumulated gain-based evaluation of IR techniques ACM Transactions on Information Systems (TOIS) Volume 20 , Issue 4 (October 2002) Pages: 422 – 446

The evaluation of a IR system makes clear that how well the IR system does. Actually, there are altogether 3 field to determine the performance of the IR system:

1. The effectiveness of search result

2. Processing time and space efficiency

3. The satisfaction of users.

The effectiveness of retrieved results are the largest concern, when determing how well a IR system does. Two basic measures are used:

1. Precision: Fraction of retrieved documents that are relevant

2. Recall: Fraction of relevant documents that are retrieved.

There is a tradeoff between P and R: By retrieve more documents, we can increase R, but this might decrease P.

There are several ways to demonstrate the performance:

1.Precision at n documents retrieved:

Show P at fixed point in ranking

2.R-Precision:

P when # of revelant documents retrieved

3.Average Precision

Average P at each relevant document retrieved

4. F-measure

Question:

1. In what level the performance of a IR system can be defined as 'bad'? Is there any standard? for example, when F- measure becomes ***.

Muddiest Point for lecture 4

The lecture about boolean and vector space model is very usful, especially the vector space part. However, 1.I'm still confused about the SMART term weighting, what's the advantage?
2.How can we determine the start point and end point of vector?

Reading Notes for Lecture5

Task:

1.IIR Chapter 11 and Chapter 12

If we have some known relevant and nonrelevant documents, then we can straightforwardly start to estimate the probability of a term t appearing in a relevant document, and that this could be the basis of a classifier that decides whether documents are relevant or not. Chapter 11 introduced some probablistic-based IR model, such as Binary Inde-Model and some more advance one like Bayesian Network Model. I'm wondering that there is the fact that some terms might be relevent between themselves, if the probablity of relevance might be biased if calculated in the way that ignoring the relevance between terms?
The language modeling approach to IR, in Chapter 12, directly models the idea: a document is a good match to a query if the document model is likely to generate the query, which will in turn happen if the document contains the query words often. Such language model includes Query Likihood Model. I think the language modeling is still based on probability, but adding some probabilistic relevance between the sequence of  query and document.

Muddiest Point for Week 3

The topic in lecture 2 is about document and query processing , and the clue of it is quite clear. So, there is no questions on lecture 2.

Reading notes for Week4 (for 1st Feb)

Task:
IIR chapter4 and chapter5

Chapter4 and Chapter5 of IIR introduce some topics about index construction and index compression.
The methods used in index-building mainly include BSBI(Blocked sort-base indexing), SPIMI(single-pass in-memory indexing),distributed indexing and dynamic indexing. The BSBI, SPIMI and distributed indexing are easy to understand, however, I am still confused about the dynamic indexing.
What is the update process in dynamic indexing?
Can professor include some examples to explain the dynamic indexing?
The distributed indexing reminds me Cloud Computing, I am wondering what the indexing process would be if Cloud Computing is applied?

Muddiest Point for week 1

After the first class, I had a overall understanding of what IR and IR system are and understood the main process of search and the selection cycle. But I'm still curious about what competitive advantage such large search engines as Google.com have over others? Is it powerful server they have? or Is it the advanced IR technology? if it is, then what kind of competitive technologies they have?

Reading Notes for week 2&3(Jan 24th)

Task:
1.IIR sections 1.2, chapters 2 and 3.

The reading materials introduced some basic topics on inverted index, including the steps in building an inverted index, several approaches used in processing documents, such as how to do tokenization, and how to process querys and deal with typographical errors in the query. The information provided by IIR is detial enough for me to get an overall understanding of the logics in building an inverted index.

The major steps and related document processing and query processing techniques during each steps are:
1. Collect the documents to be indexed
{ Obtaining char sequence in a doc(decode); Choosing a document unit(precision/recall tradeoff)}
2. Tokenization and linguistic preprocessing.
{char-->token(For english: use whitespace, throw punctuations. However, there are such symbols as apostrophe to consider. Actually, how to tokenize the char depends on what kind of language is used)-->type(class of all tokens containing the same character sequence)-->term(Dropping common terms,such as of, a. Normalization(equivalence classing of terms).Stemming and lemmatization(words with similar meanings)list intersection(skip pointers).positional posting and phrase query(biwords, position index, combination schema)}3.Index documents that each term occurs in.

Though the description is quite detail, it only introduce the techniques individually. How the techniques are combined in practice? And all the information are about boolean query, then How about the Free text query?what kind of processing method applied?

Reading notes for week 1( Jan 10)

Task:

1.      Modern Information Retrieval(MIR): 1.1 – 1.4

2.      Finding out About(FOA): 1.1- 1.5

3.      Implementing and evaluating search engines(IES): 1.1- 1.2

Reading notes:

The first chapters of these three books introduce information retrieval (IR) and information retrieval system (IRS) from different perspectives, e.g. MIR is from technical perspective, FOA is from cognitive perspective, and IES is from application perspective. The three perspectives provide reader a comprehensive understanding of IR and IR system.

MIR explains the basic concepts, IR history and practical issues and main process of IR system. It highlights the difference between information retrieval and data retrieval (1.1.1 information versus data retrieval), makes clear that IR system aims at retrieving information about a subject rather than data which satisfies a given query. In my opinion:

First, information conveys content, but not limited to content. In other words, the information includes data, but beyond the data.

Second, information is systematic, but data is isolated. The relation between information and data is what between movement system and movement organs. The movement of body cannot be done without the coordination of organs. However, organ is not body.

In section 1.2.1 The User Task，author mentions that ’In this book, we make a clear distinction between the different tasks the user of the retrieval system might be engaged in. His task might be of two distinct types: information or data retrieval and browsing.’ And that ’However, combination of retrieval and browsing is not yet a well established approach and is not the dominant paradigm.’ These words triggers my curiosity of the combination of retrieval and browsing. I would like to know how this technology is implemented, where it can be used, and what about the prospects of this technology.

Different from MIR，FOA provides a cognitive perspective on search engine technology and world wide web。This book is characterized by vivid examples and analogy to describe IR technology and application in World Wide Web.

From the introduction (1.1 Finding out about – a cognitive activity），FOA process of browsing readers can be imagined to involve three phases:

1.      Asking a question;

2.      Constructing an answer;

3.      Assessing the answer.

Just as asking a question in daily life: the questioner should first ask a question, answerer answers the question, and then questioner assesses whether the answer matches the question, if not, try to ask in another way.

After providing knowledge on the FOA process， the book also describes various elements associated with it. For example, the section 1.4 documents helps to solve the question about “use of structure of the document’. In addition to the retrieval of context information, the retrieval of meta-data can be of great importance. Meta-data is commonly used to describe three aspects of digital documents and data: 1) definition, 2) structure and 3) administration. By describing the contents and context of data files, the quality of the original data/files is greatly increased. For example, a webpage may include metadata specifying what language it's written in, what tools were used to create it, and where to go for more on the subject, allowing browsers to automatically improve the experience of users.

Based on the technological and cognitive perspectives on IR from MIR and FOA, IES describes the application of the IR system. For example, the web search (1.1.1), file system search (1.1.2), intranet search (1.1.2), etc, and the architecture of the IR system (1.2). I think the architecture could be combined with IR system process in discussing the functions of various parts of IR system.

Overall, through the reading of introduction part of these three books, I have a general understanding on IR and IR system, e.g. IR-related concepts, the architecture of IR system, and IR system workflow. And there are two issues worthy of discussion:

1.      how the combination of browsing and retrieval is implemented, where it can be used, and what about the prospects of this technology.

2.      What are the functions of various parts of IR system in the IR process.