Muddiest Point for Week 3

The topic in lecture 2 is about document and query processing , and the clue of it is quite clear. So, there is no questions on lecture 2.

Reading notes for Week4 (for 1st Feb)

Task:
IIR chapter4 and chapter5

Chapter4 and Chapter5 of IIR introduce some topics about index construction and index compression.
The methods used in index-building mainly include BSBI(Blocked sort-base indexing), SPIMI(single-pass in-memory indexing),distributed indexing and dynamic indexing. The BSBI, SPIMI and distributed indexing are easy to understand, however, I am still confused about the dynamic indexing.
What is the update process in dynamic indexing?
Can professor include some examples to explain the dynamic indexing?
The distributed indexing reminds me Cloud Computing, I am wondering what the indexing process would be if Cloud Computing is applied?

Muddiest Point for week 1

After the first class, I had a overall understanding of what IR and IR system are and understood the main process of search and the selection cycle. But I'm still curious about what competitive advantage such large search engines as Google.com have over others? Is it powerful server they have? or Is it the advanced IR technology? if it is, then what kind of competitive technologies they have?

Reading Notes for week 2&3(Jan 24th)

Task:
1.IIR sections 1.2, chapters 2 and 3.

The reading materials introduced some basic topics on inverted index, including the steps in building an inverted index, several approaches used in processing documents, such as how to do tokenization, and how to process querys and deal with typographical errors in the query. The information provided by IIR is detial enough for me to get an overall understanding of the logics in building an inverted index.

The major steps and related document processing and query processing techniques during each steps are:
1. Collect the documents to be indexed
{ Obtaining char sequence in a doc(decode); Choosing a document unit(precision/recall tradeoff)}
2. Tokenization and linguistic preprocessing.
{char-->token(For english: use whitespace, throw punctuations. However, there are such symbols as apostrophe to consider. Actually, how to tokenize the char depends on what kind of language is used)-->type(class of all tokens containing the same character sequence)-->term(Dropping common terms,such as of, a. Normalization(equivalence classing of terms).Stemming and lemmatization(words with similar meanings)list intersection(skip pointers).positional posting and phrase query(biwords, position index, combination schema)}3.Index documents that each term occurs in.

Though the description is quite detail, it only introduce the techniques individually. How the techniques are combined in practice? And all the information are about boolean query, then How about the Free text query?what kind of processing method applied?

Reading notes for week 1( Jan 10)

Task:

1.      Modern Information Retrieval(MIR): 1.1 – 1.4

2.      Finding out About(FOA): 1.1- 1.5

3.      Implementing and evaluating search engines(IES): 1.1- 1.2

Reading notes:

The first chapters of these three books introduce information retrieval (IR) and information retrieval system (IRS) from different perspectives, e.g. MIR is from technical perspective, FOA is from cognitive perspective, and IES is from application perspective. The three perspectives provide reader a comprehensive understanding of IR and IR system.

MIR explains the basic concepts, IR history and practical issues and main process of IR system. It highlights the difference between information retrieval and data retrieval (1.1.1 information versus data retrieval), makes clear that IR system aims at retrieving information about a subject rather than data which satisfies a given query. In my opinion:

First, information conveys content, but not limited to content. In other words, the information includes data, but beyond the data.

Second, information is systematic, but data is isolated. The relation between information and data is what between movement system and movement organs. The movement of body cannot be done without the coordination of organs. However, organ is not body.

In section 1.2.1 The User Task，author mentions that ’In this book, we make a clear distinction between the different tasks the user of the retrieval system might be engaged in. His task might be of two distinct types: information or data retrieval and browsing.’ And that ’However, combination of retrieval and browsing is not yet a well established approach and is not the dominant paradigm.’ These words triggers my curiosity of the combination of retrieval and browsing. I would like to know how this technology is implemented, where it can be used, and what about the prospects of this technology.

Different from MIR，FOA provides a cognitive perspective on search engine technology and world wide web。This book is characterized by vivid examples and analogy to describe IR technology and application in World Wide Web.

From the introduction (1.1 Finding out about – a cognitive activity），FOA process of browsing readers can be imagined to involve three phases:

1.      Asking a question;

2.      Constructing an answer;

3.      Assessing the answer.

Just as asking a question in daily life: the questioner should first ask a question, answerer answers the question, and then questioner assesses whether the answer matches the question, if not, try to ask in another way.

After providing knowledge on the FOA process， the book also describes various elements associated with it. For example, the section 1.4 documents helps to solve the question about “use of structure of the document’. In addition to the retrieval of context information, the retrieval of meta-data can be of great importance. Meta-data is commonly used to describe three aspects of digital documents and data: 1) definition, 2) structure and 3) administration. By describing the contents and context of data files, the quality of the original data/files is greatly increased. For example, a webpage may include metadata specifying what language it's written in, what tools were used to create it, and where to go for more on the subject, allowing browsers to automatically improve the experience of users.

Based on the technological and cognitive perspectives on IR from MIR and FOA, IES describes the application of the IR system. For example, the web search (1.1.1), file system search (1.1.2), intranet search (1.1.2), etc, and the architecture of the IR system (1.2). I think the architecture could be combined with IR system process in discussing the functions of various parts of IR system.

Overall, through the reading of introduction part of these three books, I have a general understanding on IR and IR system, e.g. IR-related concepts, the architecture of IR system, and IR system workflow. And there are two issues worthy of discussion:

1.      how the combination of browsing and retrieval is implemented, where it can be used, and what about the prospects of this technology.

2.      What are the functions of various parts of IR system in the IR process.