Mumps Book
Direct link to Mumps Book PDF

Data bases, examples and text
Mumps Pocket Guide JPG images

Term Project (25%)
Assignments (40%) Late assignments charged 5% per class day. Chronically late assignments will be charged at a higher rate.
Tests (2 at 10% each)
Final (15%).

Classes are lecture format. Cell phones, pagers, laptops and PDA's may not be used.

Assignments must include:

Projects will be presented in class the last week of classes.

All projects will be presented in class with a brief online demonstration. A vote will be taken for the best project and the winner will be exempt from the final (grade of A will be entered for the final exam grade). You may work in teams of from 1 to 3 (hint: have one person do the indexing, one the retrieval code and the other do the web interface). All group members will receive the same grade.

1. Get and install Ubuntu either native, under WUBI or Sun VirtualBox. or Cygwin for Windows
2. Get and install Mumps under Cygwin: In Cygwin, type:
   1. wget http://cns2.uni.edu/~okane/source/MUMPS-MDH/mumpscompiler-10.0.src.tar.gz
      (Check http://cns2.uni.edu/~okane/source/MUMPS-MDH/ for the latest version number).
   2. tar xvzf mumpscompiler-10.0.src.tar.gz
   3. cd mumpsc
   4. ./configure prefix=/usr
   5. make
   6. make install
3. Learn an adult editor (nano/pico are for children) : vi Tutorial
4. Learn Mumps.

1. Using the reuters-filtered dataset, write a program to calculate:
   
   - the frequency of occurrence of each word.
   - the total number of titles/articles
   
   Write out for each word its frequency of occurrence, a blank and the word.
   
   Sort the result in reverse order (high to low)
   
   Read the resulting dataset and write out for each word in
   neat columns:
   
   the frequency of occurrence of the word
   the frequency of occurrence of the word times its rank
   the word.
   
   Turn in the code and the first page of the results.
   
   Notes: 
   
   The dataset is linked from my web page (reuters-filtered)
   and it is in /f/ISR/filtered-reuters on sidhe.cs.uni.edu
   
   If you use sidhe, do not make a copy of the dataset. Create a link:
   
   ln -s /f/ISR/reuters-filtered reuters-filtered
   
   I do not want 27 copies of the dataset on the machine. one is enough.
   the ln command will create a link in your directory back to
   the original file. The link will behave as though it were the
   file itself.
   
   'rank' means the order in the file, first is rank 1, second is
   rank 2 and so forth. freq*rank is known as Zipf's constant.
   
   You can sort in reverse order with something like:
   
   sort -nr < in  > out
   
   The values from the freq*rank may need to be formatted. Use the 
   justify() function. do not print endless decimal places.
   
   Each title has xoxoxo at the beginning of the title and xoxoxo at the
   end of the title. Include title words in your dictionary. use 
   $zzScanAlnum to read the input. This will eliminate punctuation
   and convert the words to lower case.
   
   Due: Fri Feb 6
   

2. Using the Reuters data base:
   
   1. Build a file of stop list words consisting of very high and very low 
      frequency words. You may use a text editor for this.
   2. Build a document term matrix. Do not include words which are in the stop list. 
      stem (remove endings) of the words that remain. Discard stems less that 3
      characters in length. For each stem in each document, store the frequency
      of occurrence of each stem.
   3. When finished, compute a document frequency vector giving for each stem the
      number of documents it occurs in.
   
   To turn in:
   
   - The first ten document vectors giving document number, word stems and frequency.
   - The 20 stems which occur in the most number of documents.
   - Program code.
   
   Note: there is a stem function in mumps.
   
   due: Fri 20 Feb
   
   (this is about 20 lines of code)
   

3. From the document frequency vector and doc-term
   matrix in assignment 2, calculate the IDF weight
   of each term and replace the frequencies in the
   doc-term matrix by the frequency times the IDF for
   the word.
   
   From the doc-term matrix, build build the 
   transposed index(word,doc) matrix.
   
   Turn in: the top ten IDF weights, the 
   weighted document vectors for the first ten 
   documents and the weighted term vectors for
   the first ten words in index.
   
   Due Fri Mar 13.
   

4. Calculate the term-term matrix for the Reuters database 
   and then calculate the cohesion value for terms with
   a significant number of co-occurrences.
   
   what to turn in:
   
   - a sample of the term-term matrix giving the terms and their
     co-occurrence count (about ten terms and their related
     terms)
   
   - the top ten phrases based on the cohesion values.
   
   - the code.
   
   Due: Fri April 3
   

Documentation and Availability
Documentation Descriptor Data Elements
Descriptor Records
Documentation Qualifier Data Elements
Qualifier Records
Supplementary Records

Resources:

Salton, G., Automatic Informatiuon Organization and Retrieval, McGraw Hill (1968)
Salton, G., Automatic Text Processing, Addison-Wesley (1989)
Salton, G., ed., The SMART Retrieval System, Prentice-Hall (1971).
Salton, G., and McGill, M., Introduction to Modern Information Retrieval, McGraw-Hill, (1983)
Borko, H., Automated Language Processing, (1968)
The Smart System from Cornell: ftp://cs.cornell.edu/pub/smart