GLIMPSE 3.0: searching entire file systems

Glimpse version 3.0 improves the original version in many ways.  The
main changes are listed below.  Glimpse is a very powerful indexing and
query system that allows you to search through all your files very
quickly.  It can be used by individuals for their personal file systems
as well as by organizations for large data collections.  Glimpse is the
default search engine in Harvest (see below).

Glimpseindex, which you run by saying "glimpseindex DIR" builds an
index of all text files in the tree rooted at DIR.  
(e.g., glimpseindex ~ indexes all your files.) With it, glimpse can
search through all files much the same way as agrep (or any other
grep), except that you don't have to specify file names and the search
is fast.  For example,

	glimpse -1 unbelievable

will find all occurrences (in all your files!) of "unbelievable" 
allowing one spelling error;

	glimpse -F mail arizona 

will find all occurrences of "arizona" in all files with "mail" somewhere
in their name;

	glimpse  'Arizona desert;windsurfing' 

will find all lines that contain both "Arizona desert" and "windsurfing".

Glimpse supports three types of indexes: a tiny one (2-3% of the
size of all files), a small one (7-9%), and a medium one (20-30%).
The larger the index the faster the search.
Glimpse supports most of agrep's options (agrep is our powerful version
of grep, and it is part of glimpse) including approximate matching
(e.g., finding misspelled words), Boolean queries, and even some
limited forms of regular expressions.

The WWW home page for glimpse is in
    http://glimpse.cs.arizona.edu:1994/

HTML version of glimpse manual pages is in
    http://glimpse.cs.arizona.edu:1994/glimpsehelp.html
HTML version of glimpseindex manual pages is in
    http://glimpse.cs.arizona.edu:1994/glimpseindexhelp.html
HTML version of glimpseserver manual pages is in
    http://glimpse.cs.arizona.edu:1994/glimpseserverhelp.html

The complete source code for glimpse and glimpseindex, as well
as manual pages and other stuff can be obtained from
    ftp://cs.arizona.edu/glimpse/glimpse-3.0.src.tar.Z
Sparc SunOS 4.1.1 executables are in
    ftp://cs.arizona.edu/glimpse/glimpse-3.0.bin.sunos.4.1.1.tar.Z 
Sparc SunOS 4.1.3 executables are in
    ftp://cs.arizona.edu/glimpse/glimpse-3.0.bin.sunos.4.1.3.tar.Z 
Solaris executables are in
    ftp://cs.arizona.edu/glimpse/glimpse-3.0.bin.solaris.tar.Z 
Dec Alpha OSF/1 executables are in
    ftp://cs.arizona.edu/glimpse/glimpse-3.0.bin.alpha.tar.Z

There are also binaries for AIX, SGI, Linux, and HP.  
(We thank Heinrich Stamerjohanns, Chris Dalton, cmelikian@VNET.IBM.COM,
Jim Hurley, and Piroz Mohseni for those.) 
More binaries will be posted as they become available.

An article describing the ideas behind the design of glimpse is in
ftp://cs.arizona.edu/glimpse/glimpse.ps.Z

GlimpseHTTP home page is in
    http://glimpse.cs.arizona.edu:1994/ghttp/
(GlimpseHTTP is a Glimpse add-on tool to provide full-text
search for your WWW server)

Harvest's WWW home page is
	http://harvest.cs.colorado.edu/
(Harvest is an integrated set of tools to gather, extract,
organize, search, cache, and replicate relevant information
across the Internet.) 

Mail glimpse-request@cs.arizona.edu to be added to the glimpse mailing list.
Mail glimpse@cs.arizona.edu to report bugs, ask questions, discuss
tricks for using glimpse, etc.  (This is a moderated mailing list.)

Udi Manber, Burra Gopal, and Sun Wu.

Main changes and additions in version 3.0:

2.1 ---> 3.0

- added a data structure (in .glimpse_turbo) that speeds up queries
  using -w and -i considerably for large indexes.  It is meant mostly for
  servers using glimpse (e.g., Harvest and glimpseHTTP servers),
  but it benefits everyone.  With this "turbo" option, typical queries
  take less than a second even for very large indexes.
  This was so successful that we made it the default rather than an
  option (it used to be -T in some earlier versions).
  If the .glimpse_turbo file is deleted, glimpse will still work properly
  (but glimpseindex -f and -a require it).
- incremental indexing is now fully supported (even for -b).  Deletion
  from the index is supported.  glimpseindex -d filename(s) completely
  deletes the files from the index;  glimpseindex -D filename(s) deletes
  the files only from the file list.
- the index has been improved in several ways (transparently except for
  speed and space).  As a result, indices built with earlier versions of
  glimpseindex will not work with 3.0 -- you must reindex again. 
- several options were added to glimpseindex and glimpse:
  glimpseindex -E indexes all file without attempting to run the filetype
  filtering (but excluded files or suffixes still apply).
  glimpse -Q extends -N in a nice way giving much more information about
  the matches in the index.
  glimpse -L has more options:  -L x | x:y | x:y:z
  if one number is given, it is a limit on the total number of matches.
  Glimpse outputs only the first x matches.
  If two numbers are given (x:y), then y is an added limit on the total
  number of files.
  If three numbers are given (x:y:z), then z is an added limit on the
  number of matches per file.
  If any of the x, y, or z is set to 0, it means to ignore it
  (in other words 0 = infinity in this case);  for example,
  -L 0:10 will output all matches to the first 10 files that
  contain a match.
  (There are also some undocumented-as-yet options. We are running out
  of letters.  Only -j and -Y are not used!)
- glimpse 3.0 still has a LOT of makefiles (one per architecture / OS). 
  We hope to include autoconf support for glimpse in the future: 
  but these should be sufficient for most purposes.
- several bugs were fixed, and the whole package is now more portable.
  Binaries and make files for the following platforms are now available:
  AIX-3.2.5, HPPA, HPMC68K, IBM-RS6000, Linux, SGI. (Binaries and make
  files for SUNOS4.1.1, SUNOS4.1.3, SOLARIS 5.3 and DEC OSF/1 (ALPHA)
  are avaialable as usual.) See README.install for more details.

