(:source
  :version		3
  :ip-address		"128.138.243.151"
  :ip-name		"ftp.cs.colorado.edu"
  :tcp-port		8000
  :database-name 	"aftp-cs-colorado-edu"
  :cost 		0.00
  :cost-unit 		:free
  :maintainer 		"hardy@cs.colorado.edu"
  :description 	
  "You may use this WAIS server to search and retrieve files from the
anonymous ftp archive on ftp.cs.colorado.edu [128.138.243.151].  We
used Essence, a resource discovery system based on semantic file
indexing, to build the WAIS index for this server.

Essence exploits file semantics to index both textual and binary
files.  By exploiting semantics, Essence extracts keywords that
summarize a file, and generates a compact yet representative index.
Essence understands nested file structures (such as uuencoded,
compressed, ``tar'' files), and recursively unravels such files to
generate summaries for them.  Essence generates indexes that are ten
times smaller than WAIS indexes, but retain the fine-grained
information access that WAIS's full-text indexes provide.

Furthermore, Essence generates WAIS-compatible indexes allowing WAIS
users to make use of Essence's indexing capabilities.  This is one of
the ways that the Networked Resource Discovery Project at the
University of Colorado has extended the conceptual paradigm of the type
of information that WAIS handles.

If you would like to learn more about Essence, you can obtain the
source to the Essence prototype and a paper which appears in the 1993
Winter USENIX Technical Conference, San Diego, CA, January 1993.  Both
the paper and the prototype are available via anonymous ftp from
ftp.cs.colorado.edu in /pub/cs/distribs/essence.  Or search for the
keyword 'Essence' using this WAIS server to find all of the files on
ftp.cs.colorado.edu that are related to Essence; you will find the
files for both the paper and the prototype.

This WAIS server was created in December 1992 by Darren R. Hardy and
Michael F. Schwartz as part of the Networked Resource Discovery
Project.  You may reach them at the Department of Computer Science,
University of Colorado, Boulder, CO  80309-0430, or via email at
hardy@cs.colorado.edu and schwartz@cs.colorado.edu.

Below is some more information about the WAIS interface to Essence.

	Essence exports its indexes through WAIS's search and
	retrieval interface, allowing users to use tools such as
	waissearch and the X Windows-based graphical user interface
	xwais.  In order to generate WAIS-compatible indexes,
	Essence uses WAIS's indexing software to index the Essence
	summary files.  This mechanism generates full-text WAIS
	indexes from the Essence summary files.

	We modified the WAIS indexing mechanism to understand the
	format of the Essence summary files, so that it generates
	meaningful WAIS headlines.  These headlines provide users
	with a short description of a single file, usually a
	filename.  With Essence, headlines represent a file's core
	filename, its actual filename, and its file type.

	To support additional file types, WAIS must be recompiled
	with new procedures that understand these file types.  With
	Essence, one need only write a new summarizer, add its name
	to a configuration file, and add new heuristics for
	identifying the file type; no recompilation is necessary.
	In this sense, Essence modularizes the typed-file indexing
	extensions that WAIS can use, because it removes the
	keyword extraction process from WAIS and places it instead
	in Essence.  Essence is better suited to incorporating new
	file types, and can be quickly adapted to become a
	comprehensive indexing system.

	The following waissearch output shows an example search of
	an index generated by Essence of the ftp.cs.colorado.edu
	anonymous FTP file system.  It shows an ordered list of the
	ten files that best match the keyword netfind.  Netfind is
	an Internet user directory service.  The headlines have up
	to three fields representing the matching file: the core
	filename, the filename (if different from the core
	filename), and the file type.
 
------------------------------------------------------------

csh% waissearch netfind
   1:  /cs/ftp/techreports/schwartz/PostScript/Techniques.Wide.Area.ps.Z 
       Techniques.Wide.Area.ps PostScript

   2:  /cs/ftp/techreports/schwartz/PostScript/ALL.PS.tar.Z 
       PostScript/Techniques.Wide.Area.ps PostScript

   3:  /cs/ftp/distribs/netfind/netfind3.10.tar.Z ServerShell/nsh.c C

   4:  /cs/ftp/distribs/netfind/README  README

   5:  /cs/ftp/distribs/netfind/netfind3.10.tar.Z README README

   6:  /cs/ftp/distribs/netfind/netfind3.10.tar.Z Doc/netfind.1 ManPage

   7:  /cs/ftp/techreports/schwartz/PostScript/Proj.Overview.ps.Z 
       Proj.Overview.ps PostScript

   8:  /cs/ftp/techreports/schwartz/PostScript/RD.Comparison.ps.Z 
       RD.Comparison.ps PostScript

   9:  /cs/ftp/techreports/schwartz/PostScript/ALL.PS.tar.Z 
       PostScript/Proj.Overview.ps PostScript

   10: /cs/ftp/techreports/schwartz/PostScript/ALL.PS.tar.Z 
       PostScript/RD.Comparison.ps PostScript
csh%

------------------------------------------------------------

	Consider the effectiveness of the example search shown
	above.  The best match is a PostScript paper that discusses
	a number of techniques for distributed information systems,
	with particular emphasis on techniques demonstrated by
	Netfind; the second match is the same file, but found in
	the compressed tar distribution ALL.PS.tar.Z.  The third
	match is the C source code for the interactive user
	interface to Netfind.  The fourth match is the README file
	found in the Netfind distribution directory; the fifth
	match is the same file, but found in the compressed tar
	distribution netfind.3.10.tar.Z.  The sixth match is the
	UNIX manual page for Netfind.  The remaining matches are
	PostScript papers in which Netfind is discussed.

	In WAIS, a user retrieves files by selecting a matching
	headline.  With Essence, if the headline represents a file
	hidden within a nested file (such as the first headline in
	the example), the summary file is retrieved, instead of
	retrieving the hidden file itself.  If the headline
	represents a plain file (such as the fourth headline in the
	example), the file is retrieved.  This functionality
	requires allocating storage for both the required summary
	files and the index.  However, it allows users to browse
	through remote file systems by retrieving and viewing small
	summary files without having to retrieve complete files.
	This is useful when trying to decide whether to transfer
	large files across a slow network.
"
  )
