$Revision: 1.41 $

Welcome to inn 1.6

	This is the public release of version 1.6 of InterNet News. This
	work is sponsored by the Internet Software Consortium.

Changes:
	See below.

Who to thank:
	Rich Salz deserves a big thank-you for writing INN in the
	first place and then being kind enough to bless me and the
	ISC as the keepers of the flame. Paul Vixie at the ISC
	provided the financing necessary to keep me in espresso
	coffee through too many sleepless nights (and the boot up
	the backside to make the nights sleepless). Dave Barr did
	a fine job holding the fort while INN was in limbo, and
	getting INN into its current form without the unoff releases
	to work with would have been a much longer job.

	Many, many other people have helped in various ways.  Most
	of the code has been written by other people and my job
	has been to stitch it all together. The CONTRIBUTORS file
	lists who helped.

Support questions, bug reports etc.
	I'm gathering information on who uses INN. If you haven't done so for
	a previous version of INN, then please do the following:

		uname -a | Mail -s "1.6 usage survey" inn-survey@isc.org

	I will appreciate it. You won't get a reply.

	I'm interested in all bug reports. Not just on the programs, but on
	the documentation too. Please send *all* such reports to:

		inn-bugs@isc.org

	even if you post to usenet, please CC the above address. All other
	INN mail should go to:

		inn@isc.org
	
	For general "how do I do this" questions you should post to
	news.software.nntp as there are a lot of experienced INN users
	there, and I don't have the time necessary to help.

	Have fun.

To support the effort:
        Note that INN is supported by the Internet Software Consortium,
        and although it is free for use and redistribution and incorporation
        into vendor products and export and anything else you can think of,
        it costs money to produce.  That money comes from ISP's, hardware and
        software vendors, companies who make extensive use of the software,
        and generally kind hearted folk such as yourself.  

        The Internet Software Consortium has also commissioned a DHCP
        server implementation, handles the official support/release of
        BIND, and supports the Kerberos Version 5 effort at MIT.  You can
        learn more about the ISC's goals and accomplishments from the web
        page at <URL:http://www.isc.org/>.

James Brister
inn@isc.org		(INN related mail)
brister@vix.com		(non-INN mail)

---------------------------------------------------------------------------
Some notes:

+ There is still no GNU configure script for INN.

+ INN now uses a function (snprintf) that some systems don't have (used for
  security reasons). If yours is such a system, then build the ``newlog''
  package that's in the contrib/newlog-1.0.4.tar.gz and then include the
  appropriate ``-l'' flag to LIBS in config.data to find it. See
  README.newlog for some hints and the README file inside the newlog
  package itself.

+ Perl version 5 is now pretty-much mandatory as it's required by innlog,
  and findlow (a new script).

+ A normal 'make update' doesn't install various config files on top of your
  existing ones (for good reason). However I highly recommend you use the
  new control.ctl file. It has a lot more entries in it than the 1.4
  versions and most defaults are changed to log, rather than mail, and also
  to use pgp to verify certain control messages. Installing this file will
  reduce the amount of damage and or mailbombing that occurs with forged
  control messages. More details below.

+ If your system doesn't have a sample config.data file in the sample-configs
  directory (or if the one in that directory for your system is wrong) then
  please send me yours and I'll include it in there.

+ In ftp.isc.org:/pub/usenet/CONFIG you can find some data for
  pre-populating files (like active and newsgroups). Also, the
  administrators of many Usenet hierarchies maintain FAQs containing
  checkgroups, pgp keys and control.ctl entries. In addition the monthly
  "Usenet Hierarchies: Config Files FAQ." contains uptodate control.ctl and
  other information for a large number of hierarchies. This FAQ along with
  many of the others is crossposted to the newsgroup
  news.admin.hierarchies.

+ The control.ctl has changed (again). As before I've taken the control.ctl
  that gets posted as part of the "Usenet Hierarchies: Config Files FAQ."
  and changed a few things:

	The "Idiots List" has been dropped. Some people were insulted about
	being on it, and others made veiled threats of lawsuits.

	Because of the missing idiots list, I've tightened up defaults on
	rmgroups. This means that if you don't read your logs or relax the
	defaults, then you'll miss things.

  The monthly "Usenet Hierarchies: Config Files FAQ." contains up to date
  control.ctl and other information for a large number of hierarchies. This
  FAQ along with many of the others is crossposted to the newsgroup
  news.admin.hierarchies. It does have an idiots list (and is posted by
  someone out of the reach of the fine American justice system (cough)).


---------------------------------------------------------------------------
Changes:

Changes since 1.5.1:

	- Slaving is now done automatically via Xref header
	  parsing. XREPLIC is no longer needed. See the innd(8) manpage and
	  regarding the ``-b'' option.
	- A new ctlinnd command has been added ``lowmark'', to have innd
	  renumber the low marks in the active file from an external data file,
	  rather than scanning the spool.
	- expire can now generate a lowmark file
	- expireover can now expire from a lowmark file (-l' flag).
	- News.daily hash a new optional argument ``lowmark'' to have
	  ctlinnd, expire and expireover create and use a lowmark file.
	- Articles with illegal newsgroup names are now rejected. Legal names
	  must match :
		[a-z0-9][-a-z0-9+_]*(.[a-z0-9][-a-z0-9+_]*)*
	  (actually islower(3) and isdigit(3) are used). This was done to
	  block what may have been DoS attacks. Setting
	  TOSS_BOGUS_NEWSGROUPS to DONT in config.data will turn this off.
	- Improvements in responsiveness to new connections or ctlinnd
	  connections.
	- News.daily's mail message is improved a bit.
	- Improvements to the speed of getting data down to channel processes.
	- All IP_OPTIONS on new connections are logged and cleared.
	- A new header X-Trace is added by nnrpd when articles are posted.
	  It contains the date (in UTC seconds), the pid of the nnrpd
	  process, the IP address (dotted quad form) of the client. 
	  If NNRPD_USE_IDENT is set to DO in config.data, then ident info
	  is added too (but this is of questionable value, so the default is
	  DONT).
	- Another new header X-Complaints-To is added. This is added to new
	  postings by nnrpd. It contains the value defined by the key 
	  ``complaints'' in inn.conf (defaults to ``NEWSMASTER@FROMHOST'', 
	  where FROMHOST is as described in inn.conf(5)).
	- New 'P' value for the 'W' flag in the newsfeeds file (sends the
	  Path header to the site process/file.
	- The LIKE_PULLERS code has been moved to *after* the article is
	  sent and fixed (somewhat--it's still rather ugly) and the default
	  is now off.
	- ctlinnd now includes INN version # and contact information in its
	  'help' output.
	- The Path header is not modified if the new element to be added is
	  the same as the one added by the last machine (useful in slave or
	  certain campus situations).
	- dexpire is now included in the contrib directory. It will
	  probably be the standard expiration agent in the next release of
	  INN. 
	- nnrpd now supports a LIST MOTD command. This will return the
	  contents of the file defined by by _PATH_MOTD (in config.data). A
	  useful place for putting news system notices for clients to see.
	  No clients support this yet, as far as I know. LIST MOTD is
	  expected to be in a future RFC.
	- A new README.SGI file is svailable for Irix users.
	- There is an experimental keyword generator. It adds a Keyword
	  header to the overview with keywords extracted out of articles as
	  they come in. This is off by default as it's a potentially very
	  large cpu hog. See config.data, and overview.fmt. I believe the
	  only reader supporting this overview header is Gnus (via a
	  patch).
	- Innd no longer creates duplicate history entries when
	  REMEMBER_TRASH is DO, and a duplicate is sent with TAKETHIS.
	- If the Perl hooks are used, then the filter_mode subroutine is
	  called when innd is shutdown. The mode is "throttled" and the
	  reason string is "shutting down".
	- A perl script is supplied (innmail) that can be used in place 
	  of UCB Mail as the _PATH_MAIL definition in config.data. This is
	  recommended as UCB mail has too many security holes when used
	  non-interactively.
	- Some defaults are changed in the control.ctl, mostly to reduce
	  the amount of mail that goes out.
	- Many potential buffer overruns fixed. Fixing these requires using
	  snprintf(), which some systems don't have. An implementation is
	  supplied in the tar ball newlog-1.0.4.tar.gz
	- The active file is now be MMAPed by nnrpd when ACT_SYTLE is MMAP
	  in config.data.
	- nnrpd can be built to audit the commands it processes (but not
	  who issues them). Add -DNNRPD_AUDIT to DEFS in config.data and
	  create the directory /var/tmp/nnrpd (or change NNRPD_AUDIT_DIR in
	  nnrpd.c)
	- more sample config.data files in the sample-configs directory.
	- a svr4-style startup script is in samples/nntp.server (copied to
	  site/nntp.server during building). You need to do the right thing
	  with this if you need it. See the comments at the beginning of
	  the script.
	- innxmit will now ignore extra arguments in the batch files, and
	  will write out those same arguments if it has to respool the
	  article (thus permitting more effecient preprocessing filters to
	  the batch files, for example).
	- innwatch will monitor the log disk's free space.
	- nnrpd will accept postings with NNTP-Posting-Host headers in
	  them, but it will silently toss them and set the correct value
	  (previously it refused to accept the post if the header was there).
	- The innshellvars.csh file has had almost all its variables
	  renamed because of a limit on the length of C shell variable
	  names on some systems.
	- New config.data variables: 
			INNWATCH_LOGSPACE
			KEYWORDS
			KEYLIMIT
			ABSURD
			MAX_WORDS
			_PATH_MOTD
			INNWATCH_LOGSPACE
			HAVE_SETPRIORITY
			NNRPD_USE_IDENT
			INND_SETPRIORITY
			NNRPD_NICE_VALUE
			OVERCHAN_NICE_VALUE
			OVERCHAN_NICE_NAME
			INNFEED_NICE_VALUE
			INNFEED_NICE_LEADING_NAME
	- The necessary code and Makefile adjustments have been made to be
	  able to link in the GNU dmalloc library.
	- A line in newsfeeds(5) with only white space is no longer a fatal
	  error.
	- innd gives a reasonable error & usage message to stderr when 
	  failing to start up.
	- The installed ownership and permissions of rnews have changed a
	  bit to help with Taylor UUCP.
	- The '/s' suffix in hosts.nntp now works properly for multi-homed
	  hosts. 
	- The send-ihave script now has a '-d' option for 'delay by 24
	  hours mode'. See the comments inside the script.
	- The sendbatch script (for feeding UUCP machines) has a new '-n'
	  option for specifying the UUCP name when it's different from the
	  value given on the command line (which is often the FQDN).
	- Scanlogs has been sped up.
	- The PGP docs have been rolled into Install.ms
	- A Year-2000 bug was fixed for NEWNEWS and NEWGROUPS command
	  handling in nnrpd. 
	- Various other minor bug fixes.
	- a lot of minor documentation cleanups

			 -------------------------

Changes since 1.5:

	- security hole in parsecontrol fixed.
	- the main 'all' build rule now causes the 'subst' processing to
	  only report files that got changed. Files that didn't get changed
	  are not mentioned.
	- innxmit now has an '-l' flag to turn off the logging of the
	  reason the remote rejected an article.
	- nntpget is fixed when it offers an article to the local server,
	  but the server doesn't want it.
	- bug fix in ihave control message generation.
	- bug fix in takethis command handling (when no message id given).
	- bug fix to scanlogs for certain broken sort(1) program (HP) that
	  put a newline on the end of empty input.
	- innwatch is now started by default in rc.news
	- The full Path header is now passed to perl scripts, rather than
	  just the first entry.
	- bug fixed in expireover when processing folded headers.
	- Mailing of bad control messages is configurable (in config.data)
	- TCL include file path now configurable.
	- The sample config files are more complete.
	- Bug fix in clibrary.h that was causing dbz.c to always use MMAP.
	- Bug fixed in logging of successful posting to moderated group.
	- Bug fix in perl hooks that was corrupting return value from subrs/

			 -------------------------

Changes since 1.5b2:

	- The memory leaks in the perl hooks are plugged. These hooks look
	  to be a very valuable way of trimming make-money-fast posts and
	  excessive multi-postings.

	- If news.daily is called with the 'delayrm' argument, then the
	  temporary file expire writes to is uniquely named (the pid is
	  added to the end). If something kills expirerm before it's
	  finished unlinking expired article, then the next news.daily wont
	  delete the temp file (so you can use it to remove the
	  articles). If expirem finishes properly, then the temp file is
	  moved to its normal spot (MOST_LOGS/expire.list).

	- If a control message is malformed (such as a broken control
	  header), then the bad control message is no longer mailed to the
	  new admin and is simply logged to $MOST_LOGS/badcontrol.log. Admins
	  who lived through the recent swarm of badly forged rmgroup messages
	  will appreciate this I think.

	- Using PGP for control message verification is now the default. It
	  is turned on in for all heirarchies I know of that are managed by
	  someone signing their control messages. I don't have the pgp keys
	  for all these people though. I only have the keys for
	  tale@uunet.uu.net (who does the big-8) and for control@usenet.fr
	  (who manages fr.*).  To turn this off (and I really recommend you
	  don't do that unless you have to) you need to change control.ctl
	  and config.data.

	- If pgp doesn't exist on your system, and you forget fix config.data
	  and/or control.ctl, then all failed verifications due to missing
	  pgp binary, get logged to MOST_LOGS/failedpgp.log.

	- If pgp ran, but fails to verify a control message (e.g. due to a
	  missing signature), then the control message is logged to
	  MOST_LOGS/badpgp.log.

	- nnrpd will kill itself (via an alarm) if it gets hung in sending
	  data to a client. The timeout for this is DEFAULT_TIMEOUT in
	  config.data.

	- Bug fixed in makehistory when creating a history file with the
	  '-s' and '-i' flags. Shoulds result in better dbz performance for
	  such history files.

	- rnews exits with a non-zero exit code if the article it's
	  processing has malformed headers.

			 -------------------------

Rev 1.5b2 is mostly just bug fixes over 1.5b1.

	- a fairly major performance fix (hammering of disk the active file
	  was on).

	- slowed down some unnecessary buffer expansion.

	- bug fix when accepting articles just slightly over max-size limit.

	- The manpages for sections 1 and 8 have been cleaned up to show
	  options more clearly.

	- Various new man pages (some being split up versions of other pages).

	- A new directory authprogs that has a sample authorization program
	  for use with the nnrpd 'authinfo generic'. This is not integrated
	  in any way (not even a makefile) and is more for the curious
	  right now than for serious use. See the readme file in there.

	- A new installation step will fixup the #! line of any perl
	  scripts to match the value you defined in config.data.

	- grephistory has a new '-t' flag.

	- more sample code (commented out) in the perl filters.

	- innlog.pl understands more stuff in the log file.

	- added an innshellvars.csh for those who login as news (or anyone
	  crazy enough to program in the c-shell).

Some random notes:

	This isn't new, but I need to remind some people: don't use 1.4
	binaries with 1.5. The message protocol to/from innd defined in
	lib/inndcomm.c changed for 1.5 and so old programs, or anything
	you've written and linked to an old libinn.a, is doomed to
	fail. This is most evident with ctlinnd.

	dbz.c is distributed in pre-patched form. It's still an old dbz,
	though. 

	The LIKE_PULLERS config variable (if set to DONT--the default) can
	adversely affect certain types of news readers (gnus seems to be
	one). If a human starts complaining about slow reader response,
	they're probably using one of these clients. A more intelligent
	solution than the LIKE_PULLERS will be investigated
	(suggestions welcome). If complaints get bad, set it to DO and
	recompile.


There are still some known problems:

	- The perl hooks leak memory.

	- If the target of a funnel feed gets dropped and the funnel feed
	  itself doesn't get dropped, then the next time a article arrives
	  that should get fed to the funnel entry, innd will core dump.

	  e.g. in newsfeeds

		foo:*:Tm:bar
		bar:!*:Tf,Wnm*:

	  if you 'ctlinnd drop bar', then things go bad very very quickly.

	- The 'offered' count reported by innxmit is wrong if innxmit stops
	  issuing CHECKs and only issue TAKETHIS commands.

			 -------------------------

Differences from the 1.5a3 alpha release.

	+ A contrib directory has been included to add useful things that
	  for one reason or another (usually lack of time on my part) can't
	  be a full part of inn (yet, at least).

	+ Some bug fixes.

	+ new 'notdaily' argument for news.daily. Use this on extra times
	  during the day that you run news.daily. (dneedham).

Things actually in earlier releases but not properly noted in this README
back then:

	+ innd will catch the SIGUSR1 signal and recreate the control
	  channel (for when something deletes the named socket) (dneedham).

	+ 'cmsg' in Subject lines is ignored. The Control header is all tha
	  matters.

	+ inews now asks the server it's talking to for the moderator
	  address info, so copying a moderators file around the network is
	  no longer necessary (dneedham).

---------------------------------------------------------------------------
This is the third and last alpha of inn 1.5.

This release includes the following new features.

	- Perl support based on work done by Christophe Wolfhugel. It's had
	  some testing done, but no extensive experience with things like
	  memory footprint etc. See README.perl_hook for more details.
	- Poison newsgroup flag ``@'' in newsfeeds by John Stapleton. See
	  the newsfeeds(5) man page.
	- John Stapleton's support for misbehaving readers: (a) short timeout
	  on first read (to catch readers who connect and then go away
	  immediately) and (b) restrict the number of times a remote system
	  can connect per N seconds. (See the `-X' `-H' and `-T' flags in
	  the innd.8 man page.)
	- Support for '-e' in expire (expire on earliest crosspost) by John
	  Levine.
	- Perl and tcl versions of innshellvars.
	- If LOG_SIZE is set to DO in config.data, then the size of each
	  article will be put in the log entry.
	- If LIKE_PULLERS is set to DONT, then a small sleep is done on 
	  ARTICLE commands, and on HEAD and BODY after the first 100
	  commands done. This won't hinder normal readers, but will slow
	  down ``sucking'' feeds.
	- Updates to nntpsend an shrinkfile from chongo.
	- Default pathnames in config.data are a little more consistent. 
	- Some bug fixes.	

From here on out, all new features will have to wait for 1.6. This version
will turn into a publically advertised beta test fairly soon. Please hammer
on it and send in all bug reports to <inn@isc.org>.

---------------------------------------------------------------------------

The first alpha was a merge of several things: Rich's original 1.5 alpha
code; the 1.4unoff4 release put out by Dave Barr; Bob Halley's TCL
filtering extension.

The second alpha included:

	- PGP verification of control messages (from tale).
	- actsync program (from chongo)
	- Patches for spooling in nnrpd if innd is unavailable (from dneedham)
	- XBATCH (compressed rnews-ready batches) support (via Matthias U.)
	- The '-x' option from innd is removed and adding
	  Xref headers the standard behaviour. (from dneedham)
	- The message format used by libcomm now includes a protocol 
	  version and a byte-count header (so ctlinnd from previous 
	  version won't work with this version).
	- A new 'p' flag for the newsfeeds entries: the time the article
	  was posted (from Matthias U.)
	- A LIST SUBSCRIPTIONS command for nnrpd (from dneedham).
	- gzip compress recognition to rnews (from Matthias U.)
	- A method of limiting those peers (in hosts.nntp) that can use
	  streaming (based on Christophe W.'s idea).

	Heaps of other more minor patches from various people (see the
	CHANGES) file.

Please send *all* mail regarding inn to me at the following address:

	inn@isc.org

Even if it is going to some mailing list that I'm already on, CC to
inn@isc.org anyway.

I'm especially interested in documentation gaps. Not that I like writing
documentation.

Thanks

James

------------------------------------------------------------------------------
InterNetNews -- the Internet meets Netnews
------------------------------------------
"Remember to tell your kids about the days when USENET was store and
 forward."  -- Jim Thompson, as part of a message that said he was getting
 under 200ms propagation, disk to disk.

InterNetNews is a complete Usenet system.  The cornerstone of the package
is innd, an NNTP server that multiplexes all I/O.  Think of it as an nntpd
merged with the B News inews, or as a C News relaynews that reads multiple
NNTP streams.  Newsreading is handled by a separate server, nnrpd, that is
spawned for each client.  Both innd and nnrpd have some slight variances
from the NNTP protocol; see the manpages.

The distribution is a compressed tar file (compressed with gzip). Unpacking
the tar file creates a new directory. For example:
	; mkdir inn
	; cd inn
	; ftp ftp.isc.org
	ftp> user anonymous <you@your.host.name>
	ftp> type image
	ftp> get isc/inn/inn-1.6.tar.gz
	ftp> quit
	; gzcat < inn-1.6.tar.gz | tar xvf -


The installation instructions are in Install.ms.  This is an nroff/troff
document that uses the -ms macro package, and is about 30 typeset pages.
The distribution has this file split into
two pieces; you can join them by typing either of the following commmands:
	; make Install.ms
	; cat Install.ms.? >Install.ms

To make the Install.txt file without the nroff overstrike encoding
use the following pipe:

 	nroff -ms Install.ms | sed -e 's/.<ctrl-H>//g' > Install.txt

 	Note:   <ctrl-H> is the control-key + "h" and represents the
 		backspace, a non-printable ascii character.

You should probably print out a copy of config/config.dist when you print
out the installation manual.

Please read the COPYRIGHT.  This package has NO WARRANTY; use at your
own risk.

When updating from a previous release, you will usually want to do "make
update" from the top-level directory; this will only install the programs.
To update your scripts and config files, cd into the "site" directory and
do "make clean" -- this will remove any files that are unchanged from
the official release.  Then do "make diff >diff"; this will show you what
changes you will have to merge in.  Now merge in your changes (from
where the files are, ie. /usr/lib/news...) into the files in
$INN/site.  (You may find that due to the bug fixes and new features in
this release, you may not need to change any of the scripts, just the
configuration files).  Finally, doing "make install" will install
everything.

If you have a previous release you will probably also want to update the
pathnames, etc., in the new config file from your old config.  Here is one
way to do that:
	% cd config
	% make subst
	% cp config.dist config.data
	% ./subst -f {OLDFILE} config.data
where "{OLDFILE}" names your old config.data file.

Configuration is done using subst.   Subst is in config/subst.sh and
doc/subst.1.  The history file is written using DBZ.  The DBZ sources and
manual page are in the dbz directory.  Unlike subst, DBZ is kept
separately, to make it easier to track the C News release.  The subst
script and DBZ data utilities are currently at the "Performance Release"
patch date.  Thanks to Henry Spencer and Geoff Collyer for permission to
use and redistribute subst, and to Jon Zeef for permission to use DBZ as
modified by Henry.

This version includes new support for TCL filtering of articles (to either
reject or accept them). This work is done by Bob Halley. See the file
README.tcl_hook for more details.

This version includes support for Geoff Collyer's news overview package,
known as nov.  Nov replaces the external databases used by nn, trn, etc.,
with a common text database.  INN support includes programs to build and
maintain the overview database, and an XOVER command added to nnrpd (the
news-reading daemon) that is becoming a common extension to fetch the
overview data from an NNTP connection.  Nnrpd uses the overview database
internally, if it exists, making certain commands (e.g., XHDR) much
faster.  The nov package includes a newsreader library that you will need,
and some utilities that you will not; it is available on world.std.com in
the file src/news/nov.dist.tar.Z.  Prototypes of modified newsreaders are
in the in src/news/READER.dist.tar.Z -- most maintainers will be providing
official support very soon.  To make it explicit:  if you already have a
newsreader that can use the overview database, either via my NNTP xover
command, or by reading directly from NFS, then INN has all you need.

