$Id: THEORY,v 1.3 1993/05/17 21:02:10 alden Exp $

This document is designed to explain how nntplink works when using either
the logfile or batchfile input modes.  If this doesn't answer all your
questions please send them to me so that I may improve upon this document.

LOGFILE
=======
NNTPlink is designed to be able to read all of the information that is
necessary from the logfile (and possibly the history file) so that you
don't need the overhead of creating batchfiles.  It does this by matching
the specified sysname to each line found in the logfile.  If it matches
then nntplink grabs the message-id (which is in the logfile) and the
pathname to the article (which is in the logfile if you've applied my
patches or from the history file) and attempts to send the article.

When you age the logfile each evening you need to send nntplink the HUP
signal (which can easily be done with the "links" script - please see
extra/README.LINKS for more information).  When nntplink receives this
signal it creates a batchfile which contains the rest of the unsent
articles, closes the old logfile and opens the new logfile. But, before
it starts sending articles from the new logfile it first opens the
batchfile it just created and sends all of the articles from there first
and then continues on with the new logfile.

If you send nntplink the TERM signal then it reads to the (current) end of
the logfile, creates a batchfile from this information and then exits.

NNTPlink is able to read either "virgin" logfiles or the "new and improved"
logfiles (those that have been written by a patched version of C News, see
the file extra/README.PATCHES for more information).  For optimal use I would
suggest that you install the patches since not installing them forces
nntplink to do a history lookup for each article to find the pathname to
the article.  However, I have found that even without the patches you are
better off using the logfile input mode as opposed to the batchfile input
mode since there is quite a bit less disk i/o without batchfiles.

Please note that there are only 5 formats of logfiles which are supported.
They are the "basic" C news format with/without the nntplink patches (see
REAMDE.CNEWS for more info), the "basic" B news format with/without the
nntplink patches (see README.BNEWS for more info), and the "special" logfile
generated by innd (please see README.INN for more info).  For example, a
"basic" C news format (with the patch) is:

May 15 01:55:37.970 usc + <1991May15.055006.24652@ucselx.sdsu.edu> (alt/folklor
e/computers/12390) n8emr caen pacific.mps.ohio-state.edu

While the "basic" B news format (with the patch) is:

May 15 16:26    zaphod.mps.ohio-state.edu       <1991May15.055006.24652@ucselx.
sdsu.edu> (alt/folklore/computers/12390) sent to linac, phyvsf


BATCHFILE
=========
When nntplink is using the batchfile input mode it no longer has the problem
that used to plague nntplink (and other transmission clients), the lack of
true locking on the batchfile.  If the news system was attempting to update
the batchfile at the same time nntplink (or other transmission clients :-)
was trying to send articles from it, it was possible (and probable) that
articles would get lost.  For example, (BTW, I'm using C News pgms in this
example :-), relaynews creates a new batchfile but before it gets a chance
to actually write anything into it, nntplink sees it, opens it, finds it
empty and so removes it, in the meantime relaynews goes ahead and writes it
entry into it and then closes the batchfile - since nntplink had already
removed it, it then gets deleted from the system.

The way I've gotten around this problem (without using any high-overhead
locking schemes) is to only open the batchfile for reading.  Then I tail
the batchfile watching for newly queued articles.  Then, after <n> articles
I rename the batchfile to batchfile.<pid> and continue tailing batchfile.<pid>.
When I reach the end of batchfile.<pid> I check for a new batchfile, if there
is one then I know it is safe to remove batchfile.<pid>, so I remove it and go
back to tailing batchfile, otherwise if there isn't a new batchfile I continue
tailing batchfile.<pid> until there is one.
