\input texinfo	@c -*-texinfo-*-  Use TeXinfo 2.100 or later
@comment %**start of header (This is for running Texinfo on a region.)
@setfilename funetnje.info
@settitle FUNET-NJE 3.0 manual
@setchapternewpage odd
@comment %**end of header (This is for running Texinfo on a region.)

@iftex
@afourpaper
@finalout
@end iftex

@c
@c Notes for writing the doc:
@c
@c  - Support
@c  - History
@c  - Introduction to NJE-networking, esp. BITNET
@c  - General talk about BITNET, and BITNET-II
@c  - Installation
@c  - Configuration
@c      - link configuration et.al. (funetnje.cf)
@c      - route databases
@c      - file-exit.cf
@c      - msg-exit.cf
@c  - User programs
@c      - qrdr
@c      - receive
@c      - sendfile/print/punch/submit
@c      - send
@c      - bitsend
@c      - bitreceive
@c      - ygone
@c      - iucvtrap
@c      - transfer
@c  - System programs
@c      - bmail
@c      - mailify
@c      - ucp
@c      - namesfilter
@c      - njeroutes
@c      - acctcat
@c  - Interfacing system mailer and BITNET
@c      - Zmailer
@c      - smail 3.0
@c      - sendmail
@c  - Technical documentation
@c      - Directory structure
@c      - client -> emulator messaging
@c      - iucvtrap <-> emulator messaging
@c      - spool file format
@c      - implemented NETDATA features
@c      - emulator logics
@c  - Literature
@c

@ifinfo
This file documents FUNET-NJE version 3.0, which implements
BITNET-II -type BITNET node-to-node connectivity on UNIX systems.

Copyright @copyright{} 1994 Finnish University and Research Network,
FUNET.

Copyright @copyright{} 1994 IBiS Support OY

Permission is granted to copy and use this material for non-commercial
use without prior agreenment.
@end ifinfo

@titlepage
@title FUNET-NJE 3.0
@subtitle @titlefont{NJE-emulator for UNIX systems}
@subtitle **DRAFT**DRAFT**DRAFT**DRAFT**DRAFT**DRAFT**DRAFT**DRAFT**
@author Matti Aarnio
@author Finnish University and Research Network, FUNET
@page
@vskip 0pt plus 1filll
Copyright @copyright{} 1994 Finnish University and Research Network,
FUNET.

Copyright @copyright{} 1994 IBiS Support/Matti Aarnio

Permission is granted to copy and use this material for non-commercial
use without prior agreenment.
@end titlepage

@comment Info menus

@c ================================================================
@node   Top,    Support,   (dir),  (dir)
@comment node-name, next,       previous, up

@menu
* Support::             Source of support for FUNET-NJE
* History::             History of the FUNET-NJE package
* Introduction::        Introduction to NJE-networking
* BITNET-II::           Structure of Second Generation BITNET on virtual links
* Installation::        Installation of FUNET-NJE package
* Configuration::       Configuration of FUNET-NJE package
* User programs::       Documentation on user programs for NJE use
* System programs::     Documentation on system programs for NJE use
* Interfacing mailer::  Interfacing  sendmail/smail/Zmailer to BITNET
* Nuts and Bolts::      Technical documents about insides of things
* Literature::          Some of the documents used while developing this system
@end menu
@c detailed menu for reaching all details ?


@c ================================================================
@node Support, History, Top, Top
@chapter Support of the FUNET-NJE package

Academic version of this software is available from:
@example
  ftp://ftp.funet.fi/pub/unix/networking/bitnet/funetnje-@emph{datecode}.tar.gz
@end example

@emph{Academic version of this software comes AS-IS without
        any guarantees of working for any particular purpose,
        or that it is bug-free, or that it won't cause trouble,
        loss of valuable data, etc. -- for short, you are
        essentially on your own.}

Like all Academic software, there might come up a bugfix, or
improvement sometimes, but such effort being non-funded, and
without any schedules, there is no guarantee of anything.

User-community is welcomed to subscribe mailinglist:
@example
    $ Mail mailserver@@nic.funet.fi
    Subject: xxxx
    subscribe funetnje Your Name
@end example
Then corresponding via address: @emph{funetnje@@nic.funet.fi}

@emph{Remember: LIST administrativia goes via mailserver!}

When registering this software to your @emph{BITNET}-node,
I request that you use name: @strong{FUNETNJE}


@c Developement continues under the company  @emph{IBiS Support OY}
@c founded by Matti Aarnio
@c @c , and Gerald Hanusch,
@c for producing customized editions of this software for parties
@c who want to pay for the customization service, or just for timely
@c implementation/bug-fixing something.

If you want to pay for hand-holding service/timely bugfixing,
special customizations, etc. please contact author:
@itemize
@item @emph{<mea@@nic.funet.fi>}
@end itemize




@c ================================================================
@node History, Introduction, Support, Top
@chapter History of the FUNET-NJE package

@emph{``Credit where the credit is due to''}

Original version of this software is known as @emph{HUJI-NJE},
which is VAX-VMS software for implementing BITNET-II, and several
BISYNC modes of the NJE network.
It was made at the @emph{Hebrew University of Jerusalem} (HUJI),
by Mr. @emph{Yehavi Bourvine} over the years 1988-1993.

Sometime around 1990-1991 FUNET archive server was decided to
connect to the EARN, and the best (= free in source form) technology
for it turned to be HUJI-NJE.

The original HUJI-NJE is monsterous monolithic program which has
built-in processing of incoming email, and translation of files
(in and out) in between ASCII and EBCDIC.

At first Matti Aarnio <mea@@nic.funet.fi> did minor porting of the
program to get it to work on a Sun4/630 server (nic.funet.fi),
which did surface several problems in between VAX, and other
architectures.

Latter the HUJI-people lost their interest on the system, and when
no radical improvements were on sight, begun the more serious porting
of the system to Sun4 machine.

Then on Summer-93 came a contact from Mr. Gerald Hanusch <K000165@@ALIJKU65>,
who wanted to improve the then-present version of FUNET-NJE to be more
complete in respect of its emulation of the RSCS for replacing their
aging VM/XA CMS machine -- they had administrative systems on another
MVS machine, and communication to it was to be preserved, although the
VM/XA CMS machine was turned off.

Over the months the system improved, and it was considered that all the
necessary major parts of the NJE-emulator were functional on January-94.

Up until then the software developement was on ``fun to do'' basis,
and beside the actual job of the parties involved.

See previous chapter about @emph{Support} for more about the supported
version.


@c ================================================================
@node Introduction, BITNET-II, History, Top
@chapter Introduction to NJE-networking

Traditionally NJE-networking happens (did happen) in between two
IBM mainframes via BISYNC cable on 4800/9600 bps speed.

Used protocols do vary, but most widely used ones have multiple
interleaved streams etc. in them, and they keep a tab on line
condition by timeouting, if acknowledgements don't appear quick
enough.
The protocols also have a sort of CRCs in them for every ``packet''
they send over the wire.



Things that are possible with NJE-network are:
@table @strong
@item Unsolicited file transfer
Sending files to the recipients without
them being aware of files coming to them, nor in fact asking any
authorization prior sending it to them.

Especially this contradicts the way how the FTP works.

This happens in @emph{Store-and-Forward}-manner, that is, when the file
is sent over one link, responsibility for it is transferred to the
next node, and link-sender can delete it from its own spool.

@item Remote Job Entry
Sending batch-jobs to remote machine for execution, especially to
MVS systems.

@item Nodal Messaging
Sending short (max 120 chars) lines of messages to recipients at
other nodes.
@end table

With these basic facilities it is possible to build extended services,
like:
@itemize @bullet
@item
Remote printing

@item
Email

@item
File transfer

@item
Remote Job Execution (Batch processing)

@item
Interactive discussion one line at the time via NMR-messages

@item
Interactive control of servers accepting NMR-messages
@end itemize


This networking mechanism allows to have simple phone-line quality
links to be used to transport huge amounts of data over long distances.
It may be somewhat slow at the times, but it works@dots{}

An entry-level BITNET connectivity requires capability to communicate
via synchronous BISYNC interface, although these days BITNET-II technology
has largely surplanted it.  See on next chapter.

There exists even ASYNC NJE, although that is more like experimental
kludge for hackers, than serious production tool. (Not supported by
the FUNET-NJE.)

Names in the BITNET  are at most 8 characters long.
This applies both to the users, and the machines (@emph{nodes}).

Routing in the BITNET requires every machine to know every other, and
therefore there exists fairly large bureaucracy for keeping the linkage
information up to date.

Full routing updates are done in semi-monthly manner, which updates
can usually be incorporated into the system with minimal fuzz, or in
case of FUNET-NJE, automatically.


There are email-gateways in between the BITNET, and the rest of the world.
Email is transported inside the BITNET with five methods:
@table @strong
@item defrt1
Target system has no mailer, all email must be sent to the users directly.
Normally a mail-file is sent in PUNCH-80 format.

@item bsmtp3
Target system has a mailer, which can receive Batch-SMTP-file containing
the email.  Format of the file is PUNCH-80.

@item bsmtp3rfc
Like @strong{bsmtp3}, but addresses in RFC-822 format instead of
BITNET node style.

@item bsmtp3nd
Target system has a mailer, which can receive Batch-SMTP-file containing
the email.  Format of the file can be PUNCH-80, or NETDATA.
In case the email contains longer lines, than 80 characters, this is
recommended method, if receiver system can accept it.

@item bsmtp3ndrfc
Like @strong{bsmtp3nd}, but addresses in RFC-822 format instead of
BITNET node style.
@end table

To familiarize him-/herself with NJE-networking,  the reader should
also have access to the IBM  manual
`@emph{Network Job Entry Formats and Protocols for System/370 Program
Products}', @strong{GG22-9373-02} or @strong{SC23-0070-01} (or later).

@c ================================================================
@node BITNET-II, Installation, Introduction, Top
@chapter Structure of Second Generation BITNET on virtual links


BITNET was a traditional wires+modems network, until the NSF-net began
to grow in late 1980's.

When it became evident, that having separate wires for BITNET, and for
other networking activities (TCP/IP, NSF-net) is asking to keep old
separate lines in existance with new high-speed Internet, multiple
solutions were sought after for replacing physical wires with virtual
ones.

In 1986-1989 Ira Fuchs, Peter Olenick et.al. (Princeton University)
developed so called BITNET-II protocol, with which it became possible
to replace dedicated physical wires with virtual TCP/IP connections
over the Internet.

The protocol, and its reference implementation are described on
@file{BRFC0002 TEXT} available at least from  ftp.funet.fi:
@example
  ftp://ftp.funet.fi/pub/doc/netinfo/CREN/brfc0002.text
@end example
via anonymous FTP.

Soon after, (in 1990), Lee Varian, Peter Olenick, and Michael Gettes
(all from Princeton University) made a BITNET restructuring proposal
in which they divided fine-grained threaded mesh of the former BITNET
into 9 regions.

From their  @file{bit2plan.proposal}:
@quotation
@flushright
February 1, 1990
Revised September 21, 1990
@end flushright
@sp 1
@center Proposal to Restructure the  Network Supporting BITNET

 Historically the  physical and  logical networks  supporting
 BITNET have   been structured on leased  communications lines
 connecting member sites.   New sites  connect to the network
 based  upon  an  existing BITNET  member's  willingness  and
 ability to support the request for connection.   The cost of
 the leased communications line, which is paid for by the new
 BITNET  site,  is  also a  consideration  in choosing  which
 existing BITNET sites are  possible connection points.   The
 cost of  the communications line  may be different  based on
 factors of distance and tariff considerations.   This method
 of connecting new sites has worked quite well,  but the lack
 of  a   formal  structure  for   the  network   is  creating
 limitations  which  impair  the  efficiency  and  growth  of
 BITNET.  The lack of a structure increases the complexity of
 routing table  generation and  hampers efforts  to implement
 network  management tools.    The ability  to create  BITNET
 links  by  using  the  national  and  regional  IP  networks
 increases the confusion over where BITNET connections should
 be made.
 
 The idea of reorganizing BITNET into a more formal structure
 has been discussed for a number  of years,  but the costs of
 the additional leased lines needed  to implement such a plan
 made the project economically infeasible.   Now,  BITNET has
 the ability to use the national and regional IP networks;  a
 plan to restructure BITNET by  using the IP networks appears
 to be  cost effective and  implementable.   This  plan could
 also be implemented using private  communications facilities
 and IP routers instead  of using  the national  and regional
 networks.   The costs of  an  implementation using a private
 IP network would need to be carefully considered.
@sp 1
@center Restructuring Proposal
 
 The   restructuring    is   based   on   the    concept   of
 'regionalization',   the  separation  of  the  network  into
 geographic areas  or regions.   Each  region would  have two
 'core'  sites.    Each  core site  would  have  RSCS-over-IP
 connections to every other core site.   The core sites would
 form a 'backbone'.    The national and regional  IP networks
 are the physical  facilities that would be used  by the core
 sites  to   form  the   BITNET  backbone.     By  generating
 appropriate BITNET routing tables,  the  number of nodes and
 the amount of traffic handled by  the core sites for a given
 region can be  statically balanced.   Within a  region,  the
 core sites could  connect to a number  of 'mid-level' sites,
 again by use  of RSCS-over-IP.   This type  of structure has
 the ability to provide an alternate  path into a region if a
 core site were  out of service.   The member  sites or 'end'
 sites within a region could  connect to the mid-level sites.
 Traditional leased line  connections may  exist at any level
 within the  structure but  these types  of connections  will
 continue to  have limitations.    That is,   if a  host with
 traditional leased lines is down, no other path may exist to
 the sites supported by that leased line.
@sp 0
@dots{}
@sp 1
@center Benefits of the restructuring
 
 The purpose of the regionalization  is to impose a structure
 on the  logical network supporting BITNET.    This structure
 will  reduce  the  burden  on   the  current  hub  sites  by
 decreasing  the number  of files  which  must transit  these
 sites.  Overall network service will be improved because the
 number of 'hops'  a file must take to  reach its destination
 will be reduced or be no  greater than in the current BITNET
 topology.   The impact  on BITNET when a key  BITNET node or
 Internet connectivity fails  will be reduced because  of the
 increased number of connections between core sites.   As the
 intra-regional mid-level structure develops,  the ability of
 the  core  sites to dynamically  reroute  traffic  around  a
 disabled  core node  will provide  improved network  access.
 The three level (core, mid-level, end site) structure of the
 region  can be  expanded to  include  additional levels  and
 paths as  needed within the  region,  thereby  providing for
 dynamic rerouting within a region as well.
@end quotation

@c ================================================================
@node Installation, Configuration, BITNET-II, Top
@chapter Installation of FUNET-NJE package

Installation of this package is fairly straight-forward by
editing proper parts from to be activated from the @file{Makefile},
and editing sample configuration files for your local machine.

This system has two hard-wired files, which can be overridden
by editing @file{site_consts.h}-file, or defined in @file{Makefile}:
@table @strong
@item /etc/funetnje.cf
Main configuration file telling where rest of the system resides
@item /etc/funetnje.pid
The "pid file" giving easy pointer to the running emulator program
@end table

Via @file{/etc/funetnje.cf}, and its extensions
(@file{msg-exit.cf}, @file{file-exit.cf})
it is possible to define outgoing, and incoming spool directories, et.al.
Only notable detail is that all those directories @strong{must}
reside on same disk partition, as files are moved from within
emulator spool to user spool with @emph{link(2)}-, and
@emph{unlink(2)}-system calls. (Or  @emph{rename(2)}, if it exists.)


For inter-module access control this software uses
one UNIX group, which is used to @emph{set-gid} some programs,
and which is needed for system administrators so that they
can run the control program without a need to become super-users,
and also without a need to set-gid the control-panel (@file{ucp}).


Things to decide:
@itemize @bullet
@item
Name and number of the `bitnet' programs group (into your @file{/etc/group}).
The @file{Makefile} has variable @var{NJEGRP} for it.

@item
Directories for your BITNET transmission spool, and user spool.
@emph{Important: those must be on same partition!}

Example configurations have them as:
@itemize
@item @emph{QUEUEDIR}
@file{/var/spool/bitnet}

@item @emph{USERSPOOL}
@file{/var/spool/bitspool}
@end itemize

@item
Location of various programs, @file{Makefile} has variables:
@var{MANDIR}, @var{LIBDIR}, @var{BINDIR}, and @var{ETCDIR}.
@end itemize

Choosing proper defines for the compile of the software, especially
your platform dependent pre-processor constants:
@table @strong
@item -D_POSIX_SOURCE
Use on true @code{POSIX.1} systems

@item -DHAS_PUTENV
If your system has SysV-like  @code{putenv(3)} to set environment variable.
When this is not defined, a BSD-like @code{setenv(3)} is assumed

@item -DHAS_LSTAT
Use when your system has  @code{lstat(2)} in addition to the usual @code{stat(2)}

@item -DUSE_SOCKOPT
Use when your system has a working  @code{setsockopt(2)} for @code{SO_RCVBUF},
and @code{SO_SNDBUF}.
Most systems do!
This is a performace option, which @strong{might} help on busy systems

@item -DNBCONNECT
The system can do a @code{connect(2)} on a socket, which has been flagged
as non-blocking.  Without this connection to a system which is unreachable
would cause system to hang for synchronous timeout...

@item -DNBSTREAM
This compilation option allows for more complete non-blocking operations,
including actual @code{write(2)}s.
@strong{This is recommended mode of operation!}

@item -DNO_GETTIMEOFDAY
System internal time retrieval for various delay/speed-analysis uses
uses (per default) BSD-derived(?)  @code{gettimeofday(2)}-call, but
if your system does not have it, define this, and  @code{time(2)}
will be used.

@item -DMAILERNAME="..."
To override the default @var{MAILERNAME} (= "MAILER")
in the @file{bmail.c} source.

@item -DCONFIG_FILE=...
Override @file{consts.h} definition of system config file name

@item -DPID_FILE=...
Override @file{consts.h} definition of emulator process-id file name

@item @emph{A set of IPC methods, choose one!}

@item -DCOMMAND_MAILBOX_FIFO
Uses named pipe (@code{mkfifo(2)}) for an emulator vs. user programs
IPC channel.
Fifo path is taken from @file{/etc/funetnje.cf} @var{CMDMAILBOX} entry


@item (-DFIFO_0_READ)
A subitem on @strong{-DCOMMAND_MAILBOX_FIFO} -- Do a @code{close(2)}/@code{open(2)}
pair on a FIFO, when read from it returns 0.  Was needed for Linux 0.99pl13.

@item -DCOMMAND_MAILBOX_SOCKET
Uses @code{AF_UNIX}, @code{SOCK_STREAM} for the IPC channel.
Socket path is taken from @file{/etc/funetnje.cf} @var{CMDMAILBOX} entry

@item -DCOMMAND_MAILBOX_UDP
Uses @code{AF_INET}, @code{SOCK_DGRAM} for the IPC channel.
Socket IP address is taken from @file{/etc/funetnje.cf} @var{CMDMAILBOX}
entry (numeric), and it naturally must be on same machine, as the emulator
runs.  This is only choise for cluster nodes

@item -DBSD_SIGCHLDS
Uses @code{BSD}-style @code{SIGCLD/SIGCHLD} handling where an signal
trapper program is required to process the exited child information.
On @code{SysV}-systems it is safe (it seems) to just ignore @code{SIGCLD}
and that way to get rid of the zombied child.

@item @emph{Some compiler switches:}
@item -fno-builtin
A GNU-CC option telling it, that builtin accelerated operations should not
be used. (Alignment may cause problems on some cases!) However we have
a fairly high confidence that we have eliminated every case, where inlining
a @code{memcpy(3)} could cause problems.  Try without it!
(Possible problems manifest themselves as coredrops due to SIGBUS, as
 some memory accesses may become unaligned..)

@item @emph{Some more esoteric tricks:}
@item -DDEBUG
If you really want to see what goes on...  The amount of DEBUG-code
is decreasing all the time, and frankly we haven't compiled the code
with `-DDEBUG' for last 100 or so versions, we have just added
`logger()'-calls into it at places needing it, and adjusted @var{LOGLEVEL}
to get them into @var{LOGFILE}, and then commented or removed those loggers
out again..

@item -DUSE_ZMAILER -I/usr/local/include
For @file{mailify.c} -- for tight integration with Zmailer MTA.
(Expects Zmailer's  @file{libzmailer.a}, and @file{zmailer.h} to be
available.  See Zmailer's @code{zmailer(3)} for further information)

@item -DUSE_OWN_PROTOS
If the "prototypes.h" -function prototypes for various things can
for your system, and your system does not have ANSI-headers of its
own, then use this
@end table

Several pre-tested compiler setups are also available in the @file{Makefile}.

@c ================================================================
@node Configuration, User programs, Installation, Top
@chapter Configuration of FUNET-NJE package

@menu
* funetnje.cf::         Runtime main configuration file
* file-exit.cf::        Incoming files processing configuration file
* msg-exit.cf::         Incoming NMR's processing configuration file
* cmd-help.txt::        Text file that is sent as a responce to NMR 'HELP'
* routes::              BITNET node routing databases
@end menu

@c ----------------------------------------------------------------
@node funetnje.cf, file-exit.cf, , Configuration
@section Runtime main configuration-file: @file{funetnje.cf}

Here is a sample @file{/etc/funetnje.cf} -file:

@example
*
*       Configuration file for FUNET-NJE program
*

NAME            FINFILES                      @i{Node primary name}
*ALIAS          HAMSTER                       @i{Optional alias for this node}
*IPADDRESS      nic.funet.fi                  @i{Node primary IP address}
IPADDRESS       128.214.6.100                 @i{used on link signon msgs}
QUEUE           /usr/spool/bitnet             @i{Emulator spool directory}
CMDMAILBOX      /usr/spool/bitnet/.cmdpipe    @i{IPC Socket/Fifo/IP-address}
LOG             /usr/adm/bitnet.log           @i{Emulator log-file}
LLEVEL          1                             @i{Logging level.  Smaller==less}
RSCSACCT        /usr/adm/bitnet.acct          @i{Define for IBM RSCS-like account file}
* this table is a binary file.
TABLE           /usr/local/lib/nje/finfiles.routes   @i{NJE route database}
* @i{EBCDIC<->ASCII tables, (optional), see ebcdictbl(5)}
*EBCDICTBL      /usr/local/lib/nje/ebcdictbl
INFORM          MEA@@FINFILES ROOT@@FINFILES    @i{1+ users monitoring line states}
FILEEXITS       /usr/local/lib/nje/file-exit.cf      @i{Defines  FILE-EXITs}
MSGEXITS        /usr/local/lib/nje/msg-exit.cf       @i{Defines  MSG-EXITs}
DEFFORM         STANDARD                      @i{NJE 'FORM' default}
*DEFAULT-ROUTE  FINHUTC                       @i{Optional default route.  With this}
                                              @i{the @var{TABLE}-file can be left empty}
                                              @i{but it @strong{must} exist!}

*  @i{Example link to a VAX/VMS system with UCX, and JNET}
LINE 0 FIGBOX                           @i{Key-line starting link definition}
 TYPE           UNIX_TCP                @i{Only supported value is  UNIX_TCP}
 BUFSIZE        4096                    @i{VMNET TTB size}
 TIMEOUT        3                       @i{3 is a good value..}
 IPPORT         500                     @i{Opposite end's VMNET's IP port}
* TCPNAME       figbox.funet.fi         @i{Opposite end's name - as a comment}
 TCPNAME        128.214.6.7             @i{ .. or IP number ..}
 TCP-SIZE       8192                    @i{How much to block before write}
 MAX-STREAMS    7                       @i{Number of parellel streams}

*  @i{Example link to a Linux-PC running FUNET-NJE}
LINE 1 ALIJKU65
 TYPE           UNIX_TCP
 BUFSIZE        1024
 TIMEOUT        3
 IPPORT         175             @i{The official VMNET port}
*TCPNAME        alijku65.edvz.uni-linz.ac.at
 TCPNAME        140.78.4.34
 TCP-SIZE       3100            @i{Lower than usual - network problems}
 MAX-STREAMS    7

*  @i{Example link to an IBM VM/SP RSCSv3 system with VMNET}
LINE 2 FINHUTC
 TYPE           UNIX_TCP
 BUFSIZE        1024
 TIMEOUT        3
 IPPORT         175
*TCPNAME        finhutc.hut.fi
 TCPNAME        130.233.224.4
 TCP-SIZE       8192
 MAX-STREAMS    7

*  @i{Example link to a Convex C3480 running FUNET-NJE}
LINE 3  FINFAN
 TYPE           UNIX_TCP
 BUFSIZE        8192
 TIMEOUT        3
 IPPORT         175
 TCPNAME        convex.csc.fi
 TCP-SIZE       8192
 MAX-STREAMS    7
@end example

@c ----------------------------------------------------------------
@node file-exit.cf, msg-exit.cf, funetnje.cf, Configuration
@section Incoming files processing configuration file: @file{file-exit.cf}

This is @emph{wide} format configuration file (like they all), and
can be used to define what is done with various incoming files.

Over-wide lines are wrapped @emph{in this document} with @code{\}-characters
at the end of previous line, and at the begin of next line.
Such wrapping is not allowed in the real configuration file!


@example
#  FILE-EXIT.CF -- Configure file handling on FUNET-NJE
#
# Rule for spool disposition path, defines also system default
# user spool directory for those who don't have specially set
# something else.
# Possibilities:    ~/bitspool  -- real user's only! (~/ == users home)
#                   /usr/path/path/ -- append 'touser' as subdir,
#                                       it is spool dir.
#                                      Valid for Exits and real users.
#                   /usr/path/path  -- explicite directory.
#                   /dev/null       -- explicite file (special case).
#       When directory isn't present, it is built from upmost present
#       level down to users bottommost level hmm...
#   Question about ownership of directory/files...
#     Real users:  real protections, programs start with setuid() user.
#         Exit users:  POSTMAST  (exits start as root anyway.)
#         Exited reals: real protections, programs start with setuid()
#                       user.

Spool-Dir:      /usr/spool/bitspool/
Postmast-Dir:   /usr/spool/bitspool/POSTMAST

# Now list of things to match and then what to do
# To do keywords:       DISCARD to /dev/null.
#                       KEEP    just so.  Into default or given spool.
#                       NOTIFY  send an NJE message to someone.
#                       RUN     starts arbitary program with arbitary
#                               arguments telling about file location
#                               and its properties.
#                               If fails, well..

# Defining  SpoolDir  which shall not be attached ToUser must not be done
# with trailing "/", (and vice versa)
#       /usr/spool/bitnet/SYSIN-JOB

# Exit table begin keyword:
Exit-Table:

# Args:
# touser8 tonode8 fname8   ftype8  pun? class fruser8  frnode8  dist8    \
\ SpoolDir                        action  ExtraArguments
#
# Several special handlings (tests/illustriations)
#
#MEA     FINFILES *        *        *   *     HKS      SEARN    *        \
\ default                         DISCARD
NOBODY   FINFILES *        *        *   *     *        *        *        \
\ default                         DISCARD
FOOBAT   FINFILES *        *        *   *     *        *        *        \
\ default                         RUN   /usr/local/lib/nje/transfer \
\ MEA@@$TONODE $SPOOL
ECHO     FINFILES *        *        *   *     *        *        *        \
\ default                         RUN   /usr/local/lib/nje/transfer \
\ NOBODY@@$FRNODE $SPOOL
#LPR     FINFILES *        *        *   *     *        *        *        \
\ default                         RUN   /usr/local/lib/nje/rprint   \
\ $SPOOL lpr $FRUSER $FRNODE $TOUSER $FID
#
#  Define what we do with the SYSINs
#
*        *        *        *        SYSIN *   *        *        *        \
\ /usr/spool/bitnet/SYSIN-JOB/    RUN   /usr/local/lib/nje/sysin $SPOOL
#
#  Define a MAILER for our node!
#  And also handle directly sent mail from nodes without mailers...
#
MAILER   FINFILES *        *        PUN M   *        *        *          \
\ default                         RUN   /usr/local/lib/nje/mailify $SPOOL
*        *        *        MAIL     PUN M   *        *        *          \
\ default                         RUN   /usr/local/lib/nje/mailify $SPOOL
#
# NETINIT is pseudo id for automatically re-generating routing tables
# This is done with  netinit.sh  which must be configured by hand.
#
NETINIT  FINFILES *        *        *   *   *        *        *          \
\ default                         KEEP
#
# Finally  a `catch them all' default handling case.
#
*        *        *        *        *   *   *        *        *          \
\ default                         KEEP
@end example

@c ----------------------------------------------------------------
@node msg-exit.cf, cmd-help.txt, file-exit.cf, Configuration
@section Incoming NMR's processing configuration file: @file{msg-exit.cf}

This is @emph{wide} format configuration file (like they all), and
can be used to define what is done with various incoming NMR's.

Over-wide lines are wrapped @emph{in this document} with @code{\}-characters
at the end of previous line, and at the begin of next line.
Such wrapping is not allowed in the real configuration file!


@example
#
# MSG-EXIT.CF  for FUNET-NJE
#
# Some first ideas about patterns, et.al.:
#
# Actions:  CMD: BUILTIN, RUN
#           MSG: BRCAST, DISCARD, RUN, (PIPE)
# Argument tokens for 'RUN': TOUSER, TONODE, FRUSER, FRNODE, TEXT, ARGS
# Arguments for 'BUILTIN': "HELP", "HARDCODED", "ERROR"/ERROR msg-string",
#                          "ALIAS remap-pattern"
# Patterns work as follows:
#      TO*KEN -- match (only) any of input words: TO TOK TOKE TOKEN
#      *      -- match any token, (but not blank)
#      **     -- match any number of tokens, including nothing.
# Remap-patterns work as follows:
#      $nn    -- substitute nn:th text token into this place
#      ANYSTRING -- copy it verbatim
#

CmdHelpFile: /usr/local/lib/nje/cmd-help.txt

#$TOUSER $TONODE  $FRUSER  $FRNODE  C pattern    ACTION args
# Actually TOUSER is not tested on commands, it will be blank anyway..
.        FINFILES *        *        C "HIL*FE"   \
\ BUILTIN HELP English quick document is available via 'HELP' command.
.        FINFILES *        *        C "SOS"      \
\ BUILTIN HELP English quick document is available via 'HELP' command.
.        FINFILES *        *        C "H*ELP"    BUILTIN HELP
.        FINFILES *        *        C "M * * * **"       BUILTIN HARDCODED
.        FINFILES *        *        C "MSG * * * **"     BUILTIN HARDCODED
.        FINFILES *        *        C "M*SG **"          \
\ BUILTIN ERROR Too few arguments for the MSG
.        FINFILES *        *        C "CMD * * **"       BUILTIN HARDCODED
.        FINFILES *        *        C "CMD **"           \
\ BUILTIN ERROR Too few arguments for the CMD
.        FINFILES *        *        C "Q*UERY SYS*TEM S" BUILTIN HARDCODED
.        FINFILES *        *        C "Q*UERY SYS*TEM"   BUILTIN HARDCODED
.        FINFILES *        *        C "Q*UERY STAT*S"    BUILTIN HARDCODED
.        FINFILES *        *        C "Q*UERY *"         BUILTIN HARDCODED
.        FINFILES *        *        C "Q*UERY * Q*UEUE"  BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY N*AMES"  BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY U*SER *" BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY U*SER"   BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY LOG"     BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY CPU"     BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY CP*LEVEL" BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY IND*ICATE" BUILTIN HARDCODED
.        FINFILES *        *        C "CPQ*UERY T*IME"   BUILTIN HARDCODED
.        FINFILES *        *        C "CP*QUERY T*IME"   BUILTIN ALIAS CPQ $2
.        FINFILES *        *        C "EC*HO **"     \
\ RUN /usr/local/bin/send -s $FRUSER@@$FRNODE *CMD was '$TEXT'
# The default
.        FINFILES *        *        C "**"       BUILTIN ERROR

#$TOUSER $TONODE  $FRUSER  $FRNODE  M ACTION args
#                "." == FRUSER is blank, that is from some 'SYSTEM'..
VMNET    *        *        *        M  DISCARD
MAILER   *        *        *        M  DISCARD
MAILSERV FINFILES .        *        M  DISCARD
MAILSERV FINFILES *        *        M  \
\ RUN  /usr/local/lib/mailserver/nje-msg $FRUSER $FRNODE $TEXT
LISTSERV FINFILES .        *        M  DISCARD
LISTSERV FINFILES *        *        M  \
\ RUN  /usr/local/bin/send -u MAILSERV $FRNODE@@$FRUSER \
\ *Sorry, we do not have LISTSERV, try MAILSERV
ECHO     FINFILES .        *        M  DISCARD
ECHO     FINFILES *        *        M  \
\ RUN  /usr/local/bin/send -u echo $FRUSER@@$FRNODE *Got Message: '$TEXT'
FINGER   FINFILES .        *        M  DISCARD
FINGER   FINFILES *        *        M  \
\ RUN /usr/local/lib/nje/nje-finger $FRUSER $FRNODE $ARGS

# The default
*        *        *        *        M  BRCAST
@end example

@c ----------------------------------------------------------------
@page
@node cmd-help.txt, routes, msg-exit.cf, Configuration
@section Node NJE command 'HELP' response file: @file{cmd-help.txt}

Node help text can be changed with the @file{cmd-help.txt}-file,
which paths is defined in the @file{msg-exit.cf}.

@example
@cartouche
Commands available for FUNET-NJE emulator:
   Help / HILfe / SOS - Three commands to ask for this info 
   Query SYStem       - Show line status summary report, and activity
   Query SYStem S     - Same as "Q SYS", but with extra activity data
   Query STATs        - Show line statictics
   Query Nodename     - Show the route entry to that node
   Query linkname A/F - Available via Query SYStem
   Query linkname Q   - Show 30 first files in queue on the link
   CPQuery Names      - List all users logged on
   CPQuery User       - Tell how many users are logged on
   CPQuery User username - Look for a specific username
   CPQuery LOG        - Send the WELCOME message
   CPQuery CPU, CPLEVEL, IND, T - Machine type and time
   MSG node user text.. - A way to send a NMR to some node via msg relay.
   CMD node text..      - A way to send a NMR to some node via cmd relay.
This system has also NMR responding servers at addresses:
   ECHO@@FINFILES       -- throws the NMRs back to you.
   MAILSERV@@FINFILES   -- ask for HELP
   FINGER@@FINFILES     -- Usage: tell finger@@finfiles address@@internet
@end cartouche
@end example

@c ----------------------------------------------------------------
@page
@node routes,   , cmd-help.txt, Configuration
@section BITNET node routing databases: @file{nje.routes}

@emph{FUNET-NJE} uses @emph{IBM RSCSv1} compatible route
files.  Used data-elements are @var{SITE}, and @var{LINE}
defnitions (second, and third tokens).

The routing file (for example @file{finfiles.routes}) is
generated with @file{njeroutes} -command:
@example
@cartouche
# njeroutes  finfiles.header finfiles.netinit finfiles.routes
@end cartouche
@end example

@example
   @r{ Local override file:} @file{finfiles.header}
@cartouche
*
*  Header of FUNET-NJE/HUyNJE package routing table
*
*  Generic format:
*       SITE    LINE    FORMAT
*
 ROUTE FINFILES LOCAL ASCII
 ROUTE HAMSTER  LOCAL ASCII
 ROUTE FINFTP   LOCAL ASCII
 ROUTE FINUTU   FINUTU EBCDIC
 ROUTE ALIJKU65 ALIJKU65 EBCDIC
 ROUTE ALIJKU64 ALIJKU65 EBCDIC
 ROUTE FINHUTC  FINHUTC EBCDIC
@end cartouche
@end example

@page
@example
   @r{ Node route file:} @file{finfiles.netinit}
@cartouche
*
* GR version 91-11-19, date=94-02-04, time=00:01:34 
*
* Table generation parameters: GENROUTE FINFILES NAME=FINFILES.NETI
*                            : NIT.A 
*
* Routing table information for 
* node: FINFILES nodenum: 0963
*
*
* The routing table neither contains neighbours nor the local node.
* List of neighbours:
*
* node: FIGBOX   nodenum: 3025
*       FINHUTC           1044
*
* * The position of the network name and country *
* * assumes that the RSCS ROUTE command does not *
* * have more than two parameters.               *
ROUTE VERS9402 FINHUTC  EARN     NL
ROUTE FINABO   FIGBOX   EARN     FI
ROUTE FINHUT   FINHUTC  EARN     FI
ROUTE FINHUTA  FINHUTC  EARN     FI
ROUTE FINJYU   FIGBOX   EARN     FI
ROUTE FINKUO   FIGBOX   EARN     FI
ROUTE FINNPHI  FIGBOX   EARN     FI
ROUTE FINOU    FIGBOX   EARN     FI
ROUTE FINUH    FIGBOX   EARN     FI
ROUTE FINUHA   FIGBOX   EARN     FI
@dots{}
@end cartouche
@end example

@c ================================================================
@node User programs, System programs, Configuration, Top
@chapter Documentation of user programs for NJE use

A set of program for user to handle the files, and
communication with the BITNET.

@menu
* qrdr::        Query user's reader
* receive::     Receive files from spool
* sendfile::    File sending with sendfile/print/punch/submit
* send::        Interactive messages/commands sender
* bitsend::     Send file in multiple parts with `table of contents'
* bitreceive::  Receive `bitsent' files
* ygone::       Register to the running Emulator to be `gone'
* iucvtrap::    Trap, and pipe out incoming NMR-messages from the Emulator
* transfer::    Resend files from user's spool
@end menu


@c ----------------------------------------------------------------
@node qrdr, receive,   , User programs
@section Query user's reader: @file{qrdr}
@c      - qrdr


Shows a listing of user's BITNET spool -- the `reader'.
@example
@cartouche
QRDR: Spool dir: `/usr/spool/bitspool/MEA/'
From:             To:               FName:   FExt:    Type  Form:    SpoolID
NETSERV@@FINHUT    MEA@@FINFILES      NETSERV  NOTE     PUN A QUFINHUT 0001
NETSERV@@FINHUT    MEA@@FINFILES      NETSERV  NOTE     PUN A QUFINHUT 0002
NETSERV@@FINHUT    MEA@@FINFILES      NETSERV  NOTE     PUN A QUFINHUT 0003
@dots{}
@end cartouche
@end example

Synopsis:
@code{  qrdr [ -u @i{user} ] [ -n ] [ -l ] [ @i{dirpaths}, @i{filepaths} ]}

@emph{qrdr} gives users a tool to see what they have in their
incoming BITNET spool queues.
Depending on file protections, it also lets others to see what user
XYZ has in his/her queue, or ANY directory possibly containing
BITNET spool files, or just any specified files...

When `-u @i{user}' has not been used, the default behaviour of @emph{qrdr}
is to find out who user is making query by asking `whoami', eg. getting
value of @var{LOGNAME} environment variable.
(On SysV this is @var{USER} variable.)
If the environment doesn't give answer, @file{/etc/passwd}-file is
consulted with effective uid.

Spool directory is determined using same mechanism as in @emph{funetnje}
main program.
Sample configuration of file procesing exits gives  directory
@file{/usr/spool/bitspool}  in  which UPPERCASE userids
(or pseudoids - MAILER is one such) are  `user' specific
subdirectories containing individual spooled files.

Option `-l' gives file paths to spool files, and some heuristics of file
contents  for @emph{rdrlist(1)} program.
For program-parsing usage, the `-l' option uses singular tabulator to
separate output fields.

For the `-l' option, recognized data types are: NETDATA,
VMSDUMP, CARD (Cornell CARD), and PUNCH when none of the
previous ones match to input punch records.
For three PRINT formats there  are: PRINT (Machine Carriage Control),
PASA (ANSI Carriage Control), and PCPDS for rare 3rd alternative.


@c ----------------------------------------------------------------
@node receive, sendfile, qrdr, User programs
@section Receive files from spool: @file{receive}
@c      - receive

With @file{receive(1)} user can receive files from the spool with
a @emph{spoolid} (a decimal number in range of 1 to 9900), a filename
in the user's spool directory, or (relative) path to a file.

@example
@r{Synopsis:}
    receive [-n] [-a|-b] [-asa|-rawpun|-bitraw|-bcat]  [-d]
            [-u @i{username} ] [-o @i{outpath} ] @{@i{filepath}|@i{spoolid}@}
@end example

Options are as follows:
@table @strong
@item -n
Don't delete the spool file, when processing completes successfully.

@item -a
Do forced translate of the file contents from EBCDIC to ASCII.
(Despite of the spool file's class.)

@item -b
Consider the file to be binary, only receive it, don't translate.

@item -d
Debug outputs of the NJE spool file

@item -asa
Affects PRINT files which contain MCC carriage control codes
(see bitspool(5)) by translating them to FORTRAN-style carriage
control.

@item -rawpun
Outputs the file contents (usually 80 chars per record PUNCH records)
as is without adding anything, nor translating it.

@item -bitraw
????

@item -bcat
Output non-translated records in format:
@example
<char of reclen> <chars of record>
@end example

@item -u @i{username}
Look into @i{username}'s spool directory, instead of who invokes
the command.

@item -o @i{outpath}
Define output file, `-' denotes @var{STDOUT}.
@end table

Input files can be designated either with numeric spoolid, or
path(s) to the file(s).

@c ----------------------------------------------------------------
@node sendfile, send, receive, User programs
@section File sending with @file{sendfile/print/punch/submit}
@c      - sendfile/print/punch/submit

This program quintet is actually one binary with four invocation names.
It behaves slightly differently with each name:
@table @strong
@item sendfile/sf
Generates @emph{SYSOUT} @emph{PUNCH} file containing @emph{NETDATA}
encoded contents of the original file, and submits it to transmission
to a remote system.
@item print/bitprt
Generates @emph{SYSOUT} @emph{PRINT} file, and submits it to
transmission to a remote system.
@item punch
Generates @emph{SYSOUT} @emph{PUNCH} file, and submits it to
transmission to a remote system.
@item submit
Generates @emph{SYSIN} batch job spool file, and submits it to
transmission to a remote batch facility for further processing.
@end table

Actual usage details are on the man-page.
@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node send, bitsend, sendfile, User programs
@section Interactive messages/commands sender: @file{send}
@c      - send

@example
@r{Synopsis:}
   send [-u fromuser|-s] [-c(ommand)] [@@]node [command string]
   send [-u fromuser|-s] [-m(essage)] user@@node [message text]
@end example

The first format sends commands to designated node.
Using the option `-c' (or `-command') is unnecessary, unless the
target node is wanted to be without the `@@'-sign on the command line.

The second format sends interactive (@emph{NMR}) messages to
the recipient  @emph{user@@node}.
The maximum size of the NMR message is 120 characters per line.
Unlike for the files, the network will not guarantee, that the NMR
message will reach the destination, however it does a ``best effort''
attempt at it.

When no command string, or message text is included at the command line,
the input is taken from the @emph{STDIN}.
If the input comes from somewhere else than TTY, prompting is suppressed,
like for example when some command output is piped to @file{send}.

When @emph{STDIN} is used, message entering ends with the end-of-file
condition (Often created with `ctrl-D' from the keyboard.)

@c ----------------------------------------------------------------
@node bitsend, bitreceive, send, User programs
@section Send file in multiple parts along with `table of contents': @file{bitsend}
@c      - bitsend

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node bitreceive, ygone, bitsend, User programs
@section Receive `bitsent' files: @file{bitreceive}
@c      - bitreceive

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node ygone, iucvtrap, bitreceive, User programs
@section Register to the running Emulator to be `gone': @file{ygone}
@c      - ygone

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node iucvtrap, transfer, ygone, User programs
@section Trap, and pipe out incoming NMR messages from the Emulator: @file{iucvtrap}
@c      - iucvtrap

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node transfer,   , iucvtrap, User programs
@section Resend files from user's spool: @file{transfer}
@c      - transfer

A special utility for transfering spoolfiles to another destination.
@example
@r{Usage:}
.../transfer [-t] NEW@@DESTIN /absolute/path/to/spool/file
@end example
This is @emph{very special} program for very rare and special use.
Super-user can use @strong{-t}-option to order ``transparent'' transfer,
that is, transfer where original originator is @strong{not} altered.

For normal users the originator address (FRM:) will get rewritten
as the caller of this utility.

@c ================================================================
@node System programs, Interfacing mailer, User programs, Top
@chapter Documentation of system programs for NJE use

These programs are used by system manager, or they are utilities
needed for system MTA integration:

@menu
* bmail::       `sendfile' with a twist to send BSMTP email
* mailify::     For receiving incoming email files
* ucp::         Emulator master control panel
* njeroutes::   Route-database generator program
* namesfilter:: Mailer route extractor from BITEARN.NODES file
* acctcat::     Utility to dump the RSCS-accounting records
@end menu


@c ----------------------------------------------------------------
@node bmail, mailify,   , System programs
@section @file{bmail} -- `sendfile' with a twist to send BSMTP email
@c      - bmail

A special form of @file{sendfile} to send email-files to remote
systems -- to a MTA at there, or (gasp), direct to the user, if
no MTA exists at the node@dots{}
@example
  bmail [-u origuser] [-v] [-tag RSCS-TAG]
        [-nd Gateway-Address | -b(smtp) Gateway-address]
        FromUser@@FromNode ToUser@@ToNode [ToUser@@ToNode@dots{}]
@end example

Options are as follows:
@table @strong
@item -u origuser
Defines the originating user, e.g. local @emph{MAILER}@dots{}
Only @emph{super-user} can use this option.

@item -v
Request BSMTP `VERB ON' to be injected.  Sometimes usefull, when
debugging some IBM CMS mailers.

@item -tag RSCS-TAG
Optional RSCS-TAG string to be passed to the transporter, usually
not needed.

@item -nd Gateway-address
Send the email in NETDATA format -- implies also BSMTP.

@item -b(smtp) Gateway-address
Make the file contents to be BSMTP -- that is, Batch SMTP.
Without NETDATA the width limit is 80 chars.
@end table

The message content (along with its RFC-822 headers) is taken
from @emph{STDIN}.


@c ----------------------------------------------------------------
@node mailify, ucp, bmail, System programs
@section @file{mailify} -- incoming mail-file processing
@c      - mailify

Normal use for @file{mailify} is to invoke it from incoming file
processing (see @file{file-exit.cf}):
@example
  .../mailify /var/spool/.../NJEFILE
@end example
There are actually two versions of @file{mailify}, one which
is script (@file{mailify.sh}), and newer C-program,  which
superceeds the old script.

The @file{mailify} analyzes the incoming NJE-spool-file, and
makes the necessary parsing/transformation of it into RFC-822
format, and feeds it to the system mailer.

In the C-program there is support for @file{Zmailer}'s zmailer(3) -library,
which thus can bypass more generic use of @file{/usr/lib/sendmail}
as a feeder to the system mailer.

These incoming mail processors are at their best, when they receive
BSMTP in NETDATA file.

@c ----------------------------------------------------------------
@node ucp, namesfilter, mailify, System programs
@section @file{ucp} -- Emulator master control panel
@c      - ucp

@example
@r{Synopsis:}
  ucp cmd...
    HELP  - Show this message
    SHOW LINE/QUEUE - Show lines or queue status
    START LINE n - Start a closed line
  * START STREAM n LINE m - Start specific stream in active line
    SHUT [ABORT] - Shutdown or abort the whole program
    STOP LINE - Stop a line
    FORCE LINE - Stop a line immediately
  * STOP STREAM n LINE m - Stop a single stream in active line
  * FORCE STREAM n LINE m - Stop immediately
    QUEUE file-name [SIZE size] - To queue a file to send
    RESCAN EXITS - Rescan file and message exits.
    RESCAN QUEUE - Rescan queue and requeue files.
    RESCAN ROUTE - Reopen route database.
    DEBUG DUMP - Dump all lines buffers to logfile
  % DEBUG RESCAN - Rescan queue and requeue files.
    LOGLEVEL n - Set the loglevel to N
    ROUTE xxx TO yyy - Change the routing table.
         Route to  OFF  will delete the route entry.
    GONE username LoginDirectory - Add username to gone list
    UNGONE username - Remove a user from the Gone list.
  * - Not yet implemented,  % - obsoleted
@end example

Program for commanding the FUNET-NJE emulator to do various things.
Successfull usage of this utility @emph{requires} that the user has
supplementary group, which allows communication with the FUNET-NJE.


@c ----------------------------------------------------------------
@node namesfilter, njeroutes, ucp, System programs
@section @file{namesfilter} -- Mailer route extractor from @file{BITEARN.NODES} file
@c      - namesfilter

@example
@r{Synopsis:}
  namesfilter:  Filters @file{BITEARN.NODES} data to mailer databases
     namesfilter -zmailer  < bitearn.nodes > bitnet.routes
     namesfilter -sendmail < bitearn.nodes > bitnet.routes
     namesfilter -smail    < bitearn.nodes > bitnet.routes
@end example

This utility parses @file{BITEARN.NODES}-file, and from it
all the `@emph{:type.NJE}' entries are checked up for possible
`@emph{:servers*.}'-entries, which define mailers.
Of those, only one can exist without ``OUTONLY''-tag, and
it defines the receiving mailer.

If no non-``OUTONLY'' `@emph{:servers*.}' are defined,
default mode of ``@strong{defrt1}'' is used, othervice
some smarter version is chosen.

Originally this utility was written for ZMailer usage, other
modes are ``afterthoughts''@dots{}

@c ----------------------------------------------------------------
@node njeroutes, acctcat, namesfilter, System programs
@section @file{njeroutes} -- Route-database generator program
@c      - njeroutes

@example
@r{Synopsis:}
  njeroutes  header.file routes.file output.file
@end example

This program generates local @emph{BINTREE} copy of the route
database from couple input files.  Runtime system uses the
produced file to look up the NJE routes.

@c ----------------------------------------------------------------
@node acctcat,   , njeroutes, System programs
@section @file{acctcat} -- Utility to dump the RSCS-accounting records
@c      - acctcat

@example
@r{Synopsis:}
  acctcat /path/to/rscs/acct-file
@end example

UNIX-program that is able to dump VM/CMS RSCS-accounting records into
ASCII format. (For various reasons that format was chosen as the
accounting format of the FUNET-NJE as well.)

@c ================================================================
@node Interfacing mailer, Nuts and Bolts, System programs, Top
@chapter Interfacing Zmailer/smail/sendmail with FUNET-NJE

Here are some instructions on how to interface various MTA-systems
into the @emph{FUNET-NJE} package.

There is a separate program (@emph{mailify}) responsible for
incoming email processing, these are routers, and senders of
BITNET email (@emph{bmail}), and there is route database generator
for generating the necessary  @emph{nodename->route}  mapping
(@emph{namesfilter}).

@menu
* Zmailer::     Interfacing Toronto/FUNET Zmailer
* smail::       Interfacing smail3.0
* sendmail::    Interfacing BSD sendmail
@end menu



@c ----------------------------------------------------------------
@node    Zmailer, smail, , Interfacing mailer
@section Interfacing Toronto/FUNET Zmailer with FUNET-NJE

Doing this interface was easy due to vast amounts of easy
to understand configurability of the @emph{Zmailer}@footnote{
Zmailer is a mailer running in two modes with multiple programs
doing their parts; @file{router} (1 to many) doing email address
routing, @file{scheduler} running transport channels, like
@file{smtp}, @file{mailbox} (local delivery), etc.
There are also incoming channel programs @file{smtpserver}, and
compability interface @file{/usr/lib/sendmail}.

The Zmailer is available from
ftp://ftp.funet.fi/pub/unix/mail/zmailer/zmailer-@emph{vers}-@emph{datecode}.tar.gz,
via anonymous FTP.
} system.

Depending on if the system is to be a smart-mailer, or not,
Zmailer's @file{router.cf} shall contain transport preference definition:
@example
# The transport preference order
protocols='routes smtp usenet bitnet'
@end example
which instructs it to include those four protocols into the
route-analysis. The protocols are attempted in given order!
Host BITNET name must be defined in @file{/etc/name.bitnet} for Zmailer's
BITNET-protocol.

If it is desirable @strong{not to} have smart BITNET routing at the
system, it is possible to use lazy-routing of all "*.BITNET"-addresses
into one system somewhere else with following definition in
@file{/etc/mail.conf}:
@example
@dots{}
# rrouter.cf uses these..  Uncomment for use
# : $@{BITNETACTION:="bsmtp3nd mailer@@searn"@}
# : $@{BITNETACTION:="smtp figbox.funet.fi"@}
@end example
the first  one uses special delivery via BITNET facilities to MAILER@@SEARN,
the second uses ordinary SMTP.


BITNET email routing database (@file{$MAILVAR/db/routes.bitnet}) contains
entries like:
@example
academic         bsmtp3!mailer!academic
acmvm            bsmtp3!mailer!acmvm
acusd            defrt1!acusd
@dots{}
searn            bsmtp3nd!mailer!searn
@dots{}
@end example
which essentially mean:
@table @emph
@item academic
Use @strong{bsmtp3}-channel, and send it to @strong{mailer@@academic}
@item acusd
Use @strong{defrt1}-channel, and send mail do @strong{$user@@acusd}
@item searn
Use @strong{bsmtp3nd}-channel (bsmtp3 in NETDATA instead of PUNCH),
and send it to @strong{mailer@@searn}
@end table

That email routing database is built from @file{BITEARN NODES}-file
by use of @file{namesfilter} -program:
@example
# ./namesfilter -zmailer < bitearn.nodes | sort > routes.bitnet
@end example
Unlike the original Toronto @file{namesfilter}-package, this does not
take @file{BITEARN DOMAINS}, or @file{XMAILER NAMES}.

Channels are defined in @file{$MAILSHARE/scheduler.conf}, in which
these five are defined as:
@example
@dots{}
defrt1/*
        command="sm -c $channel defrt1"
bsmtp3/*
        command="sm -c $channel bsmtp3"
bsmtp3rfc/*
        command="sm -c $channel bsmtp3rfc"
bsmtp3nd/*
        command="sm -c $channel bsmtp3nd"
bsmtp3ndrfc/*
        command="sm -c $channel bsmtp3ndrfc"
@end example
Meaning they all are run via `sendmail-compatible transporter'.

Configuration of the `sendmail-compatible transporter' at
@file{$MAILSHARE/sm.conf} contains these five entries for the BITNET:
@example
@dots{}
#
# bitnet stuff F=hu not set
#
bsmtp3      snmSX /usr/local/lib/nje/bmail bmail -b $h -u MAILER $g $u
bsmtp3rfc   snmSX /usr/local/lib/nje/bmail bmail -b $h -u MAILER $g $u
bsmtp3nd    snmSX /usr/local/lib/nje/bmail bmail -nd $h -u MAILER $g $u
bsmtp3ndrfc snmSX /usr/local/lib/nje/bmail bmail -nd $h -u MAILER $g $u
defrt1      snS   /usr/local/lib/nje/bmail bmail $g $u
@end example



@c ----------------------------------------------------------------
@node    smail, sendmail, Zmailer, Interfacing mailer
@section Interfacing Smail3.0 with FUNET-NJE

@emph{This information is from Gerald Hanusch <root@@alijku65.edvz.uni-linz.ac.at>}

@emph{Relevant files are on subdirectory @file{smail-configs/} of the source
directory.}

In order to get SMAIL3.1.28 working with the FUNET-NJE emulator,
following steps are necessary:

@enumerate
@item
Copy the BSMTP3, BSMTP3ND and DEFRT1 BASH scripts to your
@var{SMAIL-LIBDIR}, e.g. @file{/usr/local/lib/smail/}

@item
Edit these 3 Scripts to fit your nje emulator installation,
mainly the path to the @file{bmail} program coming with the emulator
needs to be set properly.

@item
If you don't use a @file{routers} and @file{transports} file till now,
because the compiled in defaults of SMAIL fit your needs,
make a copy of these files to the @var{SMAIL-LIBDIR}.
The source distribution of SMAIL comes with generic samples,
which reflect the compiled-in configuration.

@item
Make an entry to @file{routers} file like below. Commonly between
the entry which defines the @file{path} router and the @file{uucp-neighbors}
router with the following lines

@example
bitnet: driver=pathalias,
        method=bitmethods;
        file=bitearn,
        proto=bsearch,
        -optional,
        -required,
        domain=bitnet,
@end example

The position of the entries within @file{routers} file is significant.
The decision, which path is taken, depends mainly on this position.
Please consult the SMAIL doc, resp. man 5 smail for more info on this.

@item
Make a directory @file{methods} within @var{SMAIL-LIBDIR}.
Copy the supplied @file{bitmethods} file into there.

Edit or generate a bitnet mailers routing table,
and copy it to @var{SMAIL-LIBDIR}.
It can be built from @file{BITEARN NODES}-file
by use of @file{namesfilter}-program:
@example
# ./namesfilter -smail < bitearn.nodes | sort > routes.bitnet
@end example


The table has the common pathalias format with following entries:
@example
   destdomain   transportmethod!mailername!nodename!%s
@end example

where @emph{destdomain} is the domain within bitnet,
@emph{transportmethod} is either bsmtp3, bsmtp3nd or defrt1,
@emph{mailername} is the name of the remote mailer userid which
receives email from bitnet, and
@emph{nodename} is the bitnet node running the mailer

The `@emph{%s} behind the @emph{nodename} is not needed for our purposes,
but is necessary to pass the syntax check. The meaning of the 3 transport
methods is described in @file{bmail(8)} utility.


@item
Edit the @file{transports} file in @var{SMAIL-LIBDIR} by adding
the entries bstmp3, bsmtp3nd, and defrt1.
A sample is provided with the nje emulator package.

@item
Add to your SMAIL - @file{config} file to the @emph{domains=}-line
an entry for @emph{bitnet}, separated by a ":"

@item
If the packages installed properly, reload smail configuration.
Now it should be possible to deliver mail within BITNET using nje.

@end enumerate

@c ----------------------------------------------------------------
@node    sendmail,  ,  smail, Interfacing mailer
@comment node-name,   next,  previous, up
@section Interfacing BSD-sendmail with FUNET-NJE

@emph{This presentation is by the courtesy of mr.
Jukka Ukkonen of CSC.FI, translation to english is
by Matti Aarnio}

@emph{See source-tree subdirectory @file{sendmail-configs/} for
the way how Paul Bryant <P.Bryant@@rl.ac.uk> did this.}


Somewhere in the begin of the @file{sendmail.cf}:
@example
# Bitnet route table (overriding default routes)
Kbitnettable dbm -o /usr/local/lib/mail/bitnettable
@end example

Then in @emph{S0}-rules after error-mailer, local checkup, uucp, and
mailertable:
@example
# Try a separate table for bitnet routes.
R$*<@@$-.$+.bitnet.>$*       $:<$(bitnettable .$3$)>$1<$2.$3>$4
R$*<@@$+.bitnet.>$*          $:<$(bitnettable $2$)>$1<$2>$3
R<$-!$-!$+>$*<$+>$*         $#$1$@@$3$:$2          matched a 3 part route
R<$-!$+>$*<$+>$*            $#$1$@@$2$:$3@@$4       matched a 2 part route
@end example

One completely new ruleset, though a short one, because other domains
need not stripping:
@example
#
#  Strip fake domains off in case posting directly
#  to the receipient's native mail system.
#
S22
R$*<@@$+.bitnet.>       $@@$1<@@$2>
@end example

This @emph{R=22} is only for receiver's side mailer definitions.
(Odd tens are for the sender, even tens are for the receiver.
Smaller ten is for envelope conversion, and bigger is for header
conversion in case headers call for different transformation,
than envelope, a'la S=11/31.)

If sender address need to be translated into node format (from
their propable FQDN format), there appears a need for yet another
mapping of FQDN addresses to BITNET node names, which in practice
means yet another dbm-database for maintance:
@example
  Kbitnetname dbm -o /usr/local/lib/mail/bitnetname
@end example
This needs another set like `22' above which tries to do the
lookup according to the sender's machine, othervice the default
@emph{S=11} is ok.

On the other hand those, who have @emph{S=11/31}, like I have
done it, the `31' is dangerous, because it does a lookup of
`@emph{$(generics $1 $)}', which produces long names a'la
@emph{Jukka.Ukkonen}, which in the usual BITNET-world is not
quite commonly tolerated@dots{}

A new @emph{S=xx} could be for example this S=12 below.
`@strong{$N}' can be made local default name for the BITNET;
@strong{CNfinfiles}:

@example
S12
R$*<@@$=w.>     $:$1<@@$(bitnetname $2$)>
R$*<@@$=w.>     $@@$1<@@$N>
R$*<@@$+.>      $@@$1<@@$(bitnetname $2$)>
@end example

In addition to those there is also a need for all the mailer entries,
like: `@strong{Mbsmtp3}'  (See ZMailer's ``sm''-defines for hints.)

@emph{Non-translated email continues with an apology of being
too tired to finish it ``tonight'', nor is `Jau' certain, that
these make any sense/are corrent -- early September-94.}
@example
        Cheers,
                // jau
------
  /    Jukka A. Ukkonen, M.Sc. (tech.) Centre for Scientific Computing
 /__   Internet: ukkonen@@csc.fi            Tel:  (Home) +358-0-578628
   /   Internet: jau@@cs.tut.fi                   (Work) +358-0-4573208
  v    X.400:    c=fi, admd=fumail, no prmd, org=csc, pn=jukka.ukkonen
@end example



@c ================================================================
@node Nuts and Bolts, Literature, Interfacing mailer, Top
@chapter Technical documents about insides of things

@c  - Technical documentation
@menu
* directories::         Sample directory structure, and related notes
* cmdpipe::             client -> emulator messaging
* iucvpipe::            iucvtrap <-> emulator messaging
* spool format::        Spool file format
* netdata::             Implemented NETDATA features
* emulator logics::     Details of emulator logics
@end menu

@c ----------------------------------------------------------------
@node directories, cmdpipe,   , Nuts and Bolts
@section Directory structure of the FUNET-NJE
@c      - Directory structure

@table @asis
@item @file{/etc/funetnje.cf}
Is the default location of the runtime system master configuration file.
It defines all other files and directories used by this system.

@item @file{/usr/local/funetnje/}
A possible location to place system programs, which are used by the
system only.

@item @file{/usr/local/bin/}
A possible location to place user-callable programs into.

@item @file{/usr/local/sbin/}
A possible location to place sysmanager-callable programs into.

@item @file{/var/spool/bitnet/}
Outgoing file spooling directory (and its subdirectories);
defined on @file{/etc/funetnje.cf}.

@item @file{/var/spool/bitspool/}
Incoming file spooling directory -- each user has his/her own subdir;
defined on @file{/etc/funetnje.cf}.
@end table

@c ----------------------------------------------------------------
@node cmdpipe, iucvpipe, directories, Nuts and Bolts
@section Client to Emulator messaging
@c      - client -> emulator messaging

There are multiple methods (depending upon your operating system)
on how to use the client->emulator command channel.


@c ----------------------------------------------------------------
@node iucvpipe, spool format, cmdpipe, Nuts and Bolts
@section IUCVTRAP <=> Emulator messaging
@c      - iucvtrap <-> emulator messaging

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node spool format, netdata, iucvpipe, Nuts and Bolts
@section FUNET-NJE Spool Format
@c      - spool file format

@emph{Straight copy of man-page: bitspool(5)}

@emph{Bitspool} format consists of an @strong{ASCII}
header, and a binary body containing length-tagged
``records''.
The length-tag is ``@emph{network byte order}''-ordered 16 bit short integer.
This is to facilitate transfer of spool files in between systems, and
to make it simpler to use non-homogenous hardware in clusters which
share the same @strong{BITNET} node..

Known @strong{ASCII} headers are:
@table @asis
@item @strong{FRM:} USERNAME@@HOSTNAME
Mandatory (fall-back default: ``***@@****'').
Uppercase-only data in fixed 17 chars wide field, left justified.

@item @strong{TOA:} USERNAME@@HOSTNAME
Mandatory (fall-back default: ``POSTMAST@@<localname>'').
Uppercase-only data in fixed 17 chars wide field, left justified.

@item @strong{FNM:} FILENAME
Optional, default value: ``UNKNOWN''.
Mixed-case alphanumeric data in fixed 12 chars wide field, left justified.

@item @strong{EXT:} FILEEXTN
Optional, default value: ``DATA''.
Mixed-case alphanumeric data in fixed 12 chars wide field, left justified.

@item @strong{TYP:} type
Optional, defaults to ``PUNCH''.
Controls physical device attribute on the outgoing NJE headers.
When sending PRINT files, use @strong{TYP: PRINT}.
When you want to send SYSIN jobs, use @strong{TYP: JOB},
and if it is just a PUNCH something, use @strong{TYP: PUNCH}.

Recognized obsolete keywords for the TYP are:
MAIL, BSMTP, FILE, and FASA, which are in reality synonyms for PUNCH,
and PASA, which is synonym for PRINT.

@item @strong{CLS:} class
Optional, defaulted by the @strong{TYP:}, if not defined.
This is @strong{one} (1) uppercase alpha in range A-Z.

@c .br
Customary values are:
MAIL, BSMTP files: M, NETDATA containing binary data: N,
console printouts: O, all others: A.

@item @strong{FOR:} FILEFORM
Optional, recommended.
If you don't want to see file-traversal notes from the network,
BITNET style is to start the form name with two characters: "QU".
That matches at "QUIET", "QUMAIL", ...

System wide default is taken from
@file{/etc/funetnje.cf DEFFORM}
entry, and it often has value
@strong{STANDARD}.

@item @strong{FMT:} spool-format
Mandatory (fall-back default: @strong{BINARY}).

@c .br
@strong{BINARY} for all locally generated files, and for cases
where the @strong{ASCII} headers have been mungled after
the file was received from the network.

@c .br
@strong{EBCDIC} denotes spool file which has been received
from the network, and contains all possible data blocks in it.
(Users will ignore all, but PUNCH, and PRINT records..)

@item @strong{FID:} 9900
Mandatory, sender original spoolid; A number between 1, and 9900.
File submitter should put "0000" in there, and delivery facility
will do something about it.. When system picks up this field, it
will assign there next spoolid from file sender's store, however
network delivery uses ``@strong{OID:}'', which it assigns..

@item @strong{OID:} 0000
Mandatory, for spool-id allocation at the first time it is queued
for delivery to somewhere remote.

@item @strong{TAG:} RSCS-TAG-STRING
Optional.
The IBM NJE TAG string of 136 characters.

@item @strong{DIS:} RSCS-DIST
Optional.
Fixed width, left justified 8 characters field of RSCS DIST information.
Default: 
``@strong{SYSTEM}''.

@item @strong{REC:}  nnnn
Highly preferred (almoast mandatory!)
Tells the number of records of data; necessary to get job-header
right on file transmission, altough not really mandatory for
successfull file sending.  As the count of records isn't
known before all are written (in the usual case), this entry
is written at first to reserve space for later seek, and rewite
with correct information.

@item @strong{JNM:} JOBNAME
Optional.
Contains NJE job name (8 characters).

@item @strong{PRT:} prtfiles@@tonode
Optional.
For @strong{SYSIN} jobs this defines alternate address for sending
the job PRINT outputs to. 
Default is taken from ``@strong{FRM:}''.

@item @strong{PUN:} punfiles@@tonode
Optional.
For @strong{SYSIN} jobs this defines alternate address for sending
the job PUNCH outputs to.
Default is taken from ``@strong{FRM:}''.

@item @strong{END:}
Mandatory.
Ends the @strong{ASCII}-portion of the headers.
Terminating newline must be immediately after the colon character.
@end table

With all (except @strong{END:}) above header tags there must be
exactly one space (ASCII 32) after the colon, and before data.
With the @strong{END:}, there must not be any extra blanks after
the colon, and before terminating newline.

Order of the @strong{ASCII} format headers is not fixed, except that
@strong{END:} must be last.

Binary format data follows syntax:
@example
    <16-bit length> <length number of bytes>
@end example
where the @strong{<16-bit length>} is encoded as
@strong{network-byte-order} short integer.
This is to facilitate easy copying of @emph{funetnje}(8)
spool files from one machine to another.
This also makes it possible to share common @strong{BITNET}
pool between several machines independent
of their native byte order.

Sample data formats:
@table @strong
@item PUNCH
@example
(any content, like plain text, and  @strong{NETDATA})
0x80, 80,                       0x80 = PUNCH, 80=content length
<80 bytes of EBCDIC/whatever>   The PUNCH content, 80 bytes
@end example

@item PRINT
Machine Carriage Control:
@example
0x90, 132, MCCbyte,             0x90 = PRINT MCC, 132=content length
<132 bytes of EBCDIC/whatever>  The PRINT content, 132 bytes
@end example

@item PRINT ASA
Carriage Control:
@example
0xA0, 132, ASAbyte,             0xA0 = PRINT ASA, 132=content length
<132 bytes of EBCDIC/whatever>  The PRINT content, 132 bytes
@end example

@item PRINT CPDS
@example
Carriage Control:
0xB0, 132, CPDSbyte,            0xB0 = PRINT CPDS, 132=content length
<132 bytes of EBCDIC/whatever>  The PRINT content, 132 bytes
@end example
@end table

Trailing @strong{EBCDIC}-spaces can be suppressed in the spool files,
thus above are given ranges of lengths.  If the record content is shorter,
than its defined size, receiver @emph{must} pad it with @strong{EBCDIC}-spaces.
(Of course, this way spool-size of a megabyte of spaces will be
13108 records of (2+)2 bytes + header, which makes circa 53kB.
When received, it will expand by a fractionally filled last record.)
The length-byte at each record @emph{must}
be of its nominal value, record truncation is coded with
the 16-bit-length value.

The @emph{MCCbyte}, and @emph{ASAbyte} are two kinds of
``@emph{carriage control}'' -mechanisms.
The @emph{CPDSbyte} is third kind, however documenting it is left for
further study.

IBM printers are fixed 132 characters wide, and on overall those
systems function on records, and a write in this respect means
write of whole line.
On overall the IO-operations on IBM mainframe happen on records, or
preferrably chains of records on which single start-IO instruction
can be used to start the IO-channel.

Carriage-control mechanisms contain two kinds of controls, both
pre-write, and post-write controls exist in MCC mode.
In ASA-mode only pre-writes are defined.
Pre-write controls happen before the printout line is written,
for example `jump to nearest VFU channel 1 forward' (in plain
english: Form-Feed..)

@table @emph
@item Jump to next page
ASA: `1', Pre-MCC: X'8B'

@item Overwrite; Defined only in pre-write.   
ASA: `\+', MCC: X'01'

@item Write and advance one line
ASA: Space, Pre-MCC: X'0B', Post-MCC: X'09'

@item Write and advance twolines
ASA: `0', Pre-MCC: X'13', Post-MCC: X'11'

@item Write and advance three lines
ASA: `-', Pre-MCC: X'1B', Post-MCC: X'19'

@end table


Other formats, than  0x80, 0x90, 0xA0, 0xB0  of binary portion records
should be ignored. (Received network transport layer headers are there, etc.)


@c ----------------------------------------------------------------
@node netdata, emulator logics, spool format, Nuts and Bolts
@section Implemented NETDATA features
@c      - implemented NETDATA features

@strong{**T.B.W.**}

@c ----------------------------------------------------------------
@node emulator logics,  , netdata, Nuts and Bolts
@section Emulator logics
@c      - emulator logics

@strong{**T.B.W.**}


@c ================================================================
@node Literature, ,Nuts and Bolts, Top
@c - Literature
@chapter Literature

Here is a partial list of things to read:
@itemize @bullet
@item IBM publication SC23-0070-02:
@emph{Network Job Entry -- Formats and Protocols}

@item IBM publication GG22-9373-02 @emph{(older version of previous)}

@item @file{NEWTAGS DESCRIPT} from your nearest NETSERV.

@item @file{NETDATA DESCRIPT} from your nearest NETSERV.

@item @file{BRFC0002 TEXT} from BITNIC

@item @file{TECOVER.LISTPS} from Princeton -- BITNET-II technical overview.
(LISTSERV@@PUCC ??)

@item @file{USRGUIDE.LISTPS} from Princeton -- VMNET users/operators guide.
(LISTSERV@@PUCC ??)

@end itemize

@c ================================================================
@contents
@bye
