EXTRACT(8)

NAME
extract - Display records from a tcplogger or udplogger binary log file.

SYNOPSIS
extract [ -e program ] [ -E program ] [ -f program file ] [ -F program file ] [ -d log file ] [ -o output file ] [ -a ] [ -b ] [ -n ] [ -u ] [ -U ] [ -t ] [ -h ]

DESCRIPTION
extract is used for selecting records from a binary log file created with tcplogger(8) or udplogger(8). The program consists of awk(1) like statements used to select records from the file.

OPTIONS

-e program
    Specify an extract program on the command line.

-E program
    Specify an extract program on the command line.

-f program file
    Specify a file containing an extract program.

-F program file
    Specify a file containing an extract program. Multiple -e, -f, -E, and -F options can be specified. They are all combined together and treated as a single program. Programs specified by -e and -f are appended to the program, while programs specified with -E and -F are prepended. -E and -F can be used with extract scripts to insert addition conditions at the top of the script without having to edit it each time.

-d log file
    Specify the tcplogger or udplogger binary log file to process. This defaults to stdin.

-o output file
    Specify the file where output should go. This defaults to stdout.

-a  Write selected records in ASCII. This is the default.

-b  Write selected records in binary, for subsequent reprocessing by extract. The format of the records is

    	struct timeval  tm;   /* Time of request */
    	u_long ipsrcaddr;     /* IP source address */
    	u_long ipdstaddr;     /* IP destination address */
    	u_long tcpseqno;      /* TCP sequence number */
    	u_short tcpsrcport;   /* TCP source port */
    	u_short tcpdstport;   /* TCP destination port */

-n  Do not attempt to do name lookups on IP addresses or port numbers when doing ASCII output. This can speed things up considerably.

-t  Specify that extract should work in TCP mode. TCP mode is the default.

-u  Specify that extract should work in UDP mode.

-U  Do not perform input buffering. This allows extract to read from a pipe attached to tcplogger or udplogger and process the data without lengthy buffering delays.

-h  Print out brief help message.

USAGE

  Programs

A program consists of selector-action statements. Statements have the form

    selector { action }

A missing selector means to always select the record. Extract processes the entire program for each record in the log file unless a matching statement contains a next action.

  Actions

There are two actions currently define, print and next.

Print outputs the record in the format selected using the -a and -b switches.

Next skips to the next record and restarts scanning of program.

  Selectors
The following selectors are available.

net =, != netname|ipaddr[/netmask]

Select records if the source or destination network is [not] in the
    network specified. The optional netmask allows subnets to be specified.

srcnet =, != netname|ipaddr[/netmask]

Select records if the source network is [not] in the
    network specified. The optional netmask allows subnets to be specified.

dstnet =, != netname|ipaddr[/netmask]

Select records if the destination network is [not] in the
    network specified. The optional netmask allows subnets to be specified.

host =, !=, <, >, <=, >= hostname|ipaddr

Select records based on the source or destination host.

srchost =, !=, <, >, <=, >= hostname|ipaddr

Select records based on the source host.

dsthost =, !=, <, >, <=, >= hostname|ipaddr

Select records based on the destination host.

srcport =, !=, <, >, <=, >= service-name|port

Select records based on the UDP or TCP source port.
    Service-name is an entry from the /etc/services file. Port is an integer number indicating the port.

dstport =, !=, <, >, <=, >= service-name|port

Select records based on the UDP or TCP destination port.
    Service-name is an entry from the /etc/services file. Port is an integer number indicating the port.

date =, !=, <, >, <=, >= MM/DD/YY | MM/DD | today | yesterday

Select records based on the date of the record.

time =, !=, <, >, <=, >= HH:MM:SS | HH:MM

Select records based on the time of the record.

before MM/DD/YY [HH:MM] | today [HH:MM] | yesterday [HH:MM]

Select records based on the time of the record.

since MM/DD/YY [HH:MM] | today [HH:MM] | yesterday [HH:MM]

Select records based on the time of the record.

  Combining Selectors
Selectors can be combined using the following operators.

and,&&
    Short circuited boolean AND.

or,||
    Short circuited boolean OR.

not,!
    Unary negation operators.

Parenthesis can be used to group expressions.

EXAMPLES

Print in ASCII any records showing attempts to rlogin to host myhost.

    extract -e 'dsthost = myhost && dstport = login {print}' -d tcplog

Sample script called unusual to print records for unexpected TCP traffic. This would have to be tailored to a site.

    #!/usr/local/etc/bin/extract -f
    # Script 'unusual' for printing unusual traffic
    dstport = telnet ||
    	dstport = ftp ||
    	dstport = finger {next}
    dstport > 1023 && (dstport != 2000 && dstport != 6000) {next}
    dstport = smtp && dsthost = mailhost {next}
    dstport = nntp && dsthost = newshost {next}
    # If we made it to here, then print the record
    {print}

Run the previous script, but only show records for today.

    unusual -E 'before today {next}'

SEE ALSO
tcplogger(8), udplogger(1),