\input texinfo   @c -*-texinfo-*-
@c %**start of header
@setfilename rx.info
@settitle Rx
@setchapternewpage odd
@c %**end of header

@c  This title page illustrates only one of the
@c  two methods of forming a title page.

@titlepage
@title Rx
@author Tom Lord

@center except the chapter "Posix Entry Points" 
@center from @emph{The GNU C Library} reference manual
@center by Sandra Loosemore
@center with
@center Richard M. Stallman, Roland McGrath, and Andrew Oram

@c  The following two commands
@c  start the copyright page.
@page
@vskip 0pt plus 1filll
Copyright @copyright{} 1995 Cygnus Support

except the chapter "Posix Entry Points" which is:

Copyright @copyright{} 1995 Free Software Foundation, Inc.

Permission is granted to make and distribute verbatim copies of
this manual provided the copyright notice and this permission notice
are preserved on all copies.

Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the entire
resulting derived work is distributed under the terms of a permission
notice identical to this one.

Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation approved
by the author.

@end titlepage

@node Top, Copying, (dir), (dir)
@top Rx

This document describes Rx.

@menu
* Copying::                     	Sharing is good.
* Overview::                    	Fnord
* Posix Basic Regular Expressions::  	A popular regexp syntax.
* Posix Entry Points::          	The POSIX way to regexp.
* Beyond POSIX::                	Hints about cool features.
* Rx Theory::                   	Hints about how it works.
@end menu

@node Copying, Overview, Top, Top
@chapter Copying

@center Copyright (C) 1996
@center Tom Lord
@center Berkeley, CA USA

@center except portions of "POSIX Regex Functions" which are
@center Copyright (C) 1995
@center Free Software Foundation, Inc.

@noindent
Permission to use, copy, modify, distribute, and sell this software and
its documentation for any purpose is hereby granted without fee,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation.

@center NO WARRANTY

@noindent
BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR
THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE
ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH
YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
NECESSARY SERVICING, REPAIR OR CORRECTION.

@noindent
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR
DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL
DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM
(INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED
INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF
THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR
OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

@node  Overview, Posix Basic Regular Expressions, Copying, Top
@chapter Overview

Nothing to say here, yet.

@node  Posix Basic Regular Expressions, Posix Entry Points, Overview, Top
@chapter Posix Basic Regular Expressions

The Posix Basic Regular Expression language is a notation for describing
textual patterns.  Regexps are typically used by comparing them to a
string to see if that string matches the pattern, or by searching within
a string for a substring that matches.

This chapter introduces the Posix regexp notation.  This is not a formal
or precise definition of Posix regexps -- it is an intuitive and
hopefully expository description of them.

@menu
* An Introduction to Regexps::  
* Literal Regexps::             
* Character Sets::              
* Subexpressions::              
* Repeated Subexpressions::     
* Optional Subexpressions::     
* Counted Subexpressions::      
* Alternative Subexpressions::  
* Backreferences::              
* A Summary of Regexp Syntax::  
* Ambiguous Patterns::          
@end menu


@node  An Introduction to Regexps, Literal Regexps, Posix Basic Regular Expressions, Posix Basic Regular Expressions
@section An Introduction to Regexps

In the simplest cases, a regexp is just a literal string that must
match exactly.   For example, the pattern:

@example
regexp
@end example

matches the string "regexp" and no others.

Some characters have a special meaning when they occur in a regexp.
They aren't matched literally as in the previous example, but instead
denote a more general pattern.   For example, the character @code{*}
is used to indicate that the preceeding element of a regexp may be
repeated 0, 1, or more times.  In the pattern:

@example
smooo*th
@end example

the @code{*} indicates that the preceeding @code{o} can be repeated 0 or
more times.   So the pattern matches:

@example
smooth
smoooth
smooooth
smoooooth
...
@end example

Suppose you want to write a pattern that literally matches a special
character like @code{*} -- in other words, you don't want to @code{*} to
indicate a permissible repetition, but to match @code{*} literally.  This
is accomplished by quoting the special character with a backslash.  
The pattern:

@example
smoo\*th
@end example

matches the string:

@example
smoo*th
@end example


and no other strings.

In seven cases, the pattern is reversed -- a backslash makes the
character special instead of making a special character normal.  The
characters @code{+}, @code{?}, @code{|}, @code{(}, and @code{)} are
normal but the sequences @code{\+}, @code{\?}, @code{\|}, @code{\(},
@code{\)}, @code{\@{}, and @code{\@}} are special (their meaning is
described later).

The remaining sections of this chapter introduce and explain the various
special characters that can occur in regexps.

@node  Literal Regexps, Character Sets, An Introduction to Regexps, Posix Basic Regular Expressions
@section Literal Regexps

A literal regexp is a string which contains no special characters.
A literal regexp matches an identical string, but no other characters.
For example:

@example
literally
@end example

matches

@example
literally
@end example

and nothing else.

Generally, whitespace characters, numbers, and letters are not special.
Some punctuation characters are special and some are not (the syntax
summary at the end of this chapter makes a convenient reference for
which characters are special and which aren't).


@node  Character Sets, Subexpressions, Literal Regexps, Posix Basic Regular Expressions
@section Character Sets

This section introduces the special characters @code{.} and @code{[}.

@code{.} matches any character except the NULL character.  For example:

@example
p.ck
@end example

matches

@example
pick
pack
puck
pbck
pcck
p.ck

...
@end example


@code{[} begins a @dfn{character set}.  A character set is similar to
@code{.} in that it matches not a single, literal character, but any
of a set of characters.   @code{[} is different from @code{.} in that
with @code{[}, you define the set of characters explicitly.

There are three basic forms a character set can take.

In the first form, the character set is spelled out:

@example
[<cset-spec>]	-- every character in <cset-spec> is in the set.
@end example

In the second form, the character set indicated is the negation of
a character set is explicitly spelled out:

@example
[^<cset-spec>]	-- every character *not* in <cset-spec> is in the set.
@end example

A @code{<cset-spec>} is more or less an explicit enumeration of a set
of characters.  It can be written as a string of individual characters:

@example
[aeiou]
@end example

or as a range of characters:

@example
[0-9]
@end example

These two forms can be mixed:

@example
[A-za-z0-9_$]
@end example

Note that special regexp characters (such as @code{*}) are @emph{not}
special within a character set.  @code{-}, as illustrated above,
@emph{is} special, except, as illustrated below, when it is the first
character mentioned.

This is a four-character set:

@example
[-+*/]
@end example

The third form of a character set makes use of a pre-defined "character
class":

@example
[[:class-name:]] -- every character described by class-name is in the set.
@end example

The supported character classes are:

@example
alnum	- the set of alpha-numeric characters
alpha	- the set of alphabetic characters
blank	- tab and space
cntrl	- the control characters
digit	- decimal digits
graph	- all printable characters except space
lower	- lower case letters
print	- the "printable" characters
punct	- punctuation
space	- whitespace characters
upper	- upper case letters
xdigit	- hexidecimal digits
@end example

Finally, character class sets can also be inverted:

@example
[^[:space:]] - all non-whitespace characters
@end example

Character sets can be used in a regular expression anywhere a literal
character can.

@node  Subexpressions, Repeated Subexpressions, Character Sets, Posix Basic Regular Expressions
@section Subexpressions

A subexpression is a regular expression enclosed in @code{\(} and
@code{\)}.  A subexpression can be used anywhere a single character or
character set can be used.

Subexpressions are useful for grouping regexp constructs.  For example,
the repeat operator, @code{*}, usually applies to just the preceeding
character.   Recall that:

@example
smooo*th
@end example

matches

@example
smooth
smoooth
...
@end example


Using a subexpression, we can apply @code{*} to a longer string:

@example
banan\(an\)*a
@end example

matches

@example
banana
bananana
banananana
...
@end example

Subexpressions also have a special meaning with regard to backreferences
and substitutions (see @xref{Backreferences}).


@node  Repeated Subexpressions, Optional Subexpressions, Subexpressions, Posix Basic Regular Expressions
@section Repeated Subexpressions


@code{*} is the repeat operator.  It applies to the preceeding
character, character set, subexpression or backreference.  It indicates
that the preceeding element can be matched 0 or more times:

@example
bana\(na\)*
@end example

matches

@example
bana
banana
bananana
banananana
...
@end example



@code{\+} is similar to @code{*} except that @code{\+} requires the
preceeding element to be matched at least once.  So while:

@example
bana\(na\)*
@end example

matches

@example
bana
@end example

@example
bana(na\)\+
@end example

does not.   Both match 

@example
banana
bananana
banananana
...
@end example

Thus, @code{bana\(na\)+} is short-hand for @code{banana\(na\)*}.


@node  Optional Subexpressions, Counted Subexpressions, Repeated Subexpressions, Posix Basic Regular Expressions
@section Optional Subexpressions

@code{\?} indicates that the preceeding character, character set, or
subexpression is optional.  It is permitted to match, or to be skipped:

@example
CSNY\?
@end example

matches both

@example
CSN
@end example

and 

@example
CSNY
@end example


@node  Counted Subexpressions, Alternative Subexpressions, Optional Subexpressions, Posix Basic Regular Expressions
@section Counted Subexpressions

An interval expression, @code{\@{m,n\@}} where @code{m} and @code{n} are
non-negative integers with @code{n >= m}, applies to the preceeding
character, character set, subexpression or backreference.  It indicates
that the preceeding element must match at least @code{m} times and may
match as many as @code{n} times.

For example:

@example
c\([ad]\)\@{1,4\@}
@end example

matches

@example
car
cdr
caar
cdar
...
caaar
cdaar
...
cadddr
cddddr
@end example



@node  Alternative Subexpressions, Backreferences, Counted Subexpressions, Posix Basic Regular Expressions
@section Alternative Subexpressions

An alternative is written:

@example
regexp-1\|regexp-2\|regexp-3\|...
@end example

It matches anything matched by some @code{regexp-n}.  For example:

@example
Crosby, Stills, \(and Nash\|Nash, and Young\)
@end example

matches

@example
Crosby, Stills, and Nash
@end example

and

@example
Crosby, Stills, Nash, and Young
@end example


@node  Backreferences, A Summary of Regexp Syntax, Alternative Subexpressions, Posix Basic Regular Expressions
@section Backreferences, Extractions and Substitutions

A backreference is written @code{\n} where @code{n} is some single digit
other than 0.  To be a valid backreference, there must be at least
@code{n} parenthesized subexpressions in the pattern prior to the
backreference.

A backreference matches a literal copy of whatever was matched by the
corresponding subexpression.  For example,

@example
\(.*\)-\1
@end example

matches:

@example
go-go
ha-ha
wakka-wakka
...
@end example

In some applications, subexpressions are used to extract substrings.
For example, Emacs has the functions @code{match-beginnning} and
@code{match-end} which report the positions of strings matched by
subexpressions.  These functions use the same numbering scheme for
subexpressions as backreferences, with the additional rule that
subexpression 0 is defined to be the whole regexp.

In some applications, subexpressions are used in string substitution.
This again uses the backreference numbering scheme. For example, this sed
command:

@example
s/From:.*<\(.*\)>/To: \1/
@end example

first matches the line:

@example
From: Joe Schmoe <schmoe@@uspringfield.edu>
@end example

when it does, subexpression 1 matches "schmoe@@uspringfield.edu".
The command replaces the matched line with "To: \1" after doing
subexpression substitution on it to get:

@example
To: schmoe@@uspringfield.edu
@end example


@node  A Summary of Regexp Syntax, Ambiguous Patterns, Backreferences, Posix Basic Regular Expressions
@section A Summary of Regexp Syntax

In summary, regexps can be:

@code{abcd} -- matching a string literally

@code{.} -- matching everything except NULL

@code{[a-z_?]}, @code{^[a-z_?]}, @code{[[:alpha:]]} and
@code{[^[:alpha:]]} -- matching character sets

@code{\(subexp\)} -- grouping an expression into a subexpression.

@code{\n} -- match a copy of whatever was matched by the nth subexpression.

The following special characters and sequences can be applied to a
character, character set, subexpression, or backreference:

@code{*} -- repeat the preceeding element 0 or more times.

@code{\+} -- repeat the preceeding element 1 or more times.

@code{\?} -- match the preceeding element 0 or 1 time.

@code{@{m,n@}} -- match the preceeding element at least @code{m}, and as
many as @code{n} times.

@code{regexp-1\|regexp-2\|..} -- match any regexp-n.

A special character, like @code{.} or @code{*} can be made into a literal
character by prefixing it with @code{\}.

A special sequence, like @code{\+} or @code{\?} can be made into a
literal character by dropping the @code{\}.


@node  Ambiguous Patterns,  , A Summary of Regexp Syntax, Posix Basic Regular Expressions
@section Ambiguous Patterns

Sometimes a regular expression appears to be ambiguous.  For example, suppose
we compare the pattern:

@example
begin\|beginning
@end example

to the string

@example
beginning
@end example

either just the first 5 characters will match, or the whole string will match.

In every case like this, the longer match is preferred.  The whole
string will match.

Sometimes there is ambiguity not about how many characters to match, but
where the subexpressions occur within the match.  This can effect
extraction functions like Emacs' @code{match-beginning} or rewrite
functions like sed's @code{s} command.  For example, consider matching
the pattern:

@example
b\(\[^q]*\)\(ing\)?
@end example

against the string

@example
beginning
@end example

One possibility is that the first subexpression matches "eginning" and
the second is skipped.  Another possibility is that the first
subexpression matches "eginn" and the second matches "ing".

The rule is that consistant with matching as many characters as 
possible, the length of lower numbered subexpressions is maximized
in preference to maximizing the length of later subexpressions.

In the case of the above example, the two possible matches are equal in
overall length.  Therefore, it comes down to maximizing the
lower-numbered subexpression, \1.  The correct answer is that \1 matches
"eginning" and \2 is skipped.

@node Posix Entry Points, Beyond POSIX, Posix Basic Regular Expressions, Top
@chapter Posix Entry Points

This section is excerpted from @emph{The GNU C Library} reference manual
by Sandra Loosemore with Richard M. Stallman, Roland McGrath, and Andrew
Oram.

The GNU C library supports the standard POSIX.2 interface.  Programs
using this interface should include the header file @file{rxposix.h}.

@menu
* POSIX Regexp Compilation::    Using @code{regcomp} to prepare to match.
* Flags for POSIX Regexps::     Syntax variations for @code{regcomp}.
* Matching POSIX Regexps::      Using @code{regexec} to match the compiled
				   pattern that you get from @code{regcomp}.
* Regexp Subexpressions::       Finding which parts of the string were matched.
* Subexpression Complications::  Find points of which parts were matched.
* Regexp Cleanup::              Freeing storage; reporting errors.
@end menu

@node POSIX Regexp Compilation, Flags for POSIX Regexps, Posix Entry Points, Posix Entry Points
@section POSIX Regular Expression Compilation

Before you can actually match a regular expression, you must
@dfn{compile} it.  This is not true compilation---it produces a special
data structure, not machine instructions.  But it is like ordinary
compilation in that its purpose is to enable you to ``execute'' the
pattern fast.  (@xref{Matching POSIX Regexps}, for how to use the
compiled regular expression for matching.)

There is a special data type for compiled regular expressions:

@comment regex.h
@comment POSIX.2
@deftp {Data Type} regex_t
This type of object holds a compiled regular expression.
It is actually a structure.  It has just one field that your programs
should look at:

@table @code
@item re_nsub
This field holds the number of parenthetical subexpressions in the
regular expression that was compiled.
@end table

There are several other fields, but we don't describe them here, because
only the functions in the library should use them.
@end deftp

After you create a @code{regex_t} object, you can compile a regular
expression into it by calling @code{regcomp}.

@comment regex.h
@comment POSIX.2
@deftypefun int regcomp (regex_t *@var{compiled}, const char *@var{pattern}, int @var{cflags})
@deftypefunx int regncomp (regex_t *@var{compiled}, const char *@var{pattern}, int @var{len}, int @var{cflags})
The function @code{regcomp} ``compiles'' a regular expression into a
data structure that you can use with @code{regexec} to match against a
string.  The compiled regular expression format is designed for
efficient matching.  @code{regcomp} stores it into @code{*@var{compiled}}.

The parameter @var{pattern} points to the regular expression to be
compiled.  When using @code{regcomp}, @var{pattern} must be
0-terminated.  When using @code{regncomp}, @var{pattern} must be
@var{len} characters long.

@code{regncomp} is not a standard function;  strictly POSIX programs
should avoid using it.

It's up to you to allocate an object of type @code{regex_t} and pass its
address to @code{regcomp}.

Before freeing the object of type @code{regex_t} You @emph{must} pass it
to @code{regfree}.  Not doing so may cause subsequent calls to Rx
functions to behave strangely.

The argument @var{cflags} lets you specify various options that control
the syntax and semantics of regular expressions.  @xref{Flags for POSIX
Regexps}.

If you use the flag @code{REG_NOSUB}, then @code{regcomp} omits from
the compiled regular expression the information necessary to record
how subexpressions actually match.  In this case, you might as well
pass @code{0} for the @var{matchptr} and @var{nmatch} arguments when
you call @code{regexec}.

If you don't use @code{REG_NOSUB}, then the compiled regular expression
does have the capacity to record how subexpressions match.  Also,
@code{regcomp} tells you how many subexpressions @var{pattern} has, by
storing the number in @code{@var{compiled}->re_nsub}.  You can use that
value to decide how long an array to allocate to hold information about
subexpression matches.

@code{regcomp} returns @code{0} if it succeeds in compiling the regular
expression; otherwise, it returns a nonzero error code (see the table
below).  You can use @code{regerror} to produce an error message string
describing the reason for a nonzero value; see @ref{Regexp Cleanup}.
@end deftypefun

Here are the possible nonzero values that @code{regcomp} can return:

@table @code
@comment regex.h
@comment POSIX.2
@item REG_BADBR
There was an invalid @samp{\@{@dots{}\@}} construct in the regular
expression.  A valid @samp{\@{@dots{}\@}} construct must contain either
a single number, or two numbers in increasing order separated by a
comma.

@comment regex.h
@comment POSIX.2
@item REG_BADPAT
There was a syntax error in the regular expression.

@comment regex.h
@comment POSIX.2
@item REG_BADRPT
A repetition operator such as @samp{?} or @samp{*} appeared in a bad
position (with no preceding subexpression to act on).

@comment regex.h
@comment POSIX.2
@item REG_ECOLLATE
The regular expression referred to an invalid collating element (one not
defined in the current locale for string collation).  

@comment regex.h
@comment POSIX.2
@item REG_ECTYPE
The regular expression referred to an invalid character class name.

@comment regex.h
@comment POSIX.2
@item REG_EESCAPE
The regular expression ended with @samp{\}.

@comment regex.h
@comment POSIX.2
@item REG_ESUBREG
There was an invalid number in the @samp{\@var{digit}} construct.

@comment regex.h
@comment POSIX.2
@item REG_EBRACK
There were unbalanced square brackets in the regular expression.

@comment regex.h
@comment POSIX.2
@item REG_EPAREN
An extended regular expression had unbalanced parentheses,
or a basic regular expression had unbalanced @samp{\(} and @samp{\)}.

@comment regex.h
@comment POSIX.2
@item REG_EBRACE
The regular expression had unbalanced @samp{\@{} and @samp{\@}}.

@comment regex.h
@comment POSIX.2
@item REG_ERANGE
One of the endpoints in a range expression was invalid.

@comment regex.h
@comment POSIX.2
@item REG_ESPACE
@code{regcomp} ran out of memory.
@end table

@node Flags for POSIX Regexps, Matching POSIX Regexps, POSIX Regexp Compilation, Posix Entry Points
@section Flags for POSIX Regular Expressions

These are the bit flags that you can use in the @var{cflags} operand when
compiling a regular expression with @code{regcomp}.
 
@table @code
@comment regex.h
@comment POSIX.2
@item REG_EXTENDED
Treat the pattern as an extended regular expression, rather than as a
basic regular expression.

@comment regex.h
@comment POSIX.2
@item REG_ICASE
Ignore case when matching letters.

@comment regex.h
@comment POSIX.2
@item REG_NOSUB
Don't bother storing the contents of the @var{matches-ptr} array.

@comment regex.h
@comment POSIX.2
@item REG_NEWLINE
Treat a newline in @var{string} as dividing @var{string} into multiple
lines, so that @samp{$} can match before the newline and @samp{^} can
match after.  Also, don't permit @samp{.} to match a newline, and don't
permit @samp{[^@dots{}]} to match a newline.

Otherwise, newline acts like any other ordinary character.
@end table

@node Matching POSIX Regexps, Regexp Subexpressions, Flags for POSIX Regexps, Posix Entry Points
@section Matching a Compiled POSIX Regular Expression

Once you have compiled a regular expression, as described in @ref{POSIX
Regexp Compilation}, you can match it against strings using
@code{regexec}.  A match anywhere inside the string counts as success,
unless the regular expression contains anchor characters (@samp{^} or
@samp{$}).

@comment regex.h
@comment POSIX.2
@deftypefun int regexec (regex_t *@var{compiled}, char *@var{string}, size_t @var{nmatch}, regmatch_t @var{matchptr} @t{[]}, int @var{eflags})
@deftypefunx int regnexec (regex_t *@var{compiled}, char *@var{string}, int @var{len}, size_t @var{nmatch}, regmatch_t @var{matchptr} @t{[]}, int @var{eflags})
This function tries to match the compiled regular expression
@code{*@var{compiled}} against @var{string}.

@code{regexec} returns @code{0} if the regular expression matches;
otherwise, it returns a nonzero value.  See the table below for
what nonzero values mean.  You can use @code{regerror} to produce an
error message string describing the reason for a nonzero value; 
see @ref{Regexp Cleanup}.

The parameter @var{string} points to the text to search.  When using
@code{regexec}, @var{string} must be 0-terminated.  When using
@code{regnexec}, @var{string} must be @var{len} characters long.

@code{regnexec} is not a standard function;  strictly POSIX programs
should avoid using it.

The argument @var{eflags} is a word of bit flags that enable various
options.

If you want to get information about what part of @var{string} actually
matched the regular expression or its subexpressions, use the arguments
@var{matchptr} and @var{nmatch}.  Otherwise, pass @code{0} for 
@var{nmatch}, and @code{NULL} for @var{matchptr}.  @xref{Regexp
Subexpressions}.
@end deftypefun

You must match the regular expression with the same set of current
locales that were in effect when you compiled the regular expression.

The function @code{regexec} accepts the following flags in the
@var{eflags} argument:

@table @code 
@comment regex.h
@comment POSIX.2
@item REG_NOTBOL
Do not regard the beginning of the specified string as the beginning of
a line; more generally, don't make any assumptions about what text might
precede it.

@comment regex.h
@comment POSIX.2
@item REG_NOTEOL
Do not regard the end of the specified string as the end of a line; more
generally, don't make any assumptions about what text might follow it.
@end table

Here are the possible nonzero values that @code{regexec} can return:

@table @code
@comment regex.h
@comment POSIX.2
@item REG_NOMATCH
The pattern didn't match the string.  This isn't really an error.

@comment regex.h
@comment POSIX.2
@item REG_ESPACE
@code{regexec} ran out of memory.
@end table

@node Regexp Subexpressions, Subexpression Complications, Matching POSIX Regexps, Posix Entry Points
@section Match Results with Subexpressions

When @code{regexec} matches parenthetical subexpressions of
@var{pattern}, it records which parts of @var{string} they match.  It
returns that information by storing the offsets into an array whose
elements are structures of type @code{regmatch_t}.  The first element of
the array (index @code{0}) records the part of the string that matched
the entire regular expression.  Each other element of the array records
the beginning and end of the part that matched a single parenthetical
subexpression.

@comment regex.h
@comment POSIX.2
@deftp {Data Type} regmatch_t
This is the data type of the @var{matcharray} array that you pass to
@code{regexec}.  It containes two structure fields, as follows:

@table @code
@item rm_so
The offset in @var{string} of the beginning of a substring.  Add this
value to @var{string} to get the address of that part.

@item rm_eo
The offset in @var{string} of the end of the substring.
@end table
@end deftp

@comment regex.h
@comment POSIX.2
@deftp {Data Type} regoff_t
@code{regoff_t} is an alias for another signed integer type.
The fields of @code{regmatch_t} have type @code{regoff_t}.
@end deftp

The @code{regmatch_t} elements correspond to subexpressions
positionally; the first element (index @code{1}) records where the first
subexpression matched, the second element records the second
subexpression, and so on.  The order of the subexpressions is the order
in which they begin.

When you call @code{regexec}, you specify how long the @var{matchptr}
array is, with the @var{nmatch} argument.  This tells @code{regexec} how
many elements to store.  If the actual regular expression has more than
@var{nmatch} subexpressions, then you won't get offset information about
the rest of them.  But this doesn't alter whether the pattern matches a
particular string or not.

If you don't want @code{regexec} to return any information about where
the subexpressions matched, you can either supply @code{0} for
@var{nmatch}, or use the flag @code{REG_NOSUB} when you compile the
pattern with @code{regcomp}.

@node Subexpression Complications, Regexp Cleanup, Regexp Subexpressions, Posix Entry Points
@section Complications in Subexpression Matching

Sometimes a subexpression matches a substring of no characters.  This
happens when @samp{f\(o*\)} matches the string @samp{fum}.  (It really
matches just the @samp{f}.)  In this case, both of the offsets identify
the point in the string where the null substring was found.  In this
example, the offsets are both @code{1}.

Sometimes the entire regular expression can match without using some of
its subexpressions at all---for example, when @samp{ba\(na\)*} matches the
string @samp{ba}, the parenthetical subexpression is not used.  When
this happens, @code{regexec} stores @code{-1} in both fields of the
element for that subexpression.

Sometimes matching the entire regular expression can match a particular
subexpression more than once---for example, when @samp{ba\(na\)*}
matches the string @samp{bananana}, the parenthetical subexpression
matches three times.  When this happens, @code{regexec} usually stores
the offsets of the last part of the string that matched the
subexpression.  In the case of @samp{bananana}, these offsets are
@code{6} and @code{8}.

But the last match is not always the one that is chosen.  It's more
accurate to say that the last @emph{opportunity} to match is the one
that takes precedence.  What this means is that when one subexpression
appears within another, then the results reported for the inner
subexpression reflect whatever happened on the last match of the outer
subexpression.  For an example, consider @samp{\(ba\(na\)*s \)*} matching
the string @samp{bananas bas }.  The last time the inner expression
actually matches is near the end of the first word.  But it is 
@emph{considered} again in the second word, and fails to match there.
@code{regexec} reports nonuse of the ``na'' subexpression.

Another place where this rule applies is when the regular expression
@w{@samp{\(ba\(na\)*s \|nefer\(ti\)* \)*}} matches @samp{bananas nefertiti}.
The ``na'' subexpression does match in the first word, but it doesn't
match in the second word because the other alternative is used there.
Once again, the second repetition of the outer subexpression overrides
the first, and within that second repetition, the ``na'' subexpression
is not used.  So @code{regexec} reports nonuse of the ``na''
subexpression.

@node Regexp Cleanup,  , Subexpression Complications, Posix Entry Points
@section POSIX Regexp Matching Cleanup

When you are finished using a compiled regular expression, you must
free the storage it uses by calling @code{regfree}.

@comment regex.h
@comment POSIX.2
@deftypefun void regfree (regex_t *@var{compiled})
Calling @code{regfree} frees all the storage that @code{*@var{compiled}}
points to.  This includes various internal fields of the @code{regex_t}
structure that aren't documented in this manual.

@code{regfree} does not free the object @code{*@var{compiled}} itself.
@end deftypefun

You should always free the space in a @code{regex_t} structure with
@code{regfree} before using the structure to compile another regular
expression.

When @code{regcomp} or @code{regexec} reports an error, you can use
the function @code{regerror} to turn it into an error message string.

@comment regex.h
@comment POSIX.2
@deftypefun size_t regerror (int @var{errcode}, regex_t *@var{compiled}, char *@var{buffer}, size_t @var{length})
This function produces an error message string for the error code
@var{errcode}, and stores the string in @var{length} bytes of memory
starting at @var{buffer}.  For the @var{compiled} argument, supply the
same compiled regular expression structure that @code{regcomp} or
@code{regexec} was working with when it got the error.  Alternatively,
you can supply @code{NULL} for @var{compiled}; you will still get a
meaningful error message, but it might not be as detailed.

If the error message can't fit in @var{length} bytes (including a
terminating null character), then @code{regerror} truncates it.
The string that @code{regerror} stores is always null-terminated
even if it has been truncated.

The return value of @code{regerror} is the minimum length needed to
store the entire error message.  If this is less than @var{length}, then
the error message was not truncated, and you can use it.  Otherwise, you
should call @code{regerror} again with a larger buffer.

Here is a function which uses @code{regerror}, but always dynamically
allocates a buffer for the error message:

@smallexample
char *get_regerror (int errcode, regex_t *compiled)
@{
  size_t length = regerror (errcode, compiled, NULL, 0);
  char *buffer = xmalloc (length);
  (void) regerror (errcode, compiled, buffer, length);
  return buffer;
@}
@end smallexample
@end deftypefun

@node  Beyond POSIX, Rx Theory, Posix Entry Points, Top
@chapter Beyond POSIX

This section is not finished documentation, but rather a collection of
pointers towards some of the interesting, non-standard features of Rx.

@section New Regexp Operators

Rx supports some unusual regexp syntax.

@code{[[:cut N:]]} sets @code{pmatch[0].final_tag} to N and causes the
matching to stop instantly.  If N is 0, the overall match fails,
otherwise it succeeds.

@code{[[:(:]] ... [[:):]]} is just like @code{\( ... \)} except that in
the first case, no pmatch entries are changed, and the subexpression is
not counted in the numbering of parenthesized subexpressions.

@code{[[:(:]] ... [[:):]]} can be used when you do not need to know
where a subexpression matched but are only using parentheses to effect
the parsing of the regexp.  

There are two reasons to use @code{[[:(:]] ... [[:):]]}:

1. regexec will run faster.

2. Currently, only 8 backreferencable subexpressions are supported:
@code{\1 .. \9}.  Using @code{[[:(:]] ... [[:):]]} is a way to conserve
backreferencable subexpression names in an expression with many
parentheses.

@section New POSIX Functions

@code{regncomp} and @code{regnexec} are non-standard generalizations of
@code{regcomp} and @code{regexec}.

@section Tuning POSIX performance

Two mysterious parmaters can be used to trade-off performance and
memory use.

At compile-time they are @code{RX_DEFAULT_DFA_CACHE_SIZE} and
@code{RX_DEFAULT_NFA_DELAY}.  

If you want to mess with these (I generally don't advise it), I suggest
experimenting for your particular application/memory situation; frob
these by powers of two and try out the results on what you expect will
be typical regexp workloads.

You can also set those parameters at run-time (before calling any regexp
functions) by tweaking the corresponding variables:

@code{rx_default_cache->bytes_allowed}

and

@code{rx_basic_unfaniverse_delay}



@section POSIX stream-style interface

@code{rx_make_solutions}, @code{rx_next_solution}, and
@code{rx_free_solutions} are a lower level alternative to the posix
functions.  Using those functions, you can compare a compiled regexp to
a string that is not contiguous in memory or even a string that is not
entirely in memory at any one time.

The code in rxposix.c points out how those functions are used.


@section DFAs Directly

If you are only interested in pure regular expressions (no pmatch data,
no backreferences, and no counted subexpressions), you can parse a
regexp using @code{rx_parse}, convert it to an nfa using @code{rx_unfa},
and run the dfa using @code{rx_init_system}, @code{rx_advance_to_final},
and @code{rx_terminate_system}.  The dfa Scheme primitives in
@file{rgx.c} may provide some guide.

@node    Rx Theory,  , Beyond POSIX, Top
@chapter Rx Theory
@cindex Rx Theory

There are two match algorithms.  One is for truly regular regexps (those
that can be reduced to a dfa).  The other is for non-regular regexps.

The dfa algorithm implements the idea suggested in @cite{Compilers} by
Aho, Sethi and Ullman:

@quotation
[One] approach [to pattern matching regular expressions] is to use a
DFA, but avoid constructing all of the transition table by using a
technique called "lazy transition evaluation".  Here, transitions are
computed at run time [when] actually needed. [T]ransitions are
stored in a cache. [....] If the cache becomes full, we can erase some
previously computed transition to make room for the new transition.
@end quotation

The implementation in Rx is generalized from that, but the above
description covers what is used for Posix patterns. 

The non-dfa algorithm implements a "recursive decomposition" technique
described in email by Henry Spencer.  For a given pattern, this
algorithm first checks to see if a simpler, superset language,
DFA-pattern matches.  If it does, then this algorithm does the
detail-work to see if the non-DFA pattern matches.

The detail work usually involves recursing on subpatterns.  For example,
a concatentation of two subexpressions matches a string if the string
can be divided into two parts, each matching one subexpression, in the
right order.  More than one solution is often possible for a given
pattern.  This ambiguity is the subject of the "leftmost longest" rules
in the spec, and the back-tracking oriented stream-of-solution functions
@code{rx_make_solutions}, @code{rx_next_solution} and
@code{rx_free_solutions}.

@example
rxspencer.[ch]	 			-- The non-DFA algorithm
rxanal.[ch] rxsuper.[ch] rxnfa.[ch]	-- The DFA algorithm
@end example

@bye

