NAME
    Set::Files - routines to work with files, each definining a single set

SYNOPSIS
      use Set::Files;
      $Version = $Set::Files::VERSION;

      $obj     = new Set::Files(OPT => VAL, OPT => VAL, ...);

      @set     = $obj->list_sets( [TYPE] );

      @uid     = $obj->owner;
      $uid     = $obj->owner(SET);

      @set     = $obj->owned_by(UID [,TYPE]);

      @ele     = $obj->members(SET);

      $flag    = $obj->is_member(SET, ELE);

      @type    = $obj->list_types( [SET] );

      @dir     = $obj->dir;
      $dir     = $obj->dir(SET);

      %opts    = $obj->opts(SET);
      $val     = $obj->opts(SET,VAR);

      $obj->cache;

      $num     = $obj->add   (SET, FORCE, COMMIT, ELE1,ELE2,...);
      $num     = $obj->remove(SET, ELE1,ELE2,...);

      $obj->commit(SET1,SET2,...);

      $obj->delete(SET);
      $obj->delete(SET,1);

DESCRIPTION
    This is a module for working with simple sets of elements where each set
    is defined in a separate file (one file for each set to be defined).

    The advantages of putting each set in a separate file are:

    Set managment can be delegated
        If all sets are defined in a single file, management of all sets
        must be done by a single user, or by using a suid program. By
        putting each set in a separate file, different files can be owned by
        different users so management of different sets can be delegated.

    Set files are a simple format
        Because a file consists of a single set only, there is no need to
        have a complex file format which has to be parsed to get information
        about the set. As a result, set files can easily be autogenerated or
        edited with any simple text editor, and errors are less likely to be
        introduced into the file.

    The disadvantages are:

    Permissions problems
        Some applications may need to read all of the data, but since the
        different set files may be owned by different people, permissions
        may get set such that not all set files are readable.

        Applications which actually gather all of the data will need to be
        run as root in order to be reliable. Alternately, some means of
        enforcing the appropriate permissions needs to be in place.

    No central data location
        Usually, when you want to define sets, the data ultimately needs to
        be stored in one central location (which might be a single file or
        database).

        To get around this, a wrapper must be written using this module to
        copy the data to the central location.

    Simple elements only
        Many types of sets have elements which have attributes (for example,
        a ranking within the set or some other attribute). When you start
        adding attributes, you need a more complex file structure in order
        to store this information, so that type of set is not addressed with
        this module. The only attribute that an element has is membership in
        the set.

    Slow data access
        Because the data is spread out over several files, each of which
        must be parsed, and any error checking done, accessing the data can
        be significantly slower than if the data were stored in a central
        location.

    Features of this module include:

    Data caching
        This module provides routines for caching the information from all
        the set files. This can be used to avoid the permissions problems
        (allowing user run applications access to all cached data) and
        decrease access time (no parsing is left, and error checking can be
        done prior to caching the information).

        This still requires that a privileged user or suid script be used to
        update the cache.

    Multiple type of sets
        Often, it is conveniant to define different types of sets using a
        single set of files as there may be considerable overlap between the
        sets of different types.

        For example, it might be useful to create files containing sets of
        users who belong to different committees in a department. Also,
        there might be sets of users who belong to various departmental
        mailing lists. One solution is to have two different directories,
        one with set files with lists of users on the various committees;
        one with set files with lists of users on each mailing list. Since
        there might be overlap between these groups, it might be nice to
        have the two sets of files overlap. For example, some committees may
        want to have a mailing list associated with the group, others don't
        want a mailing list, and there may be mailing lists not associated
        with a committee.

        This allows you to have a single file for each set of users, but
        some sets will have mailing lists, some will be committees, and some
        will be both.

    Set ownership
        Since the different files may be owned by different people,
        operations based on set ownership can be done.

METHODS
    The following methods are available:

    VERSION
          use Set::Files;
          $Version=$Set::Files::VERSION;

        Check the module version.

    new
          $obj = new Set::Files(OPT => VAL, OPT => VAL, ...);

        This creates a new Set::Files object which reads the appropriate set
        files (or a cache of the information in set files). The
        initialization options available are described below.

    list_sets
          @set     = $obj->list_sets( [TYPE] );

        Returns a list of all defined sets or the sets of the specified
        type.

    owner
          @uid     = $obj->owner;
          $uid     = $obj->owner(SET);

        Lists all UIDs who own a set, or the owner of the specified set.

    owned_by
          @set     = $obj->owned_by(UID [,TYPE]);

        Lists all sets owned by the specified UID (or those of a specific
        type).

    members
          @ele     = $obj->members(SET);

        Lists all elements in the specified set.

    is_member
          $flag    = $obj->is_member(SET, ELE);

        Returns 1 if ELE is a member of SET.

    list_types
          @type    = $obj->list_types( [SET] );

        A list of all types defined, or the types that the specified set
        belong to.

    dir
          @dir     = $obj->dir;
          $dir     = $obj->dir(SET);

        All directories containing set files, or the directory containing
        the file of the specified set.

    opts
          %opts    = $obj->opts(SET);
          $val     = $obj->opts(SET,VAR);

        Returns a hash of all options set for a set, or the value of a
        specific option. If the specific option is not set, 0 is returned.

    delete
          $obj->delete($set);
          $obj->delete($set,1);

        This removes the specified set file. By default, it renames the set
        file to .set_files.$set (which are ignored when reading in set
        data). If the optional second argument is passed in, no backup is
        made (i.e. the set file is deleted completely).

        This method is only available to those who have write access to the
        directory containing the set file.

    cache
          $obj->cache;

        This dumps the current set information to a cache file. This method
        is only valid if the data was read in from files. If it was read in
        from the cache, this method will fail.

    add, remove
          $num = $obj->add   (SET, FORCE, COMMIT, ELE1,ELE2,...);
          $num = $obj->remove(SET, FORCE, COMMIT, ELE1,ELE2,...);

        These functions add/remove the specified elements to/from the set.

        When adding elements to a set, it is first checked to see if the
        element is already in the set, and if so, whether it is explicitely
        excluded in the set file, or comes from some other set file via. an
        INCLUDE tag.

        If the element is not in the set, it is added. If the FORCE flag is
        true, the element will be added to the set file explicitly if it is
        already in the set, but only via. an INCLUDE tag. In either case,
        any OMIT tag which removes this element will be removed from the
        list.

        When removing elements from a set, a similar set of tests are done.
        If the element is in the set, it is removed from the file (if it
        appears in the file) AND a OMIT tag is included. If the element does
        NOT appear in the set, the file is unmodified unless the FORCE flag
        is true, in which case an OMIT tag is added.

        The COMMIT flag is used to determine whether the file should be
        written out over the existing file. The file can only be written out
        if data was read from the files. If it was read in from the cache,
        this will fail.

        The return value is the number of changes made to the set.

    commit
          $obj->commit(SET1,SET2,...);

        Any changes that have been made with the add and remove methods can
        be written out to the set file(s) with this method. This method is
        only valid if the data was read in from files. If it was read in
        from the cache, this method will fail.

INIT OPTIONS
    The following options can be passed in to the new method:

    path
          path => DIR1:DIR2:...
          path => [ DIR1, DIR2, ... ]

        The set files may be stored in one or more different directories. By
        default, set files are assumed to be in the current directory, but
        using this option, the directory (or directories) can be explicitely
        set.

        One thing to note. If multiple directories are used, and a file of
        the same name exists in more than one of the directories, the first
        one found (in the order that the directories are included in the
        list) is used. A warning will be issued for files of the same name
        in other directories, but they will be ignored.

        Warnings will be issued for unreadable directories, or unreadable
        files within a directory.

    valid_file
          valid_file => REGEXP
          valid_file => !REGEXP
          valid_file => \&FUNCTION

        By default, all files in the directories are used. With this option,
        filenames are tested and only those that pass will be used. Others
        will be silently ignored.

        REGEXP is a regular expression. Only filenames which match the
        REGEXP will pass (or if !REGEXP is used, only filenames which do NOT
        match REGEXP will pass).

        If a reference to a function is passed in, the function
        &FUNCTION(dir,file) will be evaluated for each file. If it returns
        0, the file will be silently ignored. Otherwise it will be used.

    invalid_quiet
          invalid_quiet = 1

        By default, when a file is ignored due to failing a valid_file test,
        or when an element is ignored due to failing a valid_ele test, a
        warning is issued. With this option, no warning is issued.

    cache
          cache => DIR

        Data from the set files may be cached in order to speed up data
        access. If this option is used, you must specify the directory where
        the data will be cached. The directory may be the same as one of the
        directories containing the set files.

        The cache directory defaults to the first directory given in the
        path option (or the current directory if no path option is given).

    read
          read => "cache"
          read => "files"
          read => "file"

        When an application wants to use data from the set files, they can
        either read the data from set files or the cache.

        If the cache option was used, the default is to read from the cache
        if it exists, read from the files otherwise. If no cache option was
        used, the default is to read from the files. When data is read in
        from the cache, the commit and cache methods are disabled.

        If the file option is used, it reads a single set from a single file
        along with all dependancy sets (i.e. sets that are included or
        excluded via. the appropriate tags). This allows someone to make
        changes to a single set file that they own even if permissions are
        set so that they cannot read other set files. The commit method is
        available, but the cache method is disabled. The file option
        requires that the set option also be present.

        With the files option, all set files are read. Both the commit and
        cache methods are enabled.

    set
          set => SET

        This defines which set to read when the read = file> option is used.
        This option is required when read = file> and ignored for any other
        value for read.

    types
          types => TYPE
          types => [ TYPE1, TYPE2, ... ]

        Sets can be of one or more types (or they can belong to no type and
        be used solely in building other sets using the INCLUDE or EXCLUDE
        tags described in the FILE FORMAT section below).

        This option can be used to specify the names of the different types
        of sets defined by these files.

        If this option is not given, then there is only one type and by
        default, all sets belong to it.

    default_types
          default_types => [ TYPEa, TYPEb, ... ]
          default_types => "all"
          default_types => "none"
          default_typew => TYPE

        Some types of sets may be more common than others, and you may or
        may not want to have to explicitely define which types a set belong
        to.

        If a list of types are passed in, every type must be defined in the
        types option (warnings will be issued if they weren't). If a value
        of "all" is passed in, sets belong to all types by default. If a
        value of "none" is passed in, sets don't belong to any type by
        default.

        By default, sets belong to all types available.

    comment
          comment => REGEXP

        This defines a regular expression used to recognize (and strip out)
        comments from a set file. The default expression is "#.*" which
        means that all characters from a pound sign to the end of the line
        are removed.

        If REGEXP is passed in as an empty string, there are no comments.
        All lines are either empty or contain an element.

    tagchars
          tagchars => STRING

        This defines a character (or a string) which marks a line of the set
        file as containing a tag. The default value is "@".

    valid_ele
          valid_ele => REGEXP
          valid_ele => !REGEXP
          valid_ele => \&FUNCTION

        By default, every non-blank line (after comments have been stripped
        out) is treated as an element. If this option is used, elements are
        tested, and only those that pass the test are treated as valid.
        Others are invalid and produce a warning.

        If a reference to a function is passed in, the function
        &FUNCTION(set,ele) will be evaluated for each element. If it returns
        0, the element will be silently ignored. Otherwise it will be
        included in the set.

    scratch
          scratch => DIR

        When automatically updating a set file, the directory where the
        files live may or may not be writable by a user who owns a set file.

        If the directory is writable by the user, there is no problem. In
        this case, when a new set file is written, the old one is backed up
        and the new one written in it's place.

        If the directory is NOT writable by the user, the old copy is backed
        up to the scratch directory. This directory must be writable by the
        user. It defaults to /tmp.

FILE FORMAT
    A set file has a very simple format. It consists of blank lines, tags,
    and elements. Comments may be included as whole lines or part of one of
    the above lines.

    Each line is checked for comments and they are removed before any other
    processing is done. A comment is anything that matches a regular
    expression which can be set using the comment Init option. The default
    regular expression is "#.*" which means that comments start with a pound
    sign anywhere on the line and go to the end of the line.

    Tags are lines which begin with begin with a special string (which can
    be set with the tagchars Init option. The default string is "@". Tag
    lines are of one of the formats:

      @TAG
      @TAG VAL1,VAL2,...

    All other lines are elements. Elements are any string (one per line).

    Leading/trailing spaces are ignored in all cases.

    The set name is the name of the set file.

    The following TAGs are known:

    INCLUDE SET1,SET2,...
        This includes all members of one or more other sets in the current
        set.

    EXCLUDE SET1,SET2,...
        This excludes all members of one or more other sets from the current
        set. This overrides any members included from other sets, but does
        NOT exclude members explicitely included in the set file.

    OMIT ELE
        This exludes a specific element from the current set. This overrides
        any elements included via. an INCLUDE tag, or any elements
        explicitly included in the set file.

        Each element must be specified separately since there is no
        guarantee that elements may not contain commas.

    TYPE TYPE1,TYPE2,...
        The default types that this set belongs to are determined by the
        types and default_types Init options.

        This tag explicitely puts this set if the specified types, even if
        it is not in those types of default.

    NOTYPE TYPE1,TYPE2,....
        Similar to the TYPE tag, but this tag explicitely removes the set
        from the specified types, even if it is in them by default.

    OPTION VARIABLE [= VALUE]
        Although there is no support for element specific attributes, there
        IS support for attributes which apply to the entire set (and which
        can be made available to applications using these sets).

        Each set may have a hash associated with with key/value pairs (if no
        value is include, it defaults to 1). These attributes are available
        using the info method.

    All tag lines can be repeated any number of times, so:

      @INCLUDE foo,bar

    is equivalent to

      @INCLUDE foo
      @INCLUDE bar

    All tags are case insensitive.

    When determining the members of a set which includes and excludes other
    sets, or omits specific elements from the set, all inclusions are
    evaluted first, followed by all exclusions (i.e. all exclusions override
    all inclusions). If there is a cyclic dependancy (i.e. A depends on B
    depends on A where a dependancy can either be an INCLUDE or EXCLUDE), an
    error is reported and the cyclic dependancy is ignored.

    A few examples illustrate the use of INCLUDE, EXCLUDE, and OMIT tags. In
    the examples, the set file A contains the elements: E1, E2, E3. The set
    file B contains the elements: E3, E4, E5. The set file contains the
    following lines:

      @INCLUDE A
      @EXCLUDE B
      E5
      E6

    defines a set contains the elements: E1, E2, E5, E6. The first line
    includes E1, E2, E3. The second line excludes E3. It does NOT exclude E5
    since the EXCLUDE tag does not override elements explicitly included in
    the set file. Finally, the E5 and E6 elements are added.

    The set file containing the following lines:

      @INCLUDE A
      @EXCLUDE B
      @OMIT    E2
      @OMIT    E6
      E5
      E6

    defines a set contains the elements: E1, E5. This is similar to the
    above example, except that the OMIT tags override elements included via.
    the INCLUDE tag AND elements explicitly included in the set file.

FILES
    Several files are used by the Set::Files module. They all live in the
    directory set by the cache Init Option except for set specific files
    which live in the same directory as the set file. Files are:

    .set_files.SET
        A backup of the given set. When a set file is updated, the original
        file is stored in this file. The file is stored either in the same
        directory as the set file (if it is writable) or in the directory
        specified by the scratch Init Option.

    .set_files.SET.new
        A temporary file where a new set file (or the update to an old one)
        is written. Once completed, this file is moved into place as the new
        set file. This file lives in the same directory as the set file or
        in the scratch directory.

    .set_files.cache
        The file containing the cache. This is created using the cache
        method.

    .set_files.template
        When creating a new set file (or updating an existing one), this
        file is used (if it exists) as a starting point and then all the
        data is appended to it. This is a good place to store comments
        describin how to edit the set files, etc., that set file maintainers
        can read for help.

KNOWN PROBLEMS
    None at this point.

AUTHOR
    Sullivan Beck (sbeck@cpan.org)

