Owl Monitoring System -- Manager Installation Manual -- Installation

|  | ## Owl Monitoring System ## Manager Installation Manual

### 2. Owl Manager Installation

| 2.1. |  | Manager Installation
| 2.1.1. |  | Preparatory Steps
| 2.1.2. |  | Installing Third-Party Software
| 2.1.2.1. |  | Nagios Core and Plugins
| 2.1.2.2. |  | nagiosgraph
| 2.1.2.3. |  | rrdtool
| 2.1.2.4. |  | drraw.cgi
| 2.1.2.5. |  | Perl modules
| 2.1.3. |  | Installing the Owl Monitor Software
| 2.1.3.1. |  | Unpacking the Owl Monitor Software
| 2.1.3.2. |  | owl-dnswatch
| 2.1.3.3. |  | owl-perfdata
| 2.1.3.4. |  | owl-newsensor
| 2.1.3.5. |  | owl-transfer-mgr
| 2.1.3.6. |  | Data-Archiving Programs
| 2.2. |  | Modifying Configuration Files
| 2.2.1. |  | Save Files!
| 2.2.2. |  | nagios.cfg (Nagios)
| 2.2.3. |  | cgi.cfg (Nagios)
| 2.2.4. |  | resource.cfg (Nagios)
| 2.2.5. |  | share/config.inc.php (Nagios)
| 2.2.6. |  | nagiosgraph.conf (nagiosgraph)
| 2.2.7. |  | map (nagiosgraph)
| 2.2.8. |  | htpasswd.users (Apache)

This section gives installation procedures for setting up the Owl Monitoring software on manager hosts. There are two sections, an installation section and a configuration section. The installation instructions discuss software installation of Owl software and supporting third-party software packages. System configuration is also detailed in this section, but only for non-Owl-specific configuration. There are a number of Owl-specific configuration actions that must be performed and these are detailed in section 4.

The Owl manager requires Perl. You must install it on your system if it isn't already available.

### 2.1. Manager Installation

The basic Owl manager configuration consists of the Owl software and the Nagios monitoring system. This configuration allows for monitoring of current sensor status, but does not provide any view into past monitoring data. In order to get the historical view, especially as provided by a graphing capability, several additional software packages must be installed on the manager. Since the historical view provides a great deal of useful information, these instructions will assume that you will be installing the additional software.

It is possible (and not too difficult) to run multiple instances of the third-party software packages (e.g., Nagios, nagiosgraph) on a single host, but care must be taken when configuring, building, and installing the software. Without careful work, an existing installation may be overwritten by a later installation. These instructions assume that only one instance of each package will be installed on the manager host.

Below is a diagram of the flow of Owl sensor data on the Owl manager. The details may not make sense at this stage, but this will show how the various pieces fit together.

### 2.1.1. Preparatory Steps

Several actions must be taken on the manager host to prepare it for installing the required software. These steps are listed below. During the installation procedures, don't be shy about backing up things; it might not be necessary, but better safe than sorry.

- It is advisable to run a backup of your system before starting. Depending on your level of paranoia, it couldn't hurt to run an incremental backup after completing each step of the installation.

- An unprivileged user account must be created for the Owl manager software. An existing account may be used, but it would probably be best to use a new account as this will segregate files and processes from existing users. The user's name and group are not required to be any particular values.

  You may want to create two unprivileged users for use by the Owl manager. One would run the various Owl commands, and the other would be used for sensor data transfers (in the case that the sensor is to initiate transfers.) The first user will need rwx permission to all the Owl directories and files. The second user will need to be able to write data into the Owl data directories.

  In the following instructions, it is assumed that a single user will created for the Owl manager and it will be referred to as the Owl User.

- SSH utilities and a web server must be installed on the manager host. These instructions assume that the OpenSSH and the Apache web server will be used. Others may be used instead, but the you must determine the appropriate commands and options to use. Apache support is included in Nagios.

- Install the rrsync utility. This is a front-end to rsync that restricts the locations to which rsync may copy data. This tool is available here: http://ftp.samba.org/pub/unpacked/rsync/support/rrsync

- Create the directories required for each sensor. There must be a data repository and a data archive. Each sensor will have their own subdirectory in the repository and in the archive. There must be a data and a history directory within each sensor's repository directory.

  The following is a possible set of directories:

  | /owl/data | top-level data repository
  | /owl/old.data | top-level data archive
  | /owl/data/sensor-portland/dnstimer | sensor-portland's DNS timer-data directory
  | /owl/data/sensor-portland/history | sensor-portland's monitoring history (for Nagios)
  | /owl/old.data/sensor-portland | sensor-portland's data archive
  | /owl/data/sensor-oslo/dnstimer | sensor-oslo's DNS timer-data directory
  | /owl/data/sensor-oslo/history | sensor-oslo's monitoring history (for Nagios)
  | /owl/old.data/sensor-oslo | sensor-oslo's data archive

- Create an SSH key for the Owl user. This is commonly done with the ssh-keygen command.

- Consider the users you want to access your Owl monitor, as well as the types of access they should be allowed. These users will be added to the Owl-specific password file used by Apache and the users and access rights will be added to a Nagios configuration file. These users are web-based users only, and are not (necessarily) related to users on the Owl manager host.

- Install the software as described in the following sections. The Owl software will be installed after all the third-party software, since some of the Owl software will be installed in directories created by the third-party software.

### 2.1.2. Installing Third-Party Software

The third-party software packages should be built and installed as specified by each package. Owl-specific notes for each will be given below. While many of these packages are available as pre-built distributions, configuring and building them yourself will provide for additional control that may be useful. This includes the installation location and the user and group that will be used for executing the commands.

Nagios, along with a web server, should be installed such that they will be restarted at system boot time.

This table shows the most recent software versions that were tested with the Owl Monitor.

| Package | Version Tested with Owl
| Nagios Core | 3.4.1
| Nagios Plugins | 1.4.16
| nagiosgraph | 1.4.4
| rrdtool | 1.4.7
| drraw.cgi | 2.2b2

### 2.1.2.1. Nagios Core and Plugins

If building Nagios Core and Nagios Plugins from source, it is recommended that the configure command be given arguments that will allow specification of the installation-path prefix, the Nagios user, and the Nagios group. A possible command line could be:

        $ ./configure --with-command-group --prefix=/owl --with-nagios-user=owl-monitor --with-nagios-group=owl-monitor

The same prefix, username, and groupname must be used when building Nagios Core and Nagios Plugins.

Multiple instances of Nagios can be installed on a single machine. This can be useful for segregating different things being monitored. However, you must be careful when doing this. In particular, even if the --prefix option was given to configure, the "make install-init" and "make install-webconf" commands will overwrite existing Nagios files.

### 2.1.2.2. nagiosgraph

The README file for nagiosgraph contains instructions for automated installation (using install.pl) and for manual installation. You should read this file and decide which method you want to use.

If you choose the manual installation, you may skip most of this section and pick up at the discussion of the map file.

If you choose to install with install.pl, you must know that it looks like several of the steps are not performed by the script. After running install.pl you must verify that the manual steps from "Restart nagios" to the end have actually been taken.

install.pl will configure the package, install its files in the appropriate places, and modify Nagios and httpd files as needed. A variety of options allow for site-specific installation layouts. In addition, you will be prompted for many directory and file names prior to install.pl completing its work. If it can't find one or another file that it must update, then at the end of execution it will provide a set of steps that you must manually perform in order to complete the installation.

The --dry-run option will show you what install.pl would do, so that you may verify where things are being copied. It isn't required, but it wouldn't be a bad idea to use this option prior to the actual install.

One thing install.pl is likely to say is that you must add a Nagios command file. This file probably already exists, and you must find it. It is likely to be called something like commands.cfg and be located in the etc/objects directory in your Nagios installation. Check names carefully when making this modification, as commands.cfg already has a process-service-perfdata command and install.pl will have you add a process-service-perfdata-by-nagiosgraph command.

install.pl logs all its actions, including the manual-intervention actions, in the install-log file. Unfortunately, the command line is not recorded in the log file, so it might be useful to save that somewhere yourself.

I have used this command to successfully install nagiosgraph:

        $ install.pl --prefix /owl/nagiosgraph --nagios-user owl-manager --nagios-cgi-url /nagios-owl/cgi-bin \
        --nagios-perfdata-file /owl/var/service-perfdata.out --log-dir /owl/var --layout standalone

I do not let install.pl update the Nagios or Apache configurations, which it will prompt for; I prefer to make those modifications myself. Refer to the install-log file to see what remains to be done.

The README file contains steps for manual installation and it looks like several of the steps are not performed by install.pl. If you use install.pl to install nagiosgraph, verify that the manual steps from "Restart nagios" to the end have actually been taken.

You may have to manually copy the nagiosgraph.ssi file into place. This file is in .../nagiosgraph/examples/nagiosgraph.ssi and it should be copied into your equivalent of /owl/share/nagiosgraph.ssi.

Before nagiosgraph can be used by the Owl manager, the map file must be modified to recognize data from the Owl plugins for Nagios. The required modifications are discussed in section 2.2.7.

### 2.1.2.3. rrdtool

No Owl-specific configuration or actions were required for building and installing rrdtool.

### 2.1.2.4. drraw.cgi

To install drraw.cgi, copy the drraw.cgi and the drraw.conf files into the Nagios sbin/ directory. So, if you have installed Nagios in /owl/nagios, then you should copy the two drraw files into /owl/nagios/sbin.

The drraw.conf must be edited for this installation. Some of these edits are required and some are optional. Each will be marked as to whether or not you must make the change. This file is a Perl file, so the configuration values are set according to Perl's language rules.

- Set the $title variable if you don't want the default page title in the Nagios-displayed graphing pages.
  (optional)

- Set the %datadirs variable to hold the paths to your sensors' RRD databases. There are two fields required for each entry. The first is the path to that sensor's database files, and should not have a terminating slash. The second is a name by which those databases will be identified when defining new graphs.

          '/owl/nagiosgraph/var/rrd/sensor1%20h-root%20foo.com%20A'  => '[Kyoto H foo A]',
          '/owl/nagiosgraph/var/rrd/sensor1%20h-root%20foo.com%20NS' => '[Kyoto H foo NS]',
          '/owl/nagiosgraph/var/rrd/sensor5%20h-root%20foo.com%20A'  => '[Cairo H foo A]',

  The identifier should be unique for each database directory.
  (required)

- Set the vrefresh variable for the number of seconds the graph viewer will wait before automatically refreshing its display.
  (optional)

- Set the drefresh variable for the number of seconds the dashboard viewer will wait before automatically refreshing its display.
  (optional)

- Set the @dv_def, @dv_name, and @dv_secs array variables for the graphs that will be generated each time a predefined graph is displayed. These are time based values and a particular element of one array is directly related to that same element in the other arrays.

  The @dv_def array is used to select data from the database. The @dv_name array is used to provide titles for each graph. The @dv_secs array is used to provide the count of seconds for each graph.

  The following example values show the relation between the three arrays.

          @dv_def  = ( 'end - 15 minutes', 'end - 60 minutes', 'end - 4 hours',
      		 'end - 12 hours',   'end - 24 hours',   'end - 1 week' ,
      		 'end - 1 month',    'end - 1 year');

          @dv_name = (
      	'Past 15 Minutes',
      	'Past 60 Minutes',
      	'Past 4 Hours',
      	'Past 12 hours',
      	'Past 24 Hours',
      	'Past Week',
      	'Past Month',
      	'Past Year'
          );

          @dv_secs = ( 900, 3600, 14400, 43200, 86400, 604800, 2419200, 31536000 );

  (optional)

- The $saved_dir variable defines where definitions for your predefined graphs will be stored. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:

          $saved_dir = '/owl/nagios/var/drraw/saved';

  (required)

- The $tmp_dir variable defines where cached graphing file will be stored until they time out and are deleted. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:

          $tmp_dir = '/owl/nagios/var/drraw/tmp';

  (required)

- The $ERRLOG variable defines where graphing errors will be logged. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:

          $ERRLOG = '/owl/nagios/var/drraw/errors.log';

  (required)

### 2.1.2.5. Perl modules

There is a set of Perl modules required by the Owl manager and the third-party software packages. Some of the software packages may install these packages themselves, but you must ensure that these are available. These should be available through CPAN.

Modules required by the Owl manager:

- Cwd.pm
- Fcntl.pm
- File::Path.pm
- Getopt::Long.pm
- Time::Local.pm

It is likely that some of the third-party software may require other Perl modules. The installation documentation or process should inform you of what must be installed on your system.

### 2.1.3. Installing the Owl Monitor Software

There is a small amount of manager-specific software in the Owl Monitor. Most of the work on the manager is performed by Nagios and the other third-party software. The Owl Monitor software on the manager host consists of several Nagios plugins that will interpret Owl sensor data for Nagios. There is also a small number of administrator programs for managing the large amounts of Owl sensor data.

### 2.1.3.1. Unpacking the Owl Monitor Software

The Owl Monitor software comes in a bzip2'd tar file. This contains Owl administrative commands and Owl plugins for Nagios. In most modern Unix systems, this can be unpacked with this command:

            $ tar xf owl-monitor.tar.gz

This command unpacks the Owl distribution in the current directory. This can be unpacked anywhere you want, but it is a good idea to do it in the Owl manager's home directory.

The Owl commands should be installed in a convenient directory, such as ~/owl-manager/bin. This could be anything that is suited for you; e.g., /usr/bin, /usr/local/bin, ~/mystuff, or /owl/scripts. It would be helpful to add the directory to your shell's path.

The Owl plugins must be installed in the Nagios plugin directory. This directory depends on how you installed Nagios, and it is probably something like /owl/nagios/libexec.

### 2.1.3.2. owl-dnswatch (plugin)

The owl-dnswatch command is a Nagios plugin that reads Owl monitoring data and reports the latest results to Nagios. Each execution of owl-dnswatch requires an Owl sensor name, a target host, a DNS server, and a DNS query type. The appropriate monitor datafile is consulted based on those inputs. The following actions must be taken to install owl-dnswatch.

- Set the $OWLDATA value in owl-dnswatch to the top-level directory that holds the Owl monitoring data. This path will depend on your installation hierarchy, but it will likely look something like:

              my $OWLDATA = "/owl/data";

### 2.1.3.3. owl-perfdata (plugin)

The owl-perfdata command is a Nagios plugin that acts as a front-end for inserting Owl monitoring data into an RRD database. These databases are used for graphing the monitoring data. The following actions must be taken to install owl-perfdata.

- Set the $GRAPHINSERT value in owl-perfdata to the path to the insert.pl command. This path will depend on where you have installed nagiosgraph, but it will likely look something like:

              my $GRAPHINSERT = "/owl/nagiosgraph/bin/insert.pl";

### 2.1.3.4. owl-newsensor (command)

There are a set of Nagios objects that must be created for each Owl sensor. Once the Owl manager has sensor data files in its data directory, the owl-newsensor program can be used to create these objects. The actions required for this are described in section 4.7.

No local modifications are required for owl-newsensor.

### 2.1.3.5. owl-transfer-mgr (command)

If the Owl manager transfers data from a sensor, it will use the owl-transfer-mgr to retrieve the data.

No local modifications are required for owl-newsensor.

### 2.1.3.6. Data-Archiving Programs

The following programs assist the administrator with managing the large amount of Owl monitoring data that accumulates on the Owl manager host. These programs work together to create a two-stage archival process. It is helpful to have cron run these programs, though this is not essential.

The first stage of the archive process moves old data from the sensor's data directory to the sensor's archive directory. Data is considered old if the file's timestamp (in its name) is older than two days. This part of the data archiving should be performed once per day, depending on how frequently the sensors perform their own data archiving.

The second stage of the archive process combines a month's worth of sensor data into a single compressed tarfile. This should be performed several days after the end of the previous month.

The Owl data-archival programs assume that all sensor data are stored in a common directory and all the archived data are stored in a common directory. They will segregate data by sensor in order to prevent it from getting mixed together. For example, when handling data for the sensors dresden and kvothe, these programs might expect the following directories to exist:

        /owl/data/dresden
        /owl/data/kvothe
        /owl/data-archive/dresden
        /owl/data-archive/kvothe

This first set of programs will archive data for all the sensors that report to an Owl manager. They must be told where the sensor data are stored and where the data archive is kept.

- owl-archdata - Archives data for all sensors without having to name them explicitly. The sensors in the given data directory will be archived. The data are stored in year-month subdirectories in the sensor's archive. owl-archdata is used in the first stage of the archival process.

- owl-monthly - Archives all sensors' archived data in a compressed tarfile, in order to minimize space taken by the archive. The tarfiles are stored in another subdirectory, divided by year. owl-monthly is used in the second stage of the archival process.

This second set of programs will archive data for a single sensor that reports to an Owl manager. They must be told where the sensor data are stored and where the data archive is kept.

- owl-dataarch-mgr - Archives data for a single sensor. This is intended to be run by owl-archdata, and it is a slightly modified version of the Owl sensor's owl-dataarch utility.

- owl-archold - Archives a sensor's archive data in a compressed tarfile. This is intended to be run by owl-monthly. The Owl manager's version of owl-archold is exactly same as the Owl sensor's utility of the same name.

More information about each of these programs is available in their man pages.

Copy these archiving programs to a directory which will be their home. This could be anything that is convenient for you; e.g., /usr/bin, /usr/local/bin, ~/mystuff, or /owl/scripts.

### 2.2. Modifying Configuration Files

Several files must be modified in order to provide Nagios monitoring and graphing. These files belong to Nagios, Apache, and nagiosgraph. This section describes the required modifications. These modifications do not provide any monitoring of Owl sensor data; further steps to supply that functionality will be given later.

In the basic Nagios installation, /nagios/etc holds sample configuration files, which can be modified for use.

### 2.2.1. Save Files!

Before starting to modify any existing file -- e.g., Nagios object files and Apache configuration files -- it would be a very good idea to make a back-up copy of the files. This will provide reference copies of the files in case they might be needed later.

### 2.2.2. nagios.cfg (Nagios)

This file is the primary configuration file for Nagios. There are quite a few changes that must be made to this file. The modifications listed in this section will not incorporate monitoring of Owl sensor data. That will be described in section 4.8.1.

The various configuration parameters for Nagios could be included in a small number of files, but the standard arrangement groups the parameters into separate files. The nagios.cfg file is the primary configuration file for Nagios, and at run-time it acts as the focal point to collect the disparate configuration files for the Nagios daemon.

A cfg_file line is used to tell Nagios to include the specified line. Adding or deleting these lines will control which files are included in the Nagios configuration.

As a result of the Nagios build and installation process, the nagios.cfg file should be mostly configured already. The Nagios configuration file is well-documented, so figuring out how to adjust its fields shouldn't be too difficult. There are a few things that must be done to prepare it for Owl monitoring. These directions assume that Owl monitoring is the only monitoring that will be performed by this installation.

- Make sure these cfg_file lines are deleted or commented out:
  - localhost.cfg
  - printer.cfg
  - switch.cfg
  - windows.cfg
  (This is not strictly necessary. These files may provide some monitoring information that you would find useful.)

To allow for graphing of monitoring results, several performance-processing fields must be set in nagios.cfg. Unless otherwise stated, do not include the double quotes when setting these fields.

- Find the process_performance_data line and change it to: "process_performance_data=1"

- Find the service_perfdata_file line and change it to: "service_perfdata_file=/owl/var/service-perfdata.out"

- Add this line: "service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$ ||$SERVICEOUTPUT$||$SERVICEPERFDATA$"
  (This line was split to accomodate document formatting. Add it as a single, unbroken line without any spaces or newlines.)

- Find the service_perfdata_file_mode line and change it to: "service_perfdata_file_mode=a"

- Add this line: "service_perfdata_file_processing_interval=30"

### 2.2.3. cgi.cfg (Nagios)

This is the configuration file for CGI scripts run by Nagios. Most of it is set up during the Nagios configuration process. The following things must be set to ensure your installation's needs are met.

- The url_html_path field contains the path portion of the URL for the Nagios monitor. For example, if the desired URL to reach your Nagios monitor is http://www.example.com/owl-dns-monitor, then url_html_path must be set to /owl-dns-monitor. Check this field to ensure it is set as required.

- The cgi.cfg file allows you to specify different types of information and actions that can be taken by different users. The authorized_for_ values may be set for multiple users. The user names are those in the htpasswd.users file.

### 2.2.4. resource.cfg (Nagios)

Ensure USER1 points to the plugins directory. This should be something like /owl/libexec.

### 2.2.5. share/config.inc.php (Nagios)

Nagios uses this file to hold a small amount of file-location data. The following actions must be taken:

- Ensure that has the correct path for the $cfg[] fields. If your configuration doesn't use the default paths, this may not be set properly.

- Add an entry for the $cfg['ng_cgi_base_url'] field to be set to the path of the nagiosgraph CGI directory. This will depend on how you installed nagiosgraph, and should look something like this:

          $cfg['ng_cgi_base_url']='/nagiosgraph/cgi-bin';
          $cfg['ng_cgi_base_url']='/nagiosgraph-owl/cgi-bin';

### 2.2.6. nagiosgraph.conf (nagiosgraph)

This is the configuration file for Nagiosgraph. Several settings must be checked and some Owl-specific setting copied in. The following actions must be taken:

- Copy the contents of ng-owl-conf after the lines setting xffs. These are reasonable values, but don't change them unless you know what you're doing.

- Ensure that nagiosgraphcgiurl path points to the appropriate directory, as defined by your web server's configuration.

- Ensure that javascript path points to the appropriate directory, as defined by your web server's configuration.

- Ensure that stylesheet path points to the appropriate directory, as defined by your web server's configuration.

### 2.2.7. map (nagiosgraph)

nagiosgraph uses a file of data definitions that describe how to extract data from Nagios plugins and how to handle the extracted data. The file is a collection of Perl expressions that are evaluated by nagiosgraph. This file is named something like /owl/nagiosgraph/etc/map.

The Owl manager distribution contains a set of additional, Owl-specific definitions that must be added to the map file. These definitions are in ng-owl-map. Copy the contents of this file into the map.

### 2.2.8. htpasswd.users (Apache)

Strictly speaking, this file is not a Nagios file, but is instead an httpd file. The Nagios configuration file for httpd gives the name of an htpasswd.users file. Use the htpasswd command to create and populate this file with the users that should be allowed to access the Owl monitor. These users must have their permissions defined in the Nagios cgi.cfg file.

Proceed to Section 3. An Interlude on Sensor Queries for important information about defining the queries gathered by your Owl manager.
If you have already read Section 3, you may continue on with Section 4. Adding Sensors to complete the installation of the manager.

Section 1. Operational Overview | Owl Monitoring System Manager Installation Manual | Section 3. An Interlude on Sensor Queries

DNSSEC Tools