Owl Monitoring System -- Full Installation Manual -- Setting Up the Owl Monitoring System

|  | ## Owl Monitoring System ## Full Installation Manual

### Setting Up the Owl Monitoring System

The Owl Monitoring System uses timed Domain Name System (DNS) queries to monitor basic network functionality. The system consists of a manager host and a set of sensor hosts. The Owl sensors perform periodic DNS queries and report to the Owl manager the time taken for each query. Over time, this shows the responsiveness of the DNS infrastructure.

The Owl Monitoring System was designed such that the Owl sensor hosts need not be under the same administrative control as the Owl manager host. In fact, each sensor in a set of Owl sensors may be under administrative control of different organizations and still report to a single Owl manager.

An Owl sensor may provide its data to more than one Owl manager. In such situations, the managers operate independently of each other and do not know of each other's existence. Throughout the installation manuals, it will be assumed that an installation will have a single manager. Special instructions for multiple-manager environments will be given where needed.

Contact between sensors and their manager may be initiated and performed by the sensor or the manager. This is a configuration decision that must be made on a case by case basis. The push or pull model may cover all of an Owl manager's sensors. For example, a particular manager may initiate sensor data retrieval from all of its sensors. Also, the push/pull model may be specific to each sensor, so a particular manager could retrieve data from one sensor but wait for another sensor to provide its data.

This document provides an operational overview of the Owl system, installation instructions for the Owl and supporting third-party software. Configuration instructions for the Owl software are also provided, along with configuration information for the required third-party software.

### How To Use This Manual

There are two installation manuals, one for the manager and one for the sensor. Sensor administrators only need to read the Owl Sensor Installation Manual. Manager administrators only need to read the Owl Manager Installation Manual As you might expect, those who are administrators for both manager and sensor hosts must read both installation guides.

| Owl Manager Installation Manual |  | Owl Sensor Installation Manual |

## Sensor Installation Manual

### 0. Setting Up Sensors for the Owl Monitoring System

The Owl Monitoring System uses timed Domain Name System (DNS) queries to monitor basic network functionality. The system consists of a manager host and a set of sensor hosts. The Owl sensors perform periodic DNS queries and report to the Owl manager the time taken for each query. Over time, this shows the responsiveness of the DNS infrastructure.

An Owl sensor is configured to send DNS queries to a set of DNS nameservers. The queries may be for a variety of DNS resource record types, and they are sent to a specific nameserver to request data about a specific target host. For example, one query might request that the D root nameserver return NS records for the "www.example.com" host, while another query might request that a local (non-root) nameserver return AAAA records for the "mail.example.com" host. Data will be collected for each configured query and given to the Owl manager to whom the sensor reports. The different sensors in the sensor set may perform different queries or the same queries.

A configuration file controls the behavior of the Owl sensor. This file defines the queries that the sensor will perform, the Owl manager (or managers) to whom the sensor reports, locations of data and log files, frequency of queries, and other important behaviors.

An Owl sensor may provide its data to more than one Owl manager. In such situations, the managers operate independently of each other and do not know of each other's existence. Throughout the installation manuals, it will be assumed that an installation will have a single manager. Special instructions for multiple-manager environments will be given where needed.

This document provides an operational overview of the Owl system, installation instructions for the Owl sensor and the third-party software that supports it. Instructions for configuring the Owl sensor are also given.

### Organization of Installation Manual

Sections:

| 0. |  | Setting Up Sensors for the Owl Monitoring System
| 1. |  | Operational Overview of the Owl Monitoring System
| 2. |  | Owl Sensor Installation
| 3. |  | An Interlude on Sensor Queries
| 4. |  | Adding Owl Sensors to an Existing Owl Installation
| 5. |  | Adding New Queries to Existing Owl Sensors
| 6. |  | Owl Sensor Commands

### 1. Operational Overview of the Owl Monitoring System

The Owl Monitoring System has a conceptually simple method of operations. There is a manager host and a set of sensor hosts. The sensors periodically make a set of DNS queries and time how long it takes to get a response. The response times are saved to data files. After a certain amount of time, the data files are transferred from the sensor to the manager host. For any particular sensor, this data transfer may be configured to be initiated by either the manager or by the sensor. The manager transfers the data into a monitoring package, which allows easy display of the data.

Below is a diagram of the flow of Owl sensor data through two Owl monitoring installations.

Take note of the following:

- There are three distinct administrative domains, two of which have Owl managers.
- Owl Manager 1 receives sensor data from all three domains.
- Owl Manager 2 receives sensor data from itself and one other domain.
- Domain 1's sensors do not provide data to other domains.
- Domain 2 contains an Owl sensor, but not an Owl manager.
- Owl sensors in the domains 2 and 3 provide data to Owl managers outside of their own domain.
- Owl sensors can query multiple name servers.

(Domains in this example are defining administrative control and are not DNS domains.)

The remainder of this section provides a little more information on the components that make up the sensor and manager software.

### 1.1. Owl Sensor Overview

An Owl sensor makes timed DNS queries and records the time required for a response. The response time data are transferred to the Owl manager after a certain period of time. The transfer may occur at the instigation of either the manager or the sensor. The Owl sensor's three primary programs that handle these tasks are described below.

The Owl sensor's actions are controlled by a configuration file. This file defines the queries that must be performed, how often the queries must happen, how frequently the response data files are transferred to the manager, data and logging information, and other required information.

The owl-dnstimer program performs periodic DNS lookups, as specified by the configuration file. These lookups are timed so that the sensor can record how long it took to make a particular request. The configuration parameters used by owl-dnstimer include the nameserver to query, the target zone to query about, the type of DNS record, and the frequency of queries.

If the Owl system is configured so that the sensor transfers its data to the manager, then the transfer is performed by the owl-transfer program. The transfers are performed using rsync over an ssh connection. The sensor host must be able to do a password-less ssh into the manager host. The frequency of transfers is defined in the configuration file. Choosing the transfer frequency is a trade-off between lower impact on the sensor, manager, and network (less frequent) and lower latency of data display on the manager (more frequent.)

The owl-sensord program is an optional controller for owl-dnstimer and owl-transfer. If used, it will run these daemons and restart them if they stop. If one of the daemons tries to restart too often in too short a time, then owl-sensord will warn an administrator (via email) and temporarily stop trying to execute the program. owl-sensord is not required in order to run owl-dnstimer and owl-transfer; the Owl sensor can run fine without it. However, it provides a convenient way to keep them running.

As part of the installation process, you must decide whether you will run the Owl sensor using the owl-sensord daemon to control execution or if you will run owl-dnstimer and owl-transfer directly.

The Owl sensor has a number of programs for administrative support. Most of these are not required, but they assist in sensor administration. In particular, the owl-dataarch and owl-heartbeat programs are intended to run as cron jobs. owl-dataarch moves "old" sensor data into an archive directory, in order to keep the sensor-manager data transfer from bogging down (within rsync) as the data collection grows. The owl-heartbeat program periodically touches a webpage hosted on the Owl manager, in order to let the manager know that the sensor host is still running. owl-heartbeat is not critical and can be left off, but owl-dataarch is essential. There are a few other administrative commands available; see section 6 for a brief description of all the Owl sensor commands.

### 1.2. Owl Manager Overview

The Owl manager's purpose is to receive sensor data and make it available for display. The majority of the Owl manager is actually third-party packages, and the Owl manager software provides the "glue" that allows the Owl sensor data to work with those packages. In particular, Owl provides data to the Nagios monitoring system, which then displays the data.

The Owl sensors may report to managers that are under separate administrative control. A particular installation of the Owl Monitoring System may use multiple manager hosts, but this document assumes that there is only one.

The manager can have either an active or passive role in the process of moving Owl sensor data from the sensor to the manager. If the manager is active, it will use the owl-transfer-mgr program to pull sensor data from the Owl sensor host. If the manager is passive, then the sensor will transfer the data when it sees fit.

The are two groups of "glue" programs that Owl uses to interface with Nagios. The owl-dnswatch and owl-perfdata inject Owl sensor DNS response data into Nagios for display and graphing. The owl-sensor-heartbeat.cgi and owl-stethoscope programs insert Owl "heartbeat" data into Nagios so it can track availability of the Owl sensors.

The Owl manager also has a number of programs for administrative support. Most of these are not required, but they assist in manager administration. In particular, the owl-archdata and owl-monthly programs are intended to run as cron jobs. owl-archdata moves "old" sensor data into an archive directory, in order to keep data transfer from the sensor from bogging down as the data collection grows. The owl-monthly program archives the previous month's sensor data into a compressed tarfile. owl-newsensor and owl-initsensor assist in setting up the manager for a new sensor. There are a few others available; see section 7 for a brief description of all the Owl manager commands.

The following third-party software packages are used in with this distribution of the Owl Monitoring System. The packages are not included with the Owl distribution, but must be retrieved as required for your operating system. These packages must be installed on the manager:

| Nagios | Monitoring system.
| Nagios plugins | Monitoring system.
| nagiosgraph | Provides graphing for Nagios.
| rrdtool | Database tool for use by nagiosgraph.
| drraw.cgi | Draws graphs for nagiosgraph of data stored by rrdtool.

Instructions are given in this document for integrating these packages with the Owl software. It would be for the best to read the installation instructions below for each package prior to installing them.

Different management, graphing, and database tools may be used instead of those listed above, if so desired. Integrating the Owl manager software with other packages is beyond the scope of this manual.

David Josephsen's Building a Monitoring Infrastructure with Nagios is very helpful for understanding Nagios and how the various pieces work together. If you're going to be running a Nagios monitor, you would be doing yourself a favor to read this book.

### 1.3. Data Retention on Owl Hosts

Owl sensors generate a large amount of DNS response data. For example, a sensor running five queries, once a minute each, generated roughly 517KB of data per day. (Due to the way the data are stored, this will be affected by the amount length of target and nameserver hostnames used.) You may or may not want to retain any of this data. The data files can be useful for rebuilding the Owl manager's rrdtool databases (or building with different parameters.)

On both the Owl manager and the Owl sensors, sensor data are stored in a data directory. After a few days, the data files are moved to a data archive directory. A full month's data files will be collapsed into a compressed format.

The Owl Monitoring System provides tools that assist with managing Owl sensor data. After the sensor data have been moved from the sensor to the manager, the sensor has no further need of the data. On the manager, after the data have been moved into the data archive, the Owl Monitoring System has no further need of the old data; it may be retained or deleted. (The data will have been moved into the manager's rrdtool databases by then, so the data is not lost.) Each installation must decide their own data retention policy.

### 2. Owl Sensor Installation

This section gives installation procedures for installing the Owl Monitoring software on sensor hosts. Installation of the Owl sensor is relatively simple. This requires the creation of a sensor account, SSH initialization, software installation, and a few other set-up activities. The most complicated aspect is likely to be coordinating SSH access with the Owl manager host. The required actions for installing an Owl sensor are described in this section.

The installation instructions discuss software installation of Owl software and supporting third-party software packages. There are a number of Owl-specific configuration actions that must be performed, and these are detailed in section 4.

The Owl sensors require Perl. You must install it on your system if it isn't already available. If the sensor host's Perl or operating system does not support threading, then each query will run in its own process, rather than all queries running in their own threads within a single process.

Below is a diagram of the flow of Owl sensor data on the Owl sensor. The details may not make sense at this stage, but this will show how the various pieces fit together.

### 2.1. Create an Owl Sensor Account

The Owl sensor will run several daemons and cron jobs. It is a good idea, but not an imperative one, for these to run as their own user. It is easier to segregate the Owl sensor's actions, processes, and files if they are performed by a dedicated user.

However, it is certainly acceptable to run them as yourself or some other existing user. If you choose to do this, you may disregard the rest of this section.

There are no requirements on the Owl username; it may be anything that makes sense to you. Some possibilities are: owl, owl-sensor, owl-user, bob, and tricorder.

To create a new user, use your operating system's user-creation command (e.g., adduser or useradd) to create a new unprivileged user.

Examples of commands to create a new user:

| Operating System |  | Example Command | Notes
| Centos |  | useradd hoots-mon | remember to add a password
| FreeBSD |  | adduser | you'll be prompted for all user information
| Mac OS X |  | "Users & Groups" in System Preferences | you'll be prompted for all user information

### 2.2. SSH Initialization

Owl sensor data are transferred to the Owl manager using the rsync and ssh commands. You must initialize the Owl sensor user's SSH environment by generating keys for the ssh command to use. This may be dependent on your operating system or the version of SSH installed.

The OpenSSH package is a widely available SSH package, and its ssh-keygen command is used to create SSH keys. This manual will assume that OpenSSH is being used.

There are no specific key parameters required by Owl. The generated key must be sufficient to allow the sensor and the manager to transfer files without having to manually enter a password.

Example of a command to generate a new SSH key:

        $ ssh-keygen -b 1024 -t dsa

If the Owl sensor will be pushing data to the manager, then the newly generated public key must be transferred to the Owl manager host. This key must be added to the list of authorized keys for the Owl manager's account. For OpenSSH, the public key will be in a file called id_rsa.pub and it will be added to the authorized_keys file on the manager host. Once the sensor's public key has been added to the Owl manager, you must ensure that the Owl sensor user may successfully transfer files to the manager without need of a password.

If the Owl manager will be pulling data from the sensor, then you must do the following things. First, install the rrsync utility. This is a front-end to rsync that restricts the locations to which rsync may copy data. rrsync is available here: http://ftp.samba.org/pub/unpacked/rsync/support/rrsync. Next, the manager's administrator must provide you with a public SSH key. This public key must be added to the authorized_keys file belonging to the Owl sensor account.

The authorized_keys entry should look something like this:

        command="/usr/local/bin/rrsync /owl/data" ssh-rsa AAAAB3Nza...A3Q== owl-manager@owl.example.com

### 2.3. Install the Owl Environment

You must select a place to install the Owl sensor's files. The files and directories may live in the Owl sensor user's home directory, a subdirectory beneath that home directory, or somewhere else entirely. For ease of finding these files later, it may be helpful to keep the Owl sensor's files all beneath a single directory.

There are a set of directories that will be used by the Owl sensor: bin, conf, and perllib. These names shouldn't be changed.

Additional modifications to the Owl sensor environment will be discussed in section 4.1.

Several installation examples are given below. For each, it is assumed the Owl sensor user is named owl-sensor.

Installation example 1: In this case, the software will be installed in the sensor user's home directory by unpacking a tar file located in /tmp/owl-sensor.tar.gz.

        $ cd ~owl-sensor
        $ tar xzf /tmp/owl-sensor.tar.gz

Installation example 2: In this case, the software will be installed in a subdirectory within the sensor user's home directory by unpacking a tar file located in /tmp/owl-sensor.tar.gz.

        $ mkdir ~owl-sensor/owl-dir
        $ cd ~owl-sensor/owl-dir
        $ tar xzf /tmp/owl-sensor.tar.gz

### 2.4. Install Required Perl Modules

There is a set of Perl modules required by the Owl sensor. These should be available through CPAN.

Modules required by the Owl sensor:

Cwd.pm Date::Format.pm FindBin.pm Getopt::Long.pm LWP::Simple.pm Log::Dispatch.pm Log::Dispatch::FileRotate.pm Net::DNS.pm |  | Net::DNS::Packet.pm Net::DNS::Resolver.pm POSIX.pm Time::HiRes.pm Time::Local.pm threads.pm threads::shared.pm

If the two threads-related modules are unavailable for your system, or if your version of Perl doesn't support threading, then the Owl sensor will run each query in its own process. In this case, these two modules are not needed.

### 3. An Interlude on Sensor Queries

Before running your Owl Monitoring System, you must consider a number of questions. How many Owl sensors will be providing data to the Owl manager? How many queries will the sensors be performing? What queries will the sensors be performing? Will each sensor be performing the same queries? Will the sensor be providing "heartbeat" information to the manager? Who will transfer the sensor data to the manager? These questions are important for the sensor administrator, but they are critical for the manager administrator.

### 3.1. Gathering and Storing Sensor Data

A query consists of three parts: a nameserver, a target host, and a query type. Responses to each query will be kept in their own datafile. The amount of data generated by each sensor will depend on the number of queries and the frequency with which the queries are performed.

The sensors will have to gather the data and more queries will result in more data that will be collected. The data will all have to be transferred from the sensor to the manager. If the manager is configured for graphing, then data from each query will be stored in its own database. These databases can be quite large, so the manager host must have a good amount of available disk space.

The administrators of an Owl sensor and the Owl manager (if they are different people) must coordinate the queries that the sensor will be performing. The manager's administrator can request that a set of queries be performed, but it is up to the sensor's administrator to actually configure the sensor for the queries. The manager is effectively at the mercy of the sensor.

Sensor data filenames contain metadata about the sensor data contained therein. Therefore, the filenames will reflect the queries that generated the files. The fields are separated by commas and have this format:

            121129.1701,seattle-sensor,example.com,a.root-servers.net,A.dns
            121129.1701,seattle-sensor,example.com,m.root-servers.net,A.dns
            121129.1701,seattle-sensor,example.com,m.root-servers.net,NS.dns
            121129.1701,portland-sensor,example.com,a.root-servers.net,A.dns

Breaking out the fields of the last example above gives this table:

| Field |  | Purpose |  | Example
| Timestamp |  | File creation timestamp; (YYMMDD.hhmm) |  | 121129.1701
| Sensor Name |  | Name of sensor that recorded data |  | portland-sensor
| Target Name |  | Host whose name was target of DNS lookups |  | example.com
| Nameserver Name |  | Name of queried nameserver |  | a.root-servers.net
| Query Type |  | Type of DNS query |  | A
| File suffix |  | File suffix |  | .dns

### 3.2. Transferring Sensor Data

The Owl Monitoring System provides utilities to transfer sensor data from the sensor to the manager. One utility, owl-transfer runs on the sensor and transfers the data to the manager. The other utility, owl-transfer-mgr runs on the manager and transfers the data from the sensor. Both utilities are front-ends for the rsync program and run using an ssh-initiated connection.

In both cases, the transferring host must be able to ssh into the remote host without a password. The FOSS rrsync ("restricted rsync", written by Joe Smith, modified by Wayne Davison) is used to restrict the transferring host's access to the remote host.

The administrators of the sensor and the manager must agree on which host will be transferring data to the manager. The appropriate SSH public keys from the transferring host must be placed on the other host, so that the password-less ssh (as described above) may be performed.

An Owl system with multiple sensors may have a mixed transfer configuration. In such a system, some sensors will transfer their data to the manager, while the manager will transfer data from other sensors.

### 3.3. Sensor Heartbeats

The "heartbeat" facility allows an Owl sensor to periodically contact a manager to let it know that the sensor is still alive and network accessible. The heartbeat is sent by contacting a particular webpage on the Owl manager.

The heartbeat facility is not necessary for Owl operation and Owl runs perfectly well without it. However, it can provide a useful assurance to the manager administrator that the sensor is still working. Enabling this facility on the sensor is at the discretion of the sensor administrator. Since this requires a webserver to be running on the manager host, that aspect is at the discretion of the manager administrator.

### 4. Adding Sensors

Whenever a new Owl sensor is added to those handled by an Owl manager, there are a number of actions that must take place on both the sensor and the manager. These actions should be followed consecutively within each section. However, there is some amount of necessary interleaving of sensor actions and manager actions. For example, a particular sensor action may be required before a particular manager action.

It is acceptable for the Owl sensor and Owl manager to be under completely different administrative control. Each host may even be owned by different commercial, educational, governmental, or other such entity. All that is required is that there be some initial coordination between administrators when the sensor is first configured, along with (probably) aperiodic support from time to time later.

The discussions on adding Owl sensors assumes that the sensor and manager are under different administrative control. Required actions will not change if they are under unified administrative control, but they may be easier to accomplish.

This section describes the configuration and testing actions that must be undertaken in order to have a working Owl sensor. It assumes the installation procedures detailed in section 2.1 have been completed.

### 4.1. Firewall Configuration

The Owl sensor and the Owl manager communicate by way of SSH and HTTP. If the Owl sensor is protected by a firewall (either on the sensor host itself or by an enterprise-level firewall) then the firewall must be configured to allow data to be transferred between the sensor and the manager. The direction of this transfer (initiated by sensor or initiated by manager) depends on the Owl configuration. These firewall modifications are far beyond the scope of this document.

### 4.2. SSH Set-up

You must generate a new SSH key for the sensor's Owl user.

If the sensor will be transferring data to the manager, then you must provide the public key to the administrator of the Owl manager. This key will allow owl-transfer to pass Owl sensor data to the manager. You must generate keys with key characteristics (e.g., length and type) that are acceptable to the manager's administrator.

If the manager will be pulling data from the sensor, then the manager's administrator must provide you with the manager host's public key. You must add this key to your authorized_keys file. This key will allow owl-transfer-mgr to retrieve Owl sensor data. If you have particular key requirements (e.g., length and type), then you must give them to the manager's administrator.

### 4.3. Get Configuration Data from Manager

There are several pieces of information that must be provided by the administrator of the Owl manager. These include values for the sensor's configuration file and the sensor's name.

The following data will allow the sensor to work with the Owl manager as expected. These data are used, in conjunction with the manager keyword, in the sensor's configuration file.

Configuration Field |  | Purpose |  | Example
ssh-user |  | User on Owl manager with which owl-transfer will connect via ssh. This is only required if the sensor will be transferring data to the manager. |  | sensor-ottawa@owl-manager.example.com
heartbeat |  | URL to provide "heartbeat" data to manager. This is only required if the sensor will be providing heartbeat data. |  | http://owl.example.com/cgi-bin/owl-sensor-heartbeat.cgi

In addition to these data, it would be a good idea for you and the manager administrator to agree on a name for the sensor. This is not a hostname, but is a name by which your sensor will be known to the Owl Monitoring System. Name agreement between sensor and manager isn't required, but it will probably make things easier for you both in the long run to refer to the sensor by the same name.

The sensor name can be very generic, such as sensor42, sensor-d, or owl-us-east. It can also be very specific, such as washdc-1600-penn-ave-nw or cheltenham-bldg4. The intent is to provide distinguishing information to the intended audience of the DNS monitoring data. You should use names that easily distinguish sensors and are acceptable to the manager and sensor administrators.

### 4.4. Set Up the Owl Configuration File

The Owl configuration file must be initialized. The data fields include sensor information, query definitions, file locations, transfer intervals, and information about contacting the manager.

The following is a sample owl.conf file:

    	host name rome-sensor
    	host sensor-args	-config /home/owl/conf/owl.conf
    	host transfer-args	-conf /home/owl/conf/owl.conf
    	host admin		owl-mgmt bob@example.com

    	query		example.com	d	ns
    	query		example.com	d	a
    	query		.		h	A
    	query		.		m

    	data	dir		data
    	data	interval	60

    	log	dir		log

    	remote	ssh-user	owl-mgr@owlmonitor.example.com

    	remote	heartbeat	http://owlmonitor.example.com/cgi-bin/owl-sensor-heartbeat.cgi

The full definition of the configuration file may be found in the owl-config.pod manpage.

The sensor configuration file must include a set of query definitions. These are best specified by the administrator of the Owl manager, but you must be willing to generate and transfer the volume of data required by those queries. The impact won't be as large for the sensor as it will be for the manager since the manager is likely to have multiple sensors for which it will store data.

In addition, the manager-contact fields must be set with data provided by the administrator of the Owl manager. The ssh-user field is required for transfer of data files. The heartbeat field is optional, but must be specified if the sensor will provide "heartbeat" information to the manager.

When translated to the configuration file format, the example lines from section 4.3 will look like the following:

    	manager ssh-user        sensor-ottawa@owl-manager.example.com
    	manager heartbeat	http://owl.example.com/cgi-bin/owl-sensor-heartbeat.cgi

### 4.5. Test Transfer to Manager

At this point, the manager should be ready to accept data from the Owl sensor. Test that this will succeed with the following steps:

1. Ensure the Owl manager's administrator is ready for this test.
2. Put a file in the data directory -- specified in the data dir field of the configuration file. It doesn't matter what the file's name is nor its contents.
3. Inform the manager's administrator of the name of the file.
4. Run a test transfer in one of these ways:
   - If the sensor will be transferring data to the manager, then run "owl-transfer -config <config-file> -foreground".
   - If the manager will be transferring data from the sensor, then the manager's administrator must run the owl-transfer-mgr command.

If the file was successfully transferred, then the Owl sensor is ready to start generating data and providing it to the manager.

If not, you must determine why the file wasn't transferred. owl-transfer uses the rsync and ssh commands to perform the transfer, so problem diagnosis should start with them.

### 4.6. Add Start-up Entries

As discussed in section 1.1, you must decide whether you will run the Owl sensor using the owl-sensord daemon or if you will run owl-dnstimer and owl-transfer directly. Both methods are acceptable; the first is more robust in the case of problems.

In any event, you must add entries to your operating system's boot process to start the appropriate Owl daemons. This is highly dependent on the operating system, and may involve such things as adding some lines to /etc/rc.local or adding a start-up file to /etc/rc.d/rc2.d.

These lines executing the Owl sensor daemons may look something like this:

            /home/owl/bin/owl-sensord -config /home/owl/bin/conf/owl.conf

or

            /home/owl/bin/owl-dnstimer -config /home/owl/bin/conf/owl.conf
            /home/owl/bin/owl-transfer -config /home/owl/bin/conf/owl.conf

It is incumbent upon the installer to determine the proper method of Owl start-up and the proper place to set up execution.

### 4.7. Add cron Entries

The owl-dataarch, owl-archold, and owl-heartbeat programs provide additional services for the Owl Monitoring System. These programs are not required, but they can ease some of the burden of administering an Owl sensor. However, due to the volume of data generated by an Owl sensor, it is strongly advised that owl-dataarch be run.

These programs may be set up as cron jobs. This will remove the necessity for the administrator to run these programs manually, and it will provide regularity to their execution.

owl-dataarch periodically moves sensor data from the data directory to an archive directory. It should be run on a daily basis.

owl-archold will create a compressed tarfile of a month's data that has been previously stored in the archive directory. It should be run on a monthly basis -- but not on the first or second day of the month. If the sensor host is used for much other than as an Owl sensor, then it would be best to run owl-archold whenever the system is normally at "off hours."

owl-heartbeat touches a webpage (specified in the configuration file) and is intended to inform the Owl manager that the sensor is up and running. This may be run as often as desired. The more frequently it's run, the more up-to-date is the information on the Owl manager. The less frequently it's run, the lower the granularity of the sensor-availability information on the manager. Once per minute or hour is probably sufficient for most purposes.

Program | Frequency of Execution
owl-dataarch | once per day
owl-archold | once per month, on the 3rd of the month or later
owl-heartbeat | as desired, but once per minute or hour is reasonable

### 4.8. Start Owl Daemons

Installation and configuration are complete for the Owl sensor. The Owl sensor is now ready to gather timing data on DNS queries. If the sensor will be transferring data to the Owl manager, it is ready for that task as well. To start the Owl sensor daemons, you can either start them manually (either by running just owl-sensord or by running both owl-dnswatch and owl-transfer) or by rebooting your system.

### 5. Adding New Queries to Existing Owl Sensors

From time to time, you might decide to change the queries being performed by the Owl sensors. This could be adding new queries to some or all of the sensors, stopping some of the queries from being performed, or modifying existing queries. Regardless of the changes, there are actions that must be taken on the Owl manager and all affected Owl sensors.

It is relatively painless to modify queries for a sensor. The following actions must be performed:

1. The query entries in the Owl configuration file must be changed to reflect the new set of queries that should be performed. Add new query entries to add new queries; delete or change existing query entries to remove or modify current queries. See section 4.4 for information about query entries.

2. Restart owl-dnstimer.

If you have correctly set up your configuration file, then owl-dnstimer will start collecting data for the new queries.

Stopping collection of certain queries only requires deleting (or commenting out) the appropriate entries from the configuration file. owl-dnstimer, of course, must be restarted.

### 6. Owl Sensor Commands

The Owl sensor has a number of Owl-specific commands required for its operation. This section provides a brief overview of the available commands. These are all Perl scripts and have self-contained man pages, which contain greater details about their use. These scripts are intended to run as an unprivileged user.

This section briefly describes the software used on the Owl sensor hosts.

- owl-archold
  This script archives previously archived Owl sensor data into monthly archives. A compressed tar file is created that contains the archived data files from a particular month. It should only be run on a particular month's archive after the month has been completed. This script is exactly the same as the owl-archold on the Owl manager host.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-dataarch
  This script moves data from the Owl sensor's active data directory to a year/month-specific archive directory. This script archives Owl sensor data by moving old data from the Owl data directory to an archive directory.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-dnstimer
  This script is the primary part of the Owl sensor. It sends periodic DNS requests to the root nameservers and records the time each request takes. These query records are provided to the Owl manager for aggregation and display.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-heartbeat
  This script periodically contacts the Owl manager to let it know the sensor is still alive. owl-heartbeat is an optional service for the benefit of the Owl manager's administrators; the Owl sensor will run just fine if a sensor's administrator elects not to run it. This is intended to be a cron job that runs once a minute, but other time periods may be chosen.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-sensord
  This script executes the owl-dnstimer and owl-transfer daemons and monitors their execution. If either of the daemons exits unexpectedly, then owl-sensord will restart it. In order to keep configuration or other problems from causing rapid re-executions, owl-sensord will temporarily suspend execution of one of the daemons if a certain number of execution attempts fail within a short period of time.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-status
  This script gives the user a simple view of the status of the three primary Owl sensor components. It reports whether or not owl-dnstimer, owl-sensord, and owl-transfer are running. An additional option shows the query parameters used by owl-dnstimer and the time that each root nameserver was last queried.
  This script is assumed to live in the Owl user's home or bin directory.

- owl-transfer
  This script periodically transfers Owl sensor data from the sensor host to the manager host.
  This script is assumed to live in the Owl user's home or bin directory.

## Manager Installation Manual

### 0. Setting Up Managers for the Owl Monitoring System

The Owl Monitoring System uses timed Domain Name System (DNS) queries to monitor basic network functionality. The system consists of a manager host and a set of sensor hosts. The Owl sensors perform periodic DNS queries and report to the Owl manager the time taken for each query. Over time, this shows the responsiveness of the DNS infrastructure.

An Owl sensor is configured to send DNS queries to a set of DNS nameservers. The queries may be for a variety of DNS resource record types, and they are sent to a specific nameserver to request data about a specific target host. For example, one query might request that the D root nameserver return NS records for the "www.example.com" host, while another query might request that a local (non-root) nameserver return AAAA records for the "mail.example.com" host. Data will be collected for each configured query and given to the Owl manager to whom the sensor reports. The different sensors in the sensor set may perform different queries or the same queries.

A configuration file controls the behavior of the Owl sensor. This file defines the queries that the sensor will perform, the Owl manager (or managers) to whom the sensor reports, locations of data and log files, frequency of queries, and other important behaviors.

An Owl sensor may provide its data to more than one Owl manager. In such situations, the managers operate independently of each other and do not know of each other's existence. Throughout the installation manuals, it will be assumed that an installation will have a single manager. Special instructions for multiple-manager environments will be given where needed.

This document provides an operational overview of the Owl system, installation instructions for the Owl manager and the third-party software that supports it. Instructions for configuring the Owl manager are also given.

### Organization of Installation Manual Sections:
| 0. |  | Setting Up Managers for the Owl Monitoring System
| 1. |  | Operational Overview of the Owl Monitoring System
| 2. |  | Owl Manager Installation
| 3. |  | An Interlude on Sensor Queries
| 4. |  | Adding Owl Sensors to an Existing Owl Installation
| 5. |  | Defining Graphs
| 6. |  | Changing Queries on Existing Owl Sensors
| 7. |  | Owl Manager Commands
### 1. Operational Overview of the Owl Monitoring System The Owl Monitoring System has a conceptually simple method of operations. There is a manager host and a set of sensor hosts. The sensors periodically make a set of DNS queries and time how long it takes to get a response. The response times are saved to data files. After a certain amount of time, the data files are transferred from the sensor to the manager host. For any particular sensor, this data transfer may be configured to be initiated by either the manager or by the sensor. The manager transfers the data into a monitoring package, which allows easy display of the data. Below is a diagram of the flow of Owl sensor data through two Owl monitoring installations. Take note of the following: There are three distinct administrative domains, two of which have Owl managers. Owl Manager 1 receives sensor data from all three domains. Owl Manager 2 receives sensor data from itself and one other domain. Domain 1's sensors do not provide data to other domains. Domain 2 contains an Owl sensor, but not an Owl manager. Owl sensors in the domains 2 and 3 provide data to Owl managers outside of their own domain. Owl sensors can query multiple name servers. (Domains in this example are defining administrative control and are not DNS domains.) The remainder of this section provides a little more information on the components that make up the sensor and manager software. ### 1.1. Owl Sensor Overview An Owl sensor makes timed DNS queries and records the time required for a response. The response time data are transferred to the Owl manager after a certain period of time. The transfer may occur at the instigation of either the manager or the sensor. The Owl sensor's three primary programs that handle these tasks are described below. The Owl sensor's actions are controlled by a configuration file. This file defines the queries that must be performed, how often the queries must happen, how frequently the response data files are transferred to the manager, data and logging information, and other required information. The owl-dnstimer program performs periodic DNS lookups, as specified by the configuration file. These lookups are timed so that the sensor can record how long it took to make a particular request. The configuration parameters used by owl-dnstimer include the nameserver to query, the target zone to query about, the type of DNS record, and the frequency of queries. If the Owl system is configured so that the sensor transfers its data to the manager, then the transfer is performed by the owl-transfer program. The transfers are performed using rsync over an ssh connection. The sensor host must be able to do a password-less ssh into the manager host. The frequency of transfers is defined in the configuration file. Choosing the transfer frequency is a trade-off between lower impact on the sensor, manager, and network (less frequent) and lower latency of data display on the manager (more frequent.) The owl-sensord program is an optional controller for owl-dnstimer and owl-transfer. If used, it will run these daemons and restart them if they stop. If one of the daemons tries to restart too often in too short a time, then owl-sensord will warn an administrator (via email) and temporarily stop trying to execute the program. owl-sensord is not required in order to run owl-dnstimer and owl-transfer; the Owl sensor can run fine without it. However, it provides a convenient way to keep them running. As part of the installation process, you must decide whether you will run the Owl sensor using the owl-sensord daemon to control execution or if you will run owl-dnstimer and owl-transfer directly. The Owl sensor has a number of programs for administrative support. Most of these are not required, but they assist in sensor administration. In particular, the owl-dataarch and owl-heartbeat programs are intended to run as cron jobs. owl-dataarch moves "old" sensor data into an archive directory, in order to keep the sensor-manager data transfer from bogging down (within rsync) as the data collection grows. The owl-heartbeat program periodically touches a webpage hosted on the Owl manager, in order to let the manager know that the sensor host is still running. owl-heartbeat is not critical and can be left off, but owl-dataarch is essential. There are a few other administrative commands available; see section 6 for a brief description of all the Owl sensor commands. ### 1.2. Owl Manager Overview The Owl manager's purpose is to receive sensor data and make it available for display. The majority of the Owl manager is actually third-party packages, and the Owl manager software provides the "glue" that allows the Owl sensor data to work with those packages. In particular, Owl provides data to the Nagios monitoring system, which then displays the data. The Owl sensors may report to managers that are under separate administrative control. A particular installation of the Owl Monitoring System may use multiple manager hosts, but this document assumes that there is only one. The manager can have either an active or passive role in the process of moving Owl sensor data from the sensor to the manager. If the manager is active, it will use the owl-transfer-mgr program to pull sensor data from the Owl sensor host. If the manager is passive, then the sensor will transfer the data when it sees fit. The are two groups of "glue" programs that Owl uses to interface with Nagios. The owl-dnswatch and owl-perfdata inject Owl sensor DNS response data into Nagios for display and graphing. The owl-sensor-heartbeat.cgi and owl-stethoscope programs insert Owl "heartbeat" data into Nagios so it can track availability of the Owl sensors. The Owl manager also has a number of programs for administrative support. Most of these are not required, but they assist in manager administration. In particular, the owl-archdata and owl-monthly programs are intended to run as cron jobs. owl-archdata moves "old" sensor data into an archive directory, in order to keep data transfer from the sensor from bogging down as the data collection grows. The owl-monthly program archives the previous month's sensor data into a compressed tarfile. owl-newsensor and owl-initsensor assist in setting up the manager for a new sensor. There are a few others available; see section 7 for a brief description of all the Owl manager commands. The following third-party software packages are used in with this distribution of the Owl Monitoring System. The packages are not included with the Owl distribution, but must be retrieved as required for your operating system. These packages must be installed on the manager:
| Nagios | Monitoring system.
| Nagios plugins | Monitoring system.
| nagiosgraph | Provides graphing for Nagios.
| rrdtool | Database tool for use by nagiosgraph.
| drraw.cgi | Draws graphs for nagiosgraph of data stored by rrdtool.
- Instructions are given in this document for integrating these packages with the Owl software. It would be for the best to read the installation instructions below for each package prior to installing them. Different management, graphing, and database tools may be used instead of those listed above, if so desired. Integrating the Owl manager software with other packages is beyond the scope of this manual. David Josephsen's Building a Monitoring Infrastructure with Nagios is very helpful for understanding Nagios and how the various pieces work together. If you're going to be running a Nagios monitor, you would be doing yourself a favor to read this book. ### 1.3. Data Retention on Owl Hosts Owl sensors generate a large amount of DNS response data. For example, a sensor running five queries, once a minute each, generated roughly 517KB of data per day. (Due to the way the data are stored, this will be affected by the amount length of target and nameserver hostnames used.) You may or may not want to retain any of this data. The data files can be useful for rebuilding the Owl manager's rrdtool databases (or building with different parameters.) On both the Owl manager and the Owl sensors, sensor data are stored in a data directory. After a few days, the data files are moved to a data archive directory. A full month's data files will be collapsed into a compressed format. The Owl Monitoring System provides tools that assist with managing Owl sensor data. After the sensor data have been moved from the sensor to the manager, the sensor has no further need of the data. On the manager, after the data have been moved into the data archive, the Owl Monitoring System has no further need of the old data; it may be retained or deleted. (The data will have been moved into the manager's rrdtool databases by then, so the data is not lost.) Each installation must decide their own data retention policy. ### 2. Owl Manager Installation This section gives installation procedures for setting up the Owl Monitoring software on manager hosts. There are two sections, an installation section and a configuration section. The installation instructions discuss software installation of Owl software and supporting third-party software packages. System configuration is also detailed in this section, but only for non-Owl-specific configuration. There are a number of Owl-specific configuration actions that must be performed and these are detailed in section 4. The Owl manager requires Perl. You must install it on your system if it isn't already available. ### 2.1. Manager Installation The basic Owl manager configuration consists of the Owl software and the Nagios monitoring system. This configuration allows for monitoring of current sensor status, but does not provide any view into past monitoring data. In order to get the historical view, especially as provided by a graphing capability, several additional software packages must be installed on the manager. Since the historical view provides a great deal of useful information, these instructions will assume that you will be installing the additional software. It is possible (and not too difficult) to run multiple instances of the third-party software packages (e.g., Nagios, nagiosgraph) on a single host, but care must be taken when configuring, building, and installing the software. Without careful work, an existing installation may be overwritten by a later installation. These instructions assume that only one instance of each package will be installed on the manager host. Below is a diagram of the flow of Owl sensor data on the Owl manager. The details may not make sense at this stage, but this will show how the various pieces fit together. ### 2.1.1. Preparatory Steps Several actions must be taken on the manager host to prepare it for installing the required software. These steps are listed below. During the installation procedures, don't be shy about backing up things; it might not be necessary, but better safe than sorry. It is advisable to run a backup of your system before starting. Depending on your level of paranoia, it couldn't hurt to run an incremental backup after completing each step of the installation. An unprivileged user account must be created for the Owl manager software. An existing account may be used, but it would probably be best to use a new account as this will segregate files and processes from existing users. The user's name and group are not required to be any particular values. You may want to create two unprivileged users for use by the Owl manager. One would run the various Owl commands, and the other would be used for sensor data transfers (in the case that the sensor is to initiate transfers.) The first user will need rwx permission to all the Owl directories and files. The second user will need to be able to write data into the Owl data directories. In the following instructions, it is assumed that a single user will created for the Owl manager and it will be referred to as the Owl User. SSH utilities and a web server must be installed on the manager host. These instructions assume that the OpenSSH and the Apache web server will be used. Others may be used instead, but the you must determine the appropriate commands and options to use. Apache support is included in Nagios. Install the rrsync utility. This is a front-end to rsync that restricts the locations to which rsync may copy data. This tool is available here: http://ftp.samba.org/pub/unpacked/rsync/support/rrsync Create the directories required for each sensor. There must be a data repository and a data archive. Each sensor will have their own subdirectory in the repository and in the archive. There must be a data and a history directory within each sensor's repository directory. The following is a possible set of directories:
  | /owl/data | top-level data repository
  | /owl/old.data | top-level data archive
  | /owl/data/sensor-portland/dnstimer | sensor-portland's DNS timer-data directory
  | /owl/data/sensor-portland/history | sensor-portland's monitoring history (for Nagios)
  | /owl/old.data/sensor-portland | sensor-portland's data archive
  | /owl/data/sensor-oslo/dnstimer | sensor-oslo's DNS timer-data directory
  | /owl/data/sensor-oslo/history | sensor-oslo's monitoring history (for Nagios)
  | /owl/old.data/sensor-oslo | sensor-oslo's data archive
Create an SSH key for the Owl user. This is commonly done with the ssh-keygen command. Consider the users you want to access your Owl monitor, as well as the types of access they should be allowed. These users will be added to the Owl-specific password file used by Apache and the users and access rights will be added to a Nagios configuration file. These users are web-based users only, and are not (necessarily) related to users on the Owl manager host. Install the software as described in the following sections. The Owl software will be installed after all the third-party software, since some of the Owl software will be installed in directories created by the third-party software. ### 2.1.2. Installing Third-Party Software The third-party software packages should be built and installed as specified by each package. Owl-specific notes for each will be given below. While many of these packages are available as pre-built distributions, configuring and building them yourself will provide for additional control that may be useful. This includes the installation location and the user and group that will be used for executing the commands. Nagios, along with a web server, should be installed such that they will be restarted at system boot time. This table shows the most recent software versions that were tested with the Owl Monitor.
| Package | Version Tested with Owl
| Nagios Core | 3.4.1
| Nagios Plugins | 1.4.16
| nagiosgraph | 1.4.4
| rrdtool | 1.4.7
| drraw.cgi | 2.2b2
### 2.1.2.1. Nagios Core and Plugins If building Nagios Core and Nagios Plugins from source, it is recommended that the configure command be given arguments that will allow specification of the installation-path prefix, the Nagios user, and the Nagios group. A possible command line could be:
        $ ./configure --with-command-group --prefix=/owl --with-nagios-user=owl-monitor --with-nagios-group=owl-monitor
The same prefix, username, and groupname must be used when building Nagios Core and Nagios Plugins. Multiple instances of Nagios can be installed on a single machine. This can be useful for segregating different things being monitored. However, you must be careful when doing this. In particular, even if the --prefix option was given to configure, the "make install-init" and "make install-webconf" commands will overwrite existing Nagios files. ### 2.1.2.2. nagiosgraph The README file for nagiosgraph contains instructions for automated installation (using install.pl) and for manual installation. You should read this file and decide which method you want to use. If you choose the manual installation, you may skip most of this section and pick up at the discussion of the map file. If you choose to install with install.pl, you must know that it looks like several of the steps are not performed by the script. After running install.pl you must verify that the manual steps from "Restart nagios" to the end have actually been taken. install.pl will configure the package, install its files in the appropriate places, and modify Nagios and httpd files as needed. A variety of options allow for site-specific installation layouts. In addition, you will be prompted for many directory and file names prior to install.pl completing its work. If it can't find one or another file that it must update, then at the end of execution it will provide a set of steps that you must manually perform in order to complete the installation. The --dry-run option will show you what install.pl would do, so that you may verify where things are being copied. It isn't required, but it wouldn't be a bad idea to use this option prior to the actual install. One thing install.pl is likely to say is that you must add a Nagios command file. This file probably already exists, and you must find it. It is likely to be called something like commands.cfg and be located in the etc/objects directory in your Nagios installation. Check names carefully when making this modification, as commands.cfg already has a process-service-perfdata command and install.pl will have you add a process-service-perfdata-by-nagiosgraph command. install.pl logs all its actions, including the manual-intervention actions, in the install-log file. Unfortunately, the command line is not recorded in the log file, so it might be useful to save that somewhere yourself. I have used this command to successfully install nagiosgraph:
        $ install.pl --prefix /owl/nagiosgraph --nagios-user owl-manager --nagios-cgi-url /nagios-owl/cgi-bin \
        --nagios-perfdata-file /owl/var/service-perfdata.out --log-dir /owl/var --layout standalone
- I do not let install.pl update the Nagios or Apache configurations, which it will prompt for; I prefer to make those modifications myself. Refer to the install-log file to see what remains to be done. The README file contains steps for manual installation and it looks like several of the steps are not performed by install.pl. If you use install.pl to install nagiosgraph, verify that the manual steps from "Restart nagios" to the end have actually been taken. You may have to manually copy the nagiosgraph.ssi file into place. This file is in .../nagiosgraph/examples/nagiosgraph.ssi and it should be copied into your equivalent of /owl/share/nagiosgraph.ssi. Before nagiosgraph can be used by the Owl manager, the map file must be modified to recognize data from the Owl plugins for Nagios. The required modifications are discussed in section 2.2.7. ### 2.1.2.3. rrdtool No Owl-specific configuration or actions were required for building and installing rrdtool. ### 2.1.2.4. drraw.cgi To install drraw.cgi, copy the drraw.cgi and the drraw.conf files into the Nagios sbin/ directory. So, if you have installed Nagios in /owl/nagios, then you should copy the two drraw files into /owl/nagios/sbin. The drraw.conf must be edited for this installation. Some of these edits are required and some are optional. Each will be marked as to whether or not you must make the change. This file is a Perl file, so the configuration values are set according to Perl's language rules. Set the $title variable if you don't want the default page title in the Nagios-displayed graphing pages. (optional) Set the %datadirs variable to hold the paths to your sensors' RRD databases. There are two fields required for each entry. The first is the path to that sensor's database files, and should not have a terminating slash. The second is a name by which those databases will be identified when defining new graphs.
          '/owl/nagiosgraph/var/rrd/sensor1%20h-root%20foo.com%20A'  => '[Kyoto H foo A]',
          '/owl/nagiosgraph/var/rrd/sensor1%20h-root%20foo.com%20NS' => '[Kyoto H foo NS]',
          '/owl/nagiosgraph/var/rrd/sensor5%20h-root%20foo.com%20A'  => '[Cairo H foo A]',
- The identifier should be unique for each database directory. (required) Set the vrefresh variable for the number of seconds the graph viewer will wait before automatically refreshing its display. (optional) Set the drefresh variable for the number of seconds the dashboard viewer will wait before automatically refreshing its display. (optional) Set the @dv_def, @dv_name, and @dv_secs array variables for the graphs that will be generated each time a predefined graph is displayed. These are time based values and a particular element of one array is directly related to that same element in the other arrays. The @dv_def array is used to select data from the database. The @dv_name array is used to provide titles for each graph. The @dv_secs array is used to provide the count of seconds for each graph. The following example values show the relation between the three arrays.
          @dv_def  = ( 'end - 15 minutes', 'end - 60 minutes', 'end - 4 hours',
      		 'end - 12 hours',   'end - 24 hours',   'end - 1 week' ,
      		 'end - 1 month',    'end - 1 year');

          @dv_name = (
      	'Past 15 Minutes',
      	'Past 60 Minutes',
      	'Past 4 Hours',
      	'Past 12 hours',
      	'Past 24 Hours',
      	'Past Week',
      	'Past Month',
      	'Past Year'
          );

          @dv_secs = ( 900, 3600, 14400, 43200, 86400, 604800, 2419200, 31536000 );
- (optional) The $saved_dir variable defines where definitions for your predefined graphs will be stored. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:
          $saved_dir = '/owl/nagios/var/drraw/saved';
- (required) The $tmp_dir variable defines where cached graphing file will be stored until they time out and are deleted. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:
          $tmp_dir = '/owl/nagios/var/drraw/tmp';
- (required) The $ERRLOG variable defines where graphing errors will be logged. It must be set to match your path configuration and it must exist and be writable by the web server. It might be something like:
          $ERRLOG = '/owl/nagios/var/drraw/errors.log';
(required) ### 2.1.2.5. Perl modules There is a set of Perl modules required by the Owl manager and the third-party software packages. Some of the software packages may install these packages themselves, but you must ensure that these are available. These should be available through CPAN. Modules required by the Owl manager: Cwd.pm Fcntl.pm File::Path.pm Getopt::Long.pm Time::Local.pm It is likely that some of the third-party software may require other Perl modules. The installation documentation or process should inform you of what must be installed on your system. ### 2.1.3. Installing the Owl Monitor Software There is a small amount of manager-specific software in the Owl Monitor. Most of the work on the manager is performed by Nagios and the other third-party software. The Owl Monitor software on the manager host consists of two Nagios plugins that will interpret Owl sensor data for Nagios and add Owl sensor data to graphing databases. There is also a small number of administrator programs for managing the large amounts of Owl sensor data. ### 2.1.3.1. Unpacking the Owl Monitor Software The Owl Monitor software comes in a bzip2'd tar file. In most modern Unix systems, this can be unpacked with this command:
            $ tar xzf owl-monitor.tar.gz
- This command unpacks the Owl distribution in the current directory. This can be unpacked anywhere you want, but it is a good idea to do it in the Owl manager's home directory. Most of the Owl commands will be in the owl-manager/bin directory. These commands can stay in that directory or be moved elsewhere. If they remain in their own directory, it would be helpful to add the directory to your shell's path. ### 2.1.3.2. owl-dnswatch The owl-dnswatch command is a Nagios plugin that reads Owl monitoring data and reports the latest results to Nagios. Each execution of owl-dnswatch requires an Owl sensor name, a target host, a DNS server, and a DNS query type. The appropriate monitor datafile is consulted based on those inputs. The following actions must be taken to install owl-dnswatch. Set the $OWLDATA value in owl-dnswatch to the top-level directory that holds the Owl monitoring data. This path will depend on your installation hierarchy, but it will likely look something like:
              my $OWLDATA = "/owl/data";
- Copy owl-dnswatch to the Nagios plugin directory. This directory is probably something like /owl/nagios/libexec. ### 2.1.3.3. owl-perfdata The owl-perfdata command is a Nagios plugin that acts as a front-end for inserting Owl monitoring data into an RRD database. These databases are used for graphing the monitoring data. The following actions must be taken to install owl-perfdata. Set the $GRAPHINSERT value in owl-perfdata to the path to the insert.pl command. This path will depend on where you have installed nagiosgraph, but it will likely look something like:
              my $GRAPHINSERT = "/owl/nagiosgraph/bin/insert.pl";
Copy owl-perfdata to the Nagios plugin directory. This directory is probably something like /owl/nagios/libexec. ### 2.1.3.4. owl-newsensor There are a set of Nagios objects that must be created for each Owl sensor. Once the Owl manager has sensor data files in its data directory, the owl-newsensor program can be used to create these objects. The actions required for this are described in section 4.7. No local modifications are required for owl-newsensor. Copy owl-newsensor to a directory which will be its home. This could be anything that is convenient for you; e.g., /usr/bin, /usr/local/bin, ~/mystuff, or /owl/scripts. ### 2.1.3.5. owl-transfer-mgr If the Owl manager transfers data from a sensor, it will use the owl-transfer-mgr to retrieve the data. No local modifications are required for owl-newsensor. ### 2.1.3.6. Data-Archiving Programs The following programs assist the administrator with managing the large amount of Owl monitoring data that accumulates on the Owl manager host. These programs work together to create a two-stage archival process. It is helpful to have cron run these programs, though this is not essential. The first stage of the archive process moves old data from the sensor's data directory to the sensor's archive directory. Data is considered old if the file's timestamp (in its name) is older than two days. This part of the data archiving should be performed once per day, depending on how frequently the sensors perform their own data archiving. The second stage of the archive process combines a month's worth of sensor data into a single compressed tarfile. This should be performed several days after the end of the previous month. The Owl data-archival programs assume that all sensor data are stored in a common directory and all the archived data are stored in a common directory. They will segregate data by sensor in order to prevent it from getting mixed together. For example, when handling data for the sensors dresden and kvothe, these programs might expect the following directories to exist:
        /owl/data/dresden
        /owl/data/kvothe
        /owl/data-archive/dresden
        /owl/data-archive/kvothe
- This first set of programs will archive data for all the sensors that report to an Owl manager. They must be told where the sensor data are stored and where the data archive is kept. owl-archdata - Archives data for all sensors without having to name them explicitly. The sensors in the given data directory will be archived. The data are stored in year-month subdirectories in the sensor's archive. owl-archdata is used in the first stage of the archival process. owl-monthly - Archives all sensors' archived data in a compressed tarfile, in order to minimize space taken by the archive. The tarfiles are stored in another subdirectory, divided by year. owl-monthly is used in the second stage of the archival process. This second set of programs will archive data for a single sensor that reports to an Owl manager. They must be told where the sensor data are stored and where the data archive is kept. owl-dataarch-mgr - Archives data for a single sensor. This is intended to be run by owl-archdata, and it is a slightly modified version of the Owl sensor's owl-dataarch utility. owl-archold - Archives a sensor's archive data in a compressed tarfile. This is intended to be run by owl-monthly. The Owl manager's version of owl-archold is exactly same as the Owl sensor's utility of the same name. More information about each of these programs is available in their man pages. Copy these archiving programs to a directory which will be their home. This could be anything that is convenient for you; e.g., /usr/bin, /usr/local/bin, ~/mystuff, or /owl/scripts. ### 2.2. Modifying Configuration Files Several files must be modified in order to provide Nagios monitoring and graphing. These files belong to Nagios, Apache, and nagiosgraph. This section describes the required modifications. These modifications do not provide any monitoring of Owl sensor data; further steps to supply that functionality will be given later. In the basic Nagios installation, /nagios/etc holds sample configuration files, which can be modified for use. ### 2.2.1. Save Files! Before starting to modify any existing file -- e.g., Nagios object files and Apache configuration files -- it would be a very good idea to make a back-up copy of the files. This will provide reference copies of the files in case they might be needed later. ### 2.2.2. nagios.cfg (Nagios) This file is the primary configuration file for Nagios. There are quite a few changes that must be made to this file. The modifications listed in this section will not incorporate monitoring of Owl sensor data. That will be described in section 4.8.1. The various configuration parameters for Nagios could be included in a small number of files, but the standard arrangement groups the parameters into separate files. The nagios.cfg file is the primary configuration file for Nagios, and at run-time it acts as the focal point to collect the disparate configuration files for the Nagios daemon. A cfg_file line is used to tell Nagios to include the specified line. Adding or deleting these lines will control which files are included in the Nagios configuration. As a result of the Nagios build and installation process, the nagios.cfg file should be mostly configured already. The Nagios configuration file is well-documented, so figuring out how to adjust its fields shouldn't be too difficult. There are a few things that must be done to prepare it for Owl monitoring. These directions assume that Owl monitoring is the only monitoring that will be performed by this installation. Make sure these cfg_file lines are deleted or commented out: localhost.cfg printer.cfg switch.cfg windows.cfg (This is not strictly necessary. These files may provide some monitoring information that you would find useful.) To allow for graphing of monitoring results, several performance-processing fields must be set in nagios.cfg. Unless otherwise stated, do not include the double quotes when setting these fields. Find the process_performance_data line and change it to: "process_performance_data=1" Find the service_perfdata_file line and change it to: "service_perfdata_file=/owl/var/service-perfdata.out" Add this line: "service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$ ||$SERVICEOUTPUT$||$SERVICEPERFDATA$" (This line was split to accomodate document formatting. Add it as a single, unbroken line without any spaces or newlines.) Find the service_perfdata_file_mode line and change it to: "service_perfdata_file_mode=a" Add this line: "service_perfdata_file_processing_interval=30" ### 2.2.3. cgi.cfg (Nagios) This is the configuration file for CGI scripts run by Nagios. Most of it is set up during the Nagios configuration process. The following things must be set to ensure your installation's needs are met. The url_html_path field contains the path portion of the URL for the Nagios monitor. For example, if the desired URL to reach your Nagios monitor is http://www.example.com/owl-dns-monitor, then url_html_path must be set to /owl-dns-monitor. Check this field to ensure it is set as required. The cgi.cfg file allows you to specify different types of information and actions that can be taken by different users. The authorized_for_ values may be set for multiple users. The user names are those in the htpasswd.users file. ### 2.2.4. resource.cfg (Nagios) Ensure USER1 points to the plugins directory. This should be something like /owl/libexec. ### 2.2.5. share/config.inc.php (Nagios) Nagios uses this file to hold a small amount of file-location data. The following actions must be taken: Ensure that has the correct path for the $cfg[] fields. If your configuration doesn't use the default paths, this may not be set properly. Add an entry for the $cfg['ng_cgi_base_url'] field to be set to the path of the nagiosgraph CGI directory. This will depend on how you installed nagiosgraph, and should look something like this:
          $cfg['ng_cgi_base_url']='/nagiosgraph/cgi-bin';
          $cfg['ng_cgi_base_url']='/nagiosgraph-owl/cgi-bin';
### 2.2.6. nagiosgraph.conf (nagiosgraph) This is the configuration file for Nagiosgraph. Several settings must be checked and some Owl-specific setting copied in. The following actions must be taken: Copy the contents of ng-owl-conf After the lines setting xffs. These are reasonable values, but don't change them unless you know what you're doing. Ensure that nagiosgraphcgiurl path points to the appropriate directory, as defined by your web server's configuration. Ensure that javascript path points to the appropriate directory, as defined by your web server's configuration. Ensure that stylesheet path points to the appropriate directory, as defined by your web server's configuration. ### 2.2.7. map (nagiosgraph) nagiosgraph uses a file of data definitions that describe how to extract data from Nagios plugins and how to handle the extracted data. The file is a collection of Perl expressions that are evaluated by nagiosgraph. This file is named something like /owl/nagiosgraph/etc/map. The Owl manager distribution contains a set of additional, Owl-specific definitions that must be added to the map file. These definitions are in ng-owl-map. Copy the contents of this file into the map. ### 2.2.8. htpasswd.users (Apache) Strictly speaking, this file is not a Nagios file, but is instead an httpd file. The Nagios configuration file for httpd gives the name of an htpasswd.users file. Use the htpasswd command to create and populate this file with the users that should be allowed to access the Owl monitor. These users must have their permissions defined in the Nagios cgi.cfg file. ### 3. An Interlude on Sensor Queries Before running your Owl Monitoring System, you must consider a number of questions. How many Owl sensors will be providing data to the Owl manager? How many queries will the sensors be performing? What queries will the sensors be performing? Will each sensor be performing the same queries? Will the sensor be providing "heartbeat" information to the manager? Who will transfer the sensor data to the manager? These questions are important for the sensor administrator, but they are critical for the manager administrator. ### 3.1. Gathering and Storing Sensor Data A query consists of three parts: a nameserver, a target host, and a query type. Responses to each query will be kept in their own datafile. The amount of data generated by each sensor will depend on the number of queries and the frequency with which the queries are performed. The sensors will have to gather the data and more queries will result in more data that will be collected. The data will all have to be transferred from the sensor to the manager. If the manager is configured for graphing, then data from each query will be stored in its own database. These databases can be quite large, so the manager host must have a good amount of available disk space. The administrators of an Owl sensor and the Owl manager (if they are different people) must coordinate the queries that the sensor will be performing. The manager's administrator can request that a set of queries be performed, but it is up to the sensor's administrator to actually configure the sensor for the queries. The manager is effectively at the mercy of the sensor. Sensor data filenames contain metadata about the sensor data contained therein. Therefore, the filenames will reflect the queries that generated the files. The fields are separated by commas and have this format:
            121129.1701,seattle-sensor,example.com,a.root-servers.net,A.dns
            121129.1701,seattle-sensor,example.com,m.root-servers.net,A.dns
            121129.1701,seattle-sensor,example.com,m.root-servers.net,NS.dns
            121129.1701,portland-sensor,example.com,a.root-servers.net,A.dns
Breaking out the fields of the last example above gives this table:
| Field |  | Purpose |  | Example
| Timestamp |  | File creation timestamp; (YYMMDD.hhmm) |  | 121129.1701
| Sensor Name |  | Name of sensor that recorded data |  | portland-sensor
| Target Name |  | Host whose name was target of DNS lookups |  | example.com
| Nameserver Name |  | Name of queried nameserver |  | a.root-servers.net
| Query Type |  | Type of DNS query |  | A
| File suffix |  | File suffix |  | .dns
### 3.2. Transferring Sensor Data The Owl Monitoring System provides utilities to transfer sensor data from the sensor to the manager. One utility, owl-transfer runs on the sensor and transfers the data to the manager. The other utility, owl-transfer-mgr runs on the manager and transfers the data from the sensor. Both utilities are front-ends for the rsync program and run using an ssh-initiated connection. In both cases, the transferring host must be able to ssh into the remote host without a password. The FOSS rrsync ("restricted rsync", written by Joe Smith, modified by Wayne Davison) is used to restrict the transferring host's access to the remote host. The administrators of the sensor and the manager must agree on which host will be transferring data to the manager. The appropriate SSH public keys from the transferring host must be placed on the other host, so that the password-less ssh (as described above) may be performed. An Owl system with multiple sensors may have a mixed transfer configuration. In such a system, some sensors will transfer their data to the manager, while the manager will transfer data from other sensors. ### 3.3. Sensor Heartbeats The "heartbeat" facility allows an Owl sensor to periodically contact a manager to let it know that the sensor is still alive and network accessible. The heartbeat is sent by contacting a particular webpage on the Owl manager. The heartbeat facility is not necessary for Owl operation and Owl runs perfectly well without it. However, it can provide a useful assurance to the manager administrator that the sensor is still working. Enabling this facility on the sensor is at the discretion of the sensor administrator. Since this requires a webserver to be running on the manager host, that aspect is at the discretion of the manager administrator. ### 4. Adding Owl Sensors to an Existing Owl Installation Whenever a new Owl sensor is added to those handled by an Owl manager, there are a number of actions that must take place on both the sensor and the manager. These actions should be followed consecutively within each section. However, there is some amount of necessary interleaving of actions. For example, a particular sensor action may be required before a particular manager action. It is acceptable for the Owl sensor and Owl manager to be under completely different administrative control. Each host may even be owned by different commercial, educational, governmental, or other such entity. All that is required is that there be some initial coordination between administrators when the sensor is first configured, along with (probably) aperiodic support from time to time later. The discussions on adding Owl sensors assumes that the sensor and manager are under different administrative control. Required actions will not change if they are under unified administrative control, but they may be easier to accomplish. This section describes the various actions that must be performed on the Owl manager in order to configure a new sensor for it. This assumes the installation and configuration procedures detailed in section 2 have been completed. ### 4.1. Create Directories and Files for Sensor's Data A set of directories and files must be created for a new sensor. These are the sensor directory, the sensor's data directory, the history directory, the data archive directory. The heartbeat file, if the sensor will be providing heartbeats, must also be created. These files are best created by the owl-initsensor command, but they may also be created manually. If they are created manually, you must use the names and organization as given in the examples below. The locations of the directories may change (i.e., the /owl/data and /owl/archive portions), but that's it. For example, when adding the new sensors helsinki and canberra, you might create the following sets of directories:
sensor directory data directory heartbeat file history directory archive directory |  | /owl/data/helsinki/ /owl/data/helsinki/data/ /owl/data/helsinki/heartbeat /owl/data/helsinki/history/ /owl/archive/helsinki/ |  | /owl/data/canberra/ /owl/data/canberra/data/ /owl/data/canberra/heartbeat /owl/data/canberra/history/ /owl/archive/canberra/
The archive directory must be writable by the manager's Owl user, but the others must be writable by the manager's Owl user and the manager's web server. ### 4.2. Firewall Configuration The Owl manager and the Owl sensor communicate by way of SSH and HTTP. If the Owl manager is protected by a firewall (either on the manager host itself or by an enterprise-level firewall) then the firewall must be configured to allow data to be transferred between the manager and the sensor. The direction of this transfer (initiated by sensor or initiated by manager) depends on the Owl configuration. These firewall modifications are far beyond the scope of this document. ### 4.3. SSH Set-up First, you must generate a new SSH key for the manager's Owl user. If the manager will be pulling data from the sensor, then the you must provide the sensor administrator with your Owl user's public key. This key will allow owl-transfer-mgr to retrieve Owl sensor data. You must generate your key in compliance with particular key requirements (e.g., length and type) specified by the sensor administrator. If the sensor will be transferring data to the manager, then the sensor administrator must provide you with the public key of their Owl sensor user. This key will allow owl-transfer to pass Owl sensor data to the your manager. You must add this key to your .ssh/authorized_keys file. The key must be generated with key characteristics (e.g., length and type) that are acceptable to you. See the example below. The authorized_keys file restricts access to known sensor hosts and controls where each Owl sensor's data are stored. This file should only have one entry per sensor, and each sensor should have a unique data directory. Example entries (with abbreviated keys):
        command="/usr/local/bin/rrsync /owl/data/sensor1" ssh-rsa AAAA...Qw== sensor@sensor1.example.com
        command="/usr/local/bin/rrsync /owl/data/sensor8" ssh-rsa AAAA...gM== sensor@sensor8.example.au
The parts of the line from "ssh-rsa " to the end of the line are the public SSH key provided by the sensor's administrator. You will add everything before the "ssh-rsa ", using the proper paths for rrsync and the sensor's data directory. ### 4.4. Configuration Settings With Owl's capability of having either the sensor or the manager transfer sensor data to the manager, there are configuration settings that must be made to allow the transfer. These are described in the subsections below. Regardless of how your Owl installation handles data transfer, you should read both subsections to ensure you are making all the required settings. It would be a good idea for you and the sensor administrator to agree on a name for the sensor. This isn't required, but it will probably make things easier for you both in the long run to refer to the sensor by the same name. The sensor name can be very generic, such as sensor42, sensor-d, or owl-us-east. It can also be very specific, such as washdc-1600-penn-ave-nw or cheltenham-bldg4. The intent is to provide distinguishing information to the intended audience of the DNS monitoring data. You should use names that easily distinguish sensors and are acceptable to the manager and sensor administrators. ### 4.4.1. Provide Configuration Information to Sensor Administrator If the Owl sensor will be transferring data to the manager, then you must provide the sensor administrator with an SSH user. All Owl sensors may use this single SSH user, and the data will be distinguished by the SSH key used to connect from the sensor. The sensor will use the heartbeat URL to provide a heartbeat to the manager. If this is to be used, it must be set on the sensor regardless of who initiates sensor data transfer. The data values will be added to the sensor's configuration file. The following example data will allow the sensor to work with the Owl manager as expected. These data are used, in conjunction with the remote keyword, in the sensor's configuration file.
Configuration Field |  | Purpose |  | Example
ssh-user |  | user on Owl manager with which owl-transfer will connect via ssh |  | sensor-ottawa@owl-manager.example.com
heartbeat |  | URL to provide "heartbeat" data to manager |  | http://owl.example.com/cgi-bin/owl-sensor-heartbeat.cgi
### 4.4.2. Receive Configuration Information from Sensor Administrator If the Owl manager will be pulling data from the sensor, then the sensor administrator must provide you with an SSH user. The owl-transfer-mgr program will use this SSH user to copy data from the sensor. The following data will allow the manager to connect to the Owl sensor as expected. These data are used, in conjunction with the remote keyword, in the sensor's configuration file.
Configuration Field |  | Purpose |  | Example
ssh-user |  | user on Owl sensor with which owl-transfer-mgr will connect via ssh |  | sensor-meatcove@owl-sensor.example.com
### 4.5. Test Transfer from Sensor At this point, you are ready to test your Owl manager. When the sensor administrator reaches section 4.5 in their installation instructions, both sensor and manager are ready to test the data transfer. Coordinate with the sensor administrator to test data transfer. The sensor administrator must put a data file (of any sort, it doesn't have to be an Owl data file) in their data directory. If the manager will be transferring data, you must run owl-transfer-mgr to attempt to retrieve the test file. If the sensor will be transferring data, the sensor administrator must run owl-transfer to attempt to retrieve the test file. After the transfer command appears to have transferred the file without error, you must check that sensor's data directory on the manager to ensure the file has arrived as expected. Once the test file successfully appears in the new sensor's data directory, the sensor is ready to start collecting data and transferring it to the manager. You must inform the sensor administrator of this, so they can start the Owl sensor daemons. ### 4.6. Wait for Sensor Data The sensor should now be in the process of collecting DNS response data. Whichever host will be transferring data must have the appropriate transfer daemon executing. You must now wait for sensor data to show up in the sensor's data directory. The time this takes will depend on how frequently the data is transferred to the manager. This is set in the configuration file. You will know when the sensor is transferring data because the new sensor's data directory will start holding files whose names reflect the queries you are expecting. You may proceed when you have found files for all the queries the sensor will be performing. ### 4.7. Build Nagios Sensor Objects Once all the sensor's data directory contains files for all the queries expected from the new sensor, Nagios objects must be built for those queries. This may be done automatically using the owl-newsensor command. A file containing these Nagios objects must be created, and it will be added to the Nagios environment in section 4.8.1. owl-newsensor can be used like this to create the Nagios objects:
        $ owl-newsensor -out sensor8.cfg /owl/data/sensor8/data
- Passing owl-newsensor the -heartbeat option will cause an object to be created that will allow Nagios to display heartbeat information about the new sensor. ### 4.8. Nagios Modifications Several Nagios configuration files must be modified to account for the new sensor. These changes will allow Nagios to start reporting current status of the new sensor as well as saving historical data to be used in graphing. ### 4.8.1. nagios.cfg The following modifications must be made to the nagios.cfg file to prepare Nagios for monitoring Owl sensor data. Add the following entries to nagios.cfg:
          cfg_file=/owl/nagios/etc/objects/owl-contacts.cfg
          cfg_file=/owl/nagios/etc/objects/owl-hosts.cfg
          cfg_file=/owl/nagios/etc/objects/owl-commands.cfg
          cfg_file=/owl/nagios/etc/objects/owl-services.cfg
- These files contain basic Nagios object definitions used by the Owl sensor objects. They should follow all the other standard cfg_file lines. Once Nagios object configuration files have been created for your sensors (as described in section 4.7), nagios.cfg must be modified to include the new files. Add a new cfg_file entry for each of the sensor object files. These entries will look something like this:
          cfg_file=/owl/nagios/etc/objects/owl-sensor21.cfg
          cfg_file=/owl/nagios/etc/objects/sensor-london.cfg
- The sensor lines should follow all four of the cfg_file lines listed in the previous point. Add the following line after the sensor cfg_file lines described in the previous point.
          cfg_file=/owl/etc/objects/owl-hostgroups.cfg
Modification of the owl-hostgroups.cfg file is described below. Find the service_perfdata_command line and change it to: "service_perfdata_command=service-perfdata-for-owl" Find the service_perfdata_file_processing_command line and change it to: "service_perfdata_file_processing_command=service-perfdata-for-owl" Examples for the cfg_file modifications may be found in nagios.cfg-owl.mods in the Owl manager distribution. ### 4.8.2. owl-hostgroups.cfg This file contains the "DNS Response Time Sensors" Nagios hostgroup object that lists the Owl sensor hosts. All your Owl sensors should be listed in the "DNS Response Time Sensors" object. Add the new sensor name to the members field in the "DNS Response Time Sensors" object. The names must be separated by commas. You may add your own hostgroup objects to this file. Use the "DNS Response Time Sensors" object as a guide to create custom groups. This can be used, for example, to group all the sensors in a geographical region or those that are running a particular operating system. The hostgroup object allows you to group a set of hosts in a manner that makes sense for your purposes. ### 4.9. Graphing Modifications The new sensor's data must be made available to the drraw.cgi script. To do this, the new sensor's data sources must be added to the %datadirs hash in the drraw.conf file. See 2.1.2.4. drraw.cgi for more details. ### 4.10. Restart Nagios Nagios must be restarted in order for it to see your new sensor's objects. Prior to the restart, verify that your object modifications won't cause a problem for Nagios. Execute this command (assuming you are in the directory containing the Nagios files):
        $ nagios -v nagios.cfg
It will read the configuration files and ensure they all look okay. If there are problems, they must be resolved before Nagios is started. Once the Nagios configuration file is validated and without problems, Nagios may be restarted:
        $ nagios stop
        $ nagios start
It will probably take several minutes for Nagios to check all its services. Clicking on the "Services" link in the left-hand sidebar will bring up the configured services and you can watch the status of your new sensor. ### 5. Defining Graphs After Owl sensor data has been gathered for several hours and rrdtool has been adding it to the RRD databases, you can consider how you will graph the data. Graphing capabilities are provided by nagiosgraph and drraw.cgi. Both are very useful and have their place in your Owl monitor. It isn't a question of which to use; the question is how you want to use each and how you want to display your monitoring data. ### 5.1. Graphing with nagiosgraph nagiosgraph provides menus for easy selection of data, available through three CGI scripts. These are simple graphs that only display a single sensor/root server/DNS record per graph. For these scripts, the host is an Owl sensor and the service is a root server/DNS record type. Links for these scripts are included in the example-side.php file. show.cgi - Provides graphs for a user-selected host/service pair. A graph is provided for the current day, week, month, and year. showhost.cgi - Provides graphs for a user-selected host. A graph will be displayed for any service with that host. A graph is provided for the current day, week, and month. showservice.cgi - Provides graphs for a user-selected service. A graph will be displayed for any host with that service. A graph is provided for the current day, week, and month. ### 5.2. Graphing with drraw.cgi drraw.cgi provides a much more complex graphing capability than is available through nagiosgraph. It allows data from multiple sensor/root server/DNS record groups to be displayed on a single graph. Set-up for the graphs is not difficult and is done through a browser-based interface. However, there are a number of things that must be specified before the graph can be displayed. The graph-definition webpage talks about selecting data sources to be displayed. This is referring to the group of RRD databases you want drraw.cgi to use for this particular graph. Defining simple graphs, even those with multiple data sources, is not complicated. However, you can define fairly involved graphs (e.g., averaging data from different databases) with drraw.cgi. ### 5.3. Adding Graphs to the Nagios Sidebar The share/side.php script builds the left sidebar of links on the Nagios page. Several simple modifications will improve access to the graphing facilities. Obviously, this step must wait until after the system has been set up and site-specific graphs have been defined. The example-side.php file (in the Owl manager distribution) contains sample lines for both drraw.cgi and nagiosgraph. Add the contents of this file to the share/side.php file after the "Quick Search" text-entry box. The sample lines must be modified for your installation in the following ways: The paths in the links must match the way you installed it. If links to specific drraw.cgi graphs are to be included in the sidebar, then the graph id in the link must be set. The id for a particular graph may be found from the graph's URL, and is the field following "Graph=". ### 6. Changing Queries on Existing Owl Sensors From time to time, you might decide to change the queries being performed by the Owl sensors. This could be adding new queries to some or all of the sensors, stopping some of the queries from being performed, or modifying existing queries. Regardless of the changes, there are actions that must be taken on the Owl manager and all affected Owl sensors. ### 6.1. Adding New Queries Adding new queries to the Owl manager is a matter of informing Nagios that it should report on new data. This is accomplished by adding new Nagios objects for the sensor. The following actions must be performed: Wait for the query changes to take place in the Owl sensor. You will know this has happened when new sensor data files start appearing in the sensor's data directory, if you are adding queries, or fewer data files are arriving, if you are removing queries. Build new Nagios objects for the new queries' data files. This may be most simply accomplished by rerunning owl-newsensor for the affected sensor. This will rebuild all the sensor's Nagios objects, so you must take this into account if you have made manual changes to the object file. See section 4.7 and section 4.8 for more information on creating new Nagios objects. Perform the actions in section 4.10. in order to restart Nagios. With sensor query data arriving, you might want to consider modifying your graphs to display the changes in sensor data. See section 5 for information on defining graphs. ### 6.2. Deleting Old Queries Deleting existing queries to the Owl manager is a matter of informing Nagios that it should stop reporting on existing data. This is accomplished by deleting Nagios objects for the sensor. The following actions must be performed: Edit the Nagios object files to delete the objects for the unwanted queries. See section 4.7 and section 4.8 for more information on Nagios objects. Perform the actions in section 4.10 in order to restart Nagios. After performing these steps, the old data files will still exist on the manager. They can be archived or deleted as desired. ### 7. Owl Manager Commands The Owl manager has a number of Owl-specific commands required for its operation. This section provides a brief overview of the commands available to it. These are all Perl scripts and have self-contained man pages, which contain greater details about their use. These scripts are intended to run as an unprivileged user. This section briefly describes the software used on the Owl manager. owl-archdata This script archives Owl sensor data for all the defined sensors. Old data are moved from the Owl data directory to an archive directory. owl-archdata runs the owl-dataarch-mgr script for each sensor. This script is assumed to live in the Owl user's home or bin directory. owl-archold This script archives previously archived Owl sensor data into monthly archives. A compressed tar file is created that contains the archived data files from a particular month. It should only be run on a particular month's archive after the month has been completed. This script is exactly the same as the owl-archold on the Owl sensor hosts. owl-archold is executed by owl-monthly, but may be used on its own. This script is assumed to live in the Owl user's home or bin directory. owl-dataarch-mgr This script archives Owl sensor data for a single sensor. Old data are moved from the Owl data directory to an archive directory. owl-dataarch-mgr is executed by owl-archdata, but may be used on its own. This script is assumed to live in the Owl user's home or bin directory. owl-dnswatch This script retrieves Owl sensor DNS response data and makes it available to the Nagios monitoring system. This script should live in the Nagios libexec directory of plugins. owl-initsensor This script creates the required directories and heartbeat file when adding a new Owl sensor. This script is assumed to live in the Owl user's home or bin directory. owl-monthly This script archives previously archived Owl sensor data for all the sensors defined for an Owl manager. Compressed tar files are created that contain the archived data files from a particular months, one tar file for each sensor. owl-monthly executes owl-archold to perform the actually archiving. This script is assumed to live in the Owl user's home or bin directory. owl-newsensor This script creates required Nagios objects for a new Owl sensor. The objects to be created are based on the data files from the new sensor that have arrived on the Owl manager. This script is assumed to live in the Owl user's home or bin directory. owl-perfdata This script is translates Owl sensor data from the format provided to Nagios into the format required by the graphing and database software. This script should live in the Nagios libexec directory of plugins. owl-sensor-heartbeat.cgi This script records a "heartbeat" for Owl sensors. When a sensor runs this script (by way of the Owl manager's web server), the time of the execution will be recorded for that sensor. This script should live in the cgi-bin directory for either the web server or for Nagios. owl-stethoscope This script gathers Owl sensor "heartbeat" data and makes it available to the Nagios monitoring system. This script should live in the Nagios libexec directory of plugins. owl-transfer-mgr This script periodically transfers Owl sensor data from the sensor host to the manager host. This script is assumed to live in the Owl user's home or bin directory. ### Sponsors: This work was implemented by the following organizations: Parsons This work is funded in part by the following organizations: U.S. Department of Homeland Security/Science & Technology (S&T) The Owl logo is copyright © 2012 by Wayne Morrison (Wayne@WayneMorrison.com). All Rights Reserved. This logo is used for the Owl Monitoring System by the DNSSEC-Tools Project (dnssec-tools.org) by permission.
| Owl Monitoring System Full Installation Manual |
DNSSEC Tools