Creating Reuseable Perl Scripts

Using An OO Model

Of The ClearCase Region

Author: Leslie D. Paul

# 1. Introduction

Many companies create ClearCase Perl Scripts to expand and enhance the core functionality of the ClearCase system. Scripts can be created to generate reports, perform administrative functions, update ClearCase attributes, and deploy releases. The development and maintenance of these scripts can consume hundreds of hours. For example, scripts that contain hard coded lists of VOB names can not be reused in another Region without editing the script.

This paper presents an approach to designing Perl Scripts for reuse across regions and platforms. The design approach is to create an OO Model of the ClearCase Region defined in a module, and then to build Perl Scripts on top of the Region Object (module).

The reader should be familiar with Perl Programming, OO Modeling and ClearCase. For OO Perl Programming, the reader is referred to the online Perl documentation (execute perldoc perltoot at the command line) and to the O’Reilly (Panther) book entitled “Advanced Perl Programming” by Sriram Srinivasan and to Damian Conway’s book “Object Oriented Perl”.

# 2. Object Oriented Concepts

For those not familiar with Object Oriented Perl Programming, here are some of the key benefits and terminology.

Data is encapsulated and protected within objects. The object’s data is known as attribute values. The object’s subroutines that control access to the attributes are called methods. An object has the following:

· Identity or name

· Attributes

· Methods

A class is a formal specification for creating instances of objects. The class defines the allowed attributes names and the methods that are used within an object to access or update attribute values. Classes within Perl are defined in a separate text file called a module. A class is a Perl module that follows some simple rules (see ‘perldoc perlobj’ for more details).

The encapsulation of data within an object is one of the key benefits of OO programming. Many programs can share the same object, but the data within the object cannot be accessed directly. Instead of using global variables, OO programs rely on an Object to be the single point of control for attribute values.

# 3. A Model for ClearCase

A OO model could be created for ClearCase. Here one:

A region is composed of vobs. Each vob can have many views associated with it. A Vob consists of Elements. A Element has one or more branches (assumes a main branch). A branch consists of one or more versions.

Since the objective of this paper is to show a reuseable design approach – only the ‘region’ and ‘vob’ classes are considered here. Using just the Region and Vob classes, the approach to creating reuseable Perl scripts can be illustrated. We’ll start by defining what a region is.

# 4. Defining a Region

Careful thought should be given to the task of defining a Region. The tools and design approach presented here works best when Region(s) are clearly defined. The design and organization of a corporate network is closely tried to the concept of Region. Corporate networks involve three essential structural components: Domains(s), Organization Units (OU), and Site(s).

A domain is a logical grouping of computers for administrative and security purposes. A domain is a logical grouping that is not necessarily based on the physical location of computers. Hence, a domain can hold computers located in multiple sites, or buildings, or just a single campus. For geographical dispersed computers, a WAN or LAN is used to provide network connectivity.

A single domain can be sub-divided into Organization Units. A company with a single domain may have separate Organization Units or Divisions for Marketing, Sales, Research, Manufacturing and Distribution. Each Division may use a WAN/LAN for interconnecting multiple sites.

A ClearCase region is a conceptual subset of the hosts of a single network domain based on a logical grouping. A region could be an entire domain. A region could be defined for each Organization Unit within a domain. A region could be defined for each site within an Organization Unit or Division. In fact, a region could be defined for a single computer used by a single group.

Splitting a large Region (one with many VOBs) into two or more Regions will make the Region class execute faster – since a region’s data is cached when the Region Class is loaded by a Perl Script.

# 5. The ClearCase Region

A Region is a logical subset of a local area network where all hosts refer to all VOB and View storage areas through the same network pathnames. All hosts in a region will use the same global pathnames when referring to shared resources.

All ClearCase servers must reside in a region that is defined at the time of ClearCase installation.

To identify the region a server belongs to, execute:

Ø cleartool hostinfo -long

Client: Chicago

Product: ClearCase 5.0

Operating System : SunOS 5.6 Generic

Registry host: Chicago_sun50

Registry region: credit_unix_region

License host: Chicago_sun51

It’s possible to list all Regions defined in the ClearCase Registry using the command:

Ø cleartool lsregion

credit_unix_region

market_unix_region

To identify the region on a Unix host:

Ø cat /var/adm/atria/rgy/rgy_region.conf

credit_unix_region

To identify the region on a Windows host, open up the Registry tab of the ClearCase program in Control Panel.

A VOB is a ClearCase repository associated with a single Region. A VOB cannot span more than one region. By running a long listing of a VOB, you can display the region associated with a VOB:

Ø cleartool lsvob –long /vobs/credit

Tag: /vobs/credit

Global path: \\chicago_sun51\vobstore\credit.vbs

Server host: Chicago_sun51

Access: public

Mount options:

Region: credit_unix_region

…

By knowing the region associated with a server, you also know the set of available VOBs. To list all VOBs for a given region:

Ø cleartool lsvob –region credit_unix_region

/vobs/credit

# 6. A Layered Approach to Code Reuse

The following diagram illustrates the layered approach to the design of reuseable Perl scripts:

A Perl script can be designed so that it is completely reuseable between sites. The reuseable Perl script uses one or more instances of Objects created from a Perl Class. The Perl Classes are mostly reuseable. However, at the time the class library is installed, some minor editing may be required. For example, the location of the Configuration Files has to be added to Class.

The Region Class reads a configuration file, region.conf, that contains data for all known regions. Hence, the region.conf file is site specific.

In addition, the Perl scripts and modules presented here, use the Log::Log4perl class module available from CPAN. The Log4perl class provides an OO log management system. The Log4perl class is designed after the popular Log4j class. All Perl scripts and objects presented here will be available at the www.cpan.org web site (search for ClearCase::Region).

# 7. A OO Model of the ClearCase Region

A region consists of zero or more VOBs. This composition can be represented by the UML diagram:

This Rose model will be used to illustrate a reuseable Perl Script Design approach.

An OO model of the ClearCase Region is dependent on the network configuration, the ClearCase installation and policies. However, the specific network configuration and attributes (like region names) can be loaded into the Region class using a text file. This text file, called Region.cfg, will be read by the Region class when the class is first loaded by a calling program.

As an example, let’s consider a simple ClearCase network configuration. In this scenario, a region contains one or more VOBs. Development projects also use one or more VOBs, and all VOBs are within the same region. Also, development projects never shared the same VOB. Hence, development projects form a ‘subregion’. Within each VOB, a simple branch per release policy is used.

Here is a sample Region.cfg for this scenario:

#*****************************************************************

# Region.cfg

#*****************************************************************

# This is the configuration file that defines ClearCase region(s)

#*****************************************************************

#

# This file is read by the Region.pm module. For each region, the

# values are read and loaded into a hash with the following keys:

# subregion, description, vobs, vobdir. All of these values must

# be defined in this file.

#

# The file may contain the configuration information for multiple

# regions. Configuration information for each region must start

# with ['region_name']. All text starting with a '#' character up

# to the end-of-line are comments.

#

# NOTE 1: The keys (subregion, description, vobs, vobdir)

# must be defined in this file in lower case and listed

# in order for every record.

#

# NOTE 2: A Blank Line must terminate the subregion record (of

# keys: subregion, description, vobs, vobdir).

#

# NOTE 3: The subregion key must be the first entry listed for

# each record and uniquely defines a project within a

# region.

#

# NOTE 4: All string values must be quoted with single quotes '.

#

# NOTE 5: All array values must be specified between brackets

# (i.e. []).

#

#*****************************************************************

['market_unix_region']

subregion 'ma' # subregion id for a project

description 'Market Analysis' # project description

vobs [ 'current', 'futures' ] # array of all vobs

vobdir '/vobs'

subregion 'tr' # subregion id for a project

description 'International trades' # project description

vobs [ 'trades' ] # array of all vobs

vobdir '/vobs'

['credit_unix_region']

subregion 'cr' # subregion id for a project

description 'Credit' # project description

vobs [ 'credit' ] # array of all vobs

vobdir '/vobs'

The Region.cfg file contains the ‘rules’ for entering the data. The Region class calls a simple ‘parser’ that reads and parses the Region.cfg file when the class is first loaded. The parser is contained in a helper class, Region_Cfg_Parser.pm, whose only purpose is to create a hash object for the Region class. The ClearCase::Region_Cfg_Parser->new() method reads the region.cfg files, parses it, and returns a hash reference. So, for ‘market_unix_region’, the new() method returns a hash reference for a hash with two keys: ‘ma’ and ‘tr’. The associated values of these keys are the subregion records (stored in another hash reference).

The Region_Cfg_Parser module locates the Region.cfg configuration file by searching all the directories in the @INC array. When ClearCase::Region is downloaded from CPAN, the ‘default’ configuration file listed above is installed among with the modules. The Region.cfg file should be modified to reflect your company’s ClearCase configuration.

The Region_Cfg_Parser uses the name-value pairs in the Region.cfg configuration file for creating the internal hash structure. This makes it easy to add an attribute. For example, by adding the line:

Registry_server ‘Chicago_sun66’

after each vobdir line, the new attribute becomes part of the region hash – without any changes to the parser. This can be represented as:

Of course, a new accessor method would have to added to Region.pm for this new attribute.

Also, the Region class checks to see what ClearCase Region it is operating under. Under Unix, the Region class opens the file ‘/var/adm/atria/rgy/rgy_region.conf” to obtain the region name. Under MS-Windows, the Region class opens Registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Atria\ClearCase\CurrentVersion\InteropRegion

to determine the ClearCase region.

So, a program that uses Region class data for in the ‘market_unix_region’ sees that only the VOB names “current”, “futures”, and “trades” are valid. The same program operating in the ‘credit_unix_region’ sees that only the VOB name ‘credit’ is valid.

The Region constructor, new(), is used to create the Region object. Actually, the new() constructor creates a ‘subregion’ object having been passed a valid subregion name. So, how do we determine what subregion names are valid in any given region? For interactive Perl scripts, you can invoke the new() constructor as:

$region = Region->new(undef);

If the new() constructor get an undef value for the subregion name, it will provide a list of all valid subregion names and prompt the user to choose one. Here’s an example of passing ‘undef’ to the new constructor in the market_unix_region:

Did not get subregion for project. Must ask...

Please select project:

1) International trades uses trades with project id = "tr"

2) Market Analysis uses current futures with project id = "ma"

Enter 1, 2, ... 2): 1

Another approach to getting a valid subregion name is to first obtain a list of all valid subregion names (for the region your operating in), and then use the Getopt::long Perl Module to allow the user to ‘pass’ the choice on the command line.

use Getopt::Long;

Then the program can obtain a list of valid subregion names:

my(%options ) = (); # command line parameters

my($i ) = 0;

my($subregion ) = "";

my($subregion_name ) = undef;

my(@subregions ) = Region->subregions();

my(@subregion_flags ) = ();

Then set up the options hash (that is used by getoptions):

$i = 0;

foreach $subregion (@subregions) {

$subregion_flags[$i] = 0;

$options{$subregion} = \$subregion_flags[$i];

$i++;

}

and then call getoptions:

$retval = GetOptions(%options);

if ($retval == 0) {

$logger->error("Error in parameter specification.\n");

exit 1;

}

If the calling program has set one of the subregion options on the command line, then the %options hash will have the corresponding member of the subregion_flag array set. Of course, if an invalid subregion name is passed on the command line, the GetOptions program will now respond with an error – based on the list of valid subregion names obtained from the Region class.

The $subregion_name was initialized as ‘undef’. If the user passed the subregion name as a command line option (like –tr or –ma), then this can be determined using:

#

# Determine if the subregion was set on the command line

#

$i = 0;

foreach $bno (@subregion_flags) {

$sum += $bno;

if ( $bno > 0 ) {

$subregion_name = $subregions[$i];

}

$i++;

}

Next, we check that no more than one option was passed on the command line:

#

# Determine if more than one subregion was set on command line

#

if ($sum > 1) {

$logger->error("only one region option can be specified.\n");

exit 1;

}

If two or more options were passed on the command line (e.g. both –tr and –ma are passed), then the Region new constructor fails will an error message. Calling ‘new’:

$region = Region->new($subregion_name);

will create one subregion object. If no subregion is selected on the command line, $subregion_name is undefined and the Region class will ‘prompt’ for one (see the Region_test program example below).

The Class method, subregions(), can be used to retrieve a list of all the subregions in a region. To retrieve all the Vob names in the current region, the following Perl code fragment could be used:

my(@subregions) = Region->subregions();

foreach $subregion (@subregions) {

@vobs = Region->vobs($subregion);

$logger->info("\nFor subregion name \'$subregion\' all vobs are: @vobs\n");

}

This code was used in the ‘Region_test’ script that will be available on CPAN, along with the Region.pm class module. When the Region_test script is execute in the ‘market_unix_region’, it produces the following on in my MS Command Prompt Window:

> perl Region_test

Current region is market_unix_region

The subregions are tr ma

For subregion name 'tr' all vobs are: trades

For subregion name 'tr' the VOB directory is: /vobs

For subregion name 'ma' all vobs are: current futures

For subregion name 'ma' the VOB directory is: /vobs

Got tr : 0

Got ma : 0

Did not get subregion for project. Must ask...

Please select project:

1) International trades uses trades with project id = "tr"

2) Market Analysis uses current futures with project id = "ma"

Enter 1, 2, ... 2): 1

Created Region object for tr

>

# 8. Sample program using the ClearCase Region

The sample program, main_cos (available from CPAN) demonstrates the use of the Region class. The main_cos program is used to identify any checked out revisions on the trunk (or main branch). The main_cos program expects to be passed one subregion name on the command line:

main_cos –tr # OR

main_cos –cr

However, if the subregion name option is omitted, the Region Class will prompt:

Please select project:

1) International trades uses trades with project id = "tr"

2) Market Analysis uses current futures with project id = "ma"

Enter 1, 2, ... 2): 1

So, the main_cos script can be reused in different regions and subregions because the Region class encapsulates the relevant data. The main_cos script uses the variable ${vobdir) = Region->vobdir() to define the VOB storage directory location for all VOBs.

# 9. Using the Region Class

The Region.pm class module and all the associated sample scripts referenced in this paper are available from CPAN. The Region module is not a ‘generic’ class that can be applied to every site, so some customization will be required. The Region module assumes that a branch per release version control policy is being used and hence, contains methods that are most useful when using branch per release management. Nevertheless, the Region class is a great starting point for building reuseable objects within your company – and then layering (or building) reuseable Perl scripts on top.

Include the line:

use ClearCase::Region;

in Perl scripts that will be using the Region.pm module. This module must either be installed in one of @INC directories, or specified in one of the $PERL5LIB directories.

The Region.pm class module and the associated sample scripts (Region_test, co_backup and main_cos) all use the Log::Log4perl Class. Log::Log4perl is freely available from CPAN and SourceForge (http://log4perl.sourceforge.net). The Log::Log4perl module requires the Log::Dispatch bundle as well. If you don’t have Log::Log4perl, use the CPAN module or PPM to upgrade your Perl environment prior to using these programs.

# 10. A OO Model of the ClearCase Vob

The following Rose model is an OO model of the ClearCase VOB:

The sample script, Vob_test can be used to test the Vob object. Running Vob_test produces:

>perl Vob_test

All known vobs: trades current futures

Did not get flag for vob. Must ask...

Please select vob:

1) trades

2) current

3) futures

Enter 1, 2, ... 3): 3

You have chosen futures. Is this CORRECT?(y/n): y

You answered Y

Selected vob is futures

Vob subregion via class method is ma

Vob subregion via object method is ma

src_dir is "source"

lib_dirs is "lib,class"

relnums is "1,2,3,4,5,6"

active is "0,0,0,1,1,1"

active releases: "4 5 6"

Like the Region Class, the Vob class also uses a help class, Vob_Cfg_Parser.pm, for the task of parsing the Vob.cfg file. Here is the Vob.cfg file that is being used here:

#*****************************************************************

# Vob.cfg

#*****************************************************************

# This is the configuration file that defines ClearCase vob(s)

#*****************************************************************

#

# This file is read by the Vob.pm module. For each region, the

# values are read and loaded into a hash with the following keys:

# vob, src_dir, lib_dirs, relnums, active. All of these values must

# be defined in this file for each [region subregion] section.

#

# The file may contain the configuration information for multiple

# vobs. All configuration information for all vob located in a

# subregion must start with [region subregion] section header. All

# text starting with a '#' character up to the end-of-line are

# comments.

#

# NOTE 1: The keys (vob, src_dir, lib_dirs, relnums, active)

# must be defined in this file in lower case and listed

# in order for every record.

#

# NOTE 2: A Blank Line must terminate the vob record (of keys:

# vob, src_dir, lib_dirs, relnums, active).

#

# NOTE 3: The 'vob' key must be the first entry listed for each

# record.

#

# NOTE 4: The first number identified in the relnums array musts

# be the trunk.

#

# NOTE 5: The active releases array uses a '1' to identify

# releases that are active. There must be a one-to-one

# correspondence between the relnums array and the

# active array.

#

# NOTE 6: All string values must be quoted with single quotes '.

#

# NOTE 7: All array values must be specified between brackets

# (i.e. []).

#

#*****************************************************************

[market_unix_region ma]

vob 'current'

src_dir 'servlet/source'

lib_dirs ['servlet/lib', 'servlet/class']

relnums [ 1, 2, 3, 4, 5, 6 ] # all release numbers

active [ 0, 0, 0, 1, 1, 1 ] # array of active releases

vob 'futures'

src_dir 'source'

lib_dirs ['lib', 'class']

relnums [ 1, 2, 3, 4, 5, 6 ] # all release numbers

active [ 0, 0, 0, 1, 1, 1 ] # array of active releases

[market_unix_region tr]

vob 'trades'

src_dir 'src'

lib_dirs ['bin']

relnums [ 1, 2, 3 ] # all release numbers

active [ 1, 1, 1 ] # array of active releases

[credit_unix_region cr]

vob 'credit'

src_dir 'src'

lib_dirs ['bin']

relnums [ 2, 3, 4, 5 ] # all release numbers

active [ 1, 1, 1, 1 ] # array of active releases

The rules for setting up the Vob.cfg file are defined in the comments. Adding an additional attribute is easy. For example, the Group Id for each Vob could be added with the line:

group_id ‘bankofamerica’

after the ‘active’ line for each Vob record. The ClearCase::Vob_Cfg_Parser->new() method will then add the key, group_id, to the hash reference to class Vob, as shown here:

Of course, the Vob.pm class would have to be modified to include an accessor method as well.

# 11. Sample program using both the ClearCase Region and Vob

The sample program, co_backup (available from CPAN), demonstrates the use of both the Region class and the Vob class to create a reuseable Perl script. The co_backup Perl script creates a compress cpio backup of all checked out files in all dynamic views. Hence, the co_backup script can be used to back up files that are being worked on by developers (in dynamic views) – without having to back up the entire view.

In addition, this script only cares about the ‘active’ releases (or views). The co_backup script uses the Vob object to determine which releases (branches) are active for a VOB. The co_backup script assumes that all developer’s views follow a naming convention “<login>_<subregion>_<relno>”. Then by starting in the /view directory, all developer’s views can be listed using:

cleartool lsview –short ‘*_${subregion}_${relno}’

Of course, if your company has a different naming convention, then this line will need to change. The co_backup script will start the developer’s view (if required). The list of active releases is used to determine which developer’s views to back up. So, for the “market_unix_region” listed above, only the developer’s views:

‘*_tr_1’

‘*_tr_2’

‘*_tr_3’

‘*_ma_4’

‘*_ma_5’

‘*_ma_6’

will be backed up (provided there is some checked out files). When the co_backup program is executed on the ClearCase server in the credit_unix_region, the developer’s views:

‘*_cr_2’

‘*_cr_3’

‘*_cr_4’

‘*_cr_5’

will be backed up. Hence, the co_backup script is reuseable in different regions. If the co_backup script encounters an error, like a bad view, it logs the error message to:

$HOME/logs/${subregion}${relno}.${vob}.backup.log

using the class Log::Log4perl.

# 12. Summary

A Perl script can be designed so that it is completely reuseable between sites. The reuseable Perl script uses one or more instances of Objects created from a Perl Classes. The Perl Classes are mostly reuseable. For the co_backup script, this can be represented as:

The design of the Perl Classes depends on the network topology, defined Regions, defined subregions or projects and the version control policy being used. This paper presented a simple network configuration using a ‘branch per release’ version control policy is illustrate the design of ClearCase classes for Region and Vob.

Using the Region and Vob classes, many Perl scripts can be created that are reuseable between sites, platforms and even companies. The sample scripts provided with this paper can be used to start building your own reuseable Perl Scripts.