Mangement and Operations of the EBONE
Bernhard Stockman
February 25, 1993
V 0.3
Abstract
This paper defines the EBONE Operations Team,
its local and central functionalities and
corresponding responsibilities with the goal of
making the EBONE a highly stable network in
terms of management and operations.
1. Introduction.
EBONE is formed by its members as defined in the EBONE Memorandum of
Understanding. EBONE is a pan-European international networking
service providing international backbone service and
intercontinental neutral access service to its members. Connectivity
will be offered between EBONE member networks and the global
Internet.
The EBONE Mangement Committee (EMC), nominated by the EBONE members,
has the overall responsibility for the EBONE. The EMC nominates the
EBONE General Manager (EGM) having the daily administrative
responsibility of the EBONE service.
The EBONE Action Team (EAT) has the responsibility for the
engineering and development of the EBONE. The EBONE Action Team
consists of technical representatives nominated by the EBONE
members.
The EBONE Operations Team (EOT) has the responsibility of the
management and operations of the installed EBONE.
This document describes the EBONE Operations Team. For more
information on the EBONE contact the EBONE Secretariat.
(E-mail: ebone@rare.nl, tel: +31 20 639 1131)
2. EBONE Operations Team
2.1 Definitions and Responsibilites
Each site installing an EBONE Boundary System (EBS) according to the
EBS Technical Specification and the EBONE-92 Implementation plan is
referred to as an EBS site.
Each site having installed an EBS must have exactly one designated
EBS manager. It is recommended that a deputy EBS manager be
designated.
In order to coordinate the management of EBONE an EBONE Network
Operations Center (NOC) is established as defined below.
The management and operations of the EBONE is undertaken by the
EBONE Operations Team (EOT). The EOT is formed of the set of EBS
managers and the EBONE NOC.
To be able to undertake the task of coordination and fullfil other
defined responsibilities priviliged access to each EBS shall be
given to that EBS manager and to the EBONE NOC.
EBONE kernel configurations may not be changed by any other parties
or persons than the EBONE Operations Team as defined above. Changes
can only happen
- In normal cases after a request for a configuration change
issued by EAT and sent to EOT for implementation.
- In an emergency situation by EOT after a trouble ticket
have been issued to the relevant sysfail lists describing
the emergency and the needed action.
With EBONE configuration is here ment the EBONE core line and EBS
installations, traffic forwarding configurations such as addressing
and routing configurations, management configurations and other
basic configurable elements of the EBONE kernel.
The EBS manager and the NOC will liaison closely about the
operational state of the EBS. The NOC shall respect the executive
authority of EBS sites carefully. The EBS manager shall perform
actions requested by the NOC in a timely manner.
3. The EBONE Network Operation Center (EBONE NOC).
3.1 Specification.
The EBONE Network Operations Center has the overall responsibility
for the coordination of management and operation of the EBONE. The
NOC will coordinate all EBONE operational activities of the EBS
sites.
To fulfill this task necessary monitoring equipment and staff shall
be installed at the site providing the EBONE NOC.
The EBONE NOC personnel shall consist of the EBONE NOC manager and
EBONE NOC personnel. Sufficient personnel resources shall be
employed to fulfill the EBONE NOC responsibilities. For a detailed
job description of the EBONE NOC manager see appendix A.
The EBONE NOC reports to the EAT and EOT.
3.2 Specification of responsibilities.
The EBONE NOC is responsible for that the EBONE backbone behaves
according to the EBONE specification and is reliable.
The NOC is responsible for optimal traffic flow on EBONE.
The NOC is responsible for fault reporting, isolation and repair.
The EBONE NOC has the overall responsibility for changes in the
EBONE kernel configuration.
Any problems in undertaking these responsibilites for technical
reason shall be referred to the EAT. Any problems in undertaking
these responsibilites for administrative reasons shall be referred
to the EMC, specifically as represented by the EGM
3.3 Specification of EBONE NOC tasks
3.3.1 Mangement and Operations of the installed equipment
Normal daily operations such as
- Monitoring of EBONE equipment installed at the EBONE NOC,
- Monitoring of EBONE kernel reliability and performance,
3.3.2 Mangement and Operations of the IP service.
The management and operations of the IP service includes
- Monitoring of routing according to the routing plan,
- ensuring correct configurations are installed in all EBS routers
3.3.3 Fault management
- Issuing trouble tickets to relevant sysfail-lists in
case of faults in EBONE
- Initiation and coordinating fault repair,
- Keeping track of fault conditions,
- Reporting status of faults and problems to interested parties,
- Maintenance of logs of fault reports.
3.3.4 Operational support for EBS installations and other connected
organizations.
The EBONE NOC shall collaborate with the EBS managers to maintain
the EBONE services.
A list of EBS managers shall be maintained for easy and fast contact
with the EBS installations.
RBS operators shall have the possibility of contact the EBONE NOC
directly when there is no operators available at the relevant EBS
site. The EBONE NOC shall be possible to contact via email, phone
and fax.
It is recommended that the EBONE NOC has 7*24 operational coverage
to which the designated EBS operators may report faults. Fault
resolution will be guaranteed during local office hours and a
reasonable effort will be made at other times.
3.3.4 Monthly EBONE reports.
The EBONE NOC will have the responsibility of producing monthly
reports and to distribute these reports to each of the EBONE member
organizations. It is also recommended that these reports are being
distributed to other interested parties. The reports shall also be
made generally available using "anonymous ftp".
The monthly reports shall contain at least:
- Total input and output byte and packet at each interface for each
EBS.
- If possible per protocol and per application byte and packet shown
both as totals and as percentages.
- Availability statistics for each EBONE core line and each EBS
installation.
- Other significant events that affect the EBONE core.
The EBONE NOC shall receive montly reports from each EBS site no
later then two weeks after change of month to be able to inlcude
these in the EBONE monthly report.
The NOC shall coordinate statistics gathering with the RIPE NCC in
order to prevent redundant collection of statistics and to ensure
that the collected statistics are useful outside the scope of EBONE
as well.
3.3.5 EBONE development and service enhancements
The EBONE Network Operation Center shall in close collaboration with
the EMC and the EAT recommend, review and supervise changes and
additions to the EBONE kernel, regional and intercontinental
connections. For this reason the EBONE Network Operation Center
will have regular meetings with the EBONE Operations Team and the
EBONE Action Team.
4. EBONE Boundary System Sites
4.1 Definition
Each site installing an EBONE Boundary System (EBS) according to the
EBS Technical Specification and the EBONE-92 Implementation plan is
referred to as an EBS site.
Each site having installed an EBS must have exactly one designated
EBS manager. It is recommended that a deputy EBS manager be
designated.
4.2 Responsibilities
The task of the EBS manager is to assure the integrity and the
functionality of the EBONE defined services.
The EBS manager is responsible for local the operation of his EBS
installation and acts as an contact point between the EBONE NOC and
the EBS installation and as a contact point between the EBS
installation and regional connections to that EBS.
4.3 Specification of tasks
4.3.1 Mangement and Operations of the EBONE.
The EBS manager shall:
- Monitor load, response times and services interruptions on the
backbone and on regional connections to the backbone
- See to that the services are maintained according to the EBONE
service requirements.
- Assist the EBONE NOC in resolving problems.
- Advise the EBONE NOC on needed technical changes.
- See to it that correct configurations are being installed in the EBS.
- Maintain operational contacts with the regional networks connected
through his EBS.
- Provide the EBONE NOC with EBS montly report no later than two
weeks after change of month for inclusion in the EBONE monthly
report.
4.3.2 Collaboration with other EBS installations
The EBS manager shall collaborate with other EBS managers as needed
to resolve problems.
It shall be possible to contact each EBS manager via phone, fax and
email during local office hours.
It is recommended that the EBS manager establishes a clear,
documented contact and escalation procedure for operational
problems. To the extent possible this procedure should also cover
non office-hours.
5. EBONE Regional Boundary System responsibilities.
5.1 Specification
The EBONE Regional Boundary System is a installation directly
connected to one or more of the EBS's.
The RBS connects regional networks to the EBONE.
5.2 Responsibilities
The RBS installation shall have a nominated contact point for
problem resolution.
When a problem is discovered at a RBS site which is believed
originate in the EBONE kernel the relevant EBS site should be
contacted. If this does not succeed the EBONE NOC shall be
Regional networks have the responsibility of see to that only
traffic from EBONE connected sites are injected into the EBONE.
Problems discovered by users in a regional networks should be
reported to relevant operators within that network. Problem
reporting procedures should ensure that user complaints are not
being forwarded directly to the EBS manager or to the EBONE NOC.
6. Problem resolving procedures.
To aid the EBONE Operations Team in the resolving of problems in the
EBONE, a list of contact persons for the EBONE Operations Team shall
be maintained by the EBONE NOC. This list shall only be known to the
EBONE Operations Team and nominated responsible persons at RBS
installations.
6.1 Problem detected in the EBONE kernel.
The EBONE NOC is informed as well as all EBS sites. The NOC is then
responsible to coordinate problem resolution and inform interested
parties as required.
If the EBONE NOC is not available the EBS site first detecting the
problem will coordinate problem resolution and inform interested
parties as required until the NOC becomes available. This EBS site
will state clearly that it coordinates the problem until it can pass
the responsibility to the NOC.
When the EBONE NOC have been informed and have taken the
responsiblity for coordination of fault isolation and correction the
tasks described under 3.3.3 above shall be undertaken by the EBONE
NOC.
Depending on the nature of the problem corrective actions should be
immediately started by either the EBS site detecting the problem or
by the EBONE NOC.
In case of link failures EBS sites should
1) inform the NOC
2) contact the peer EBS sites and cooperate in determining the
cause of the failure
3) inform the carriers (or the MUX NOC) as required
It is essential to coordinate with the peer EBS site before
contacting the carriers. If the peer EBS site cannot be contacted
the EBS site and the NOC will try to diagnose the state of the EBS
at the remote site before initiating further action.
The NOC and EBS sites shall liaison closely and frequently about any
actions taken during resolution of failures.
6.2 EBONE related problems detected at a RBS site.
The RBS site shall first try to verify that the detected problem is
related to the EBONE. If so the relevant EBS site shall be contacted
who then will initiate corrective actions as described above.
6.3 EBONE related problems detected within a regional network.
Normally users within a regional network shall contact the relevant
operational responsible persons within that network. These operators
may then initiate corrective actions according to 6.2 above.
6.4 Actions when a problem is resolved.
Once a failure is resolved the NOC shall inform all parties that
have previously been made aware of the failure about the resolution.
6.5 Problem resolution flow char.
As presented in Prague and to be included here.