Problem Statement for Network Resilience

Introduction Current IP network reliability mechanisms are largely built around redundancy, fast failure detection, protection switching, and topology convergence. These mechanisms are effective for many anticipated and relatively well-bounded failures. However, as network scale, service dependency, and operational complexity increase, failures may become correlated across layers, evolve dynamically, consume the resources needed for recovery, or degrade service without causing a clear binary "up/down" state. In these conditions, static protection alone may not be sufficient to preserve service continuity. In this document, network resilience refers to the ability of a network to anticipate and reduce risk, absorb and contain failures, maintain an acceptable level of service during disruption, recover within bounded time objectives, and improve its future response based on operational feedback. The need for resilience enhancement is driven by three trends:

Stricter Service Continuity Objectives: Critical services increasingly require more than binary availability. During severe disruptions, the network is expected to maintain a minimum acceptable service level and restore the target service level within bounded recovery objectives.
Operational Complexity Beyond Isolated Detection: Existing OAM tools provide mechanisms for fault detection, fault isolation, and performance monitoring . However, visibility alone does not ensure that information from multiple layers and domains is correlated quickly enough to support coordinated mitigation in complex failure scenarios.
Growth of Correlated and Non-binary Failures: Conventional protection designs generally assume a bounded set of anticipated faults. They are less effective when failures involve shared-risk dependencies, gray degradation, control-plane instability, rapidly changing resource bottlenecks, or degradation of the systems required for diagnosis and recovery.

This document first describes representative failure sources and cross-cutting resilience challenges, then analyzes capability gaps across the failure lifecycle, and finally derives technical requirements for resilience enhancement.

Problem Statement Network resilience problems can originate from a range of physical, operational, design, protocol, performance, and security-related conditions. For analysis, this document identifies six representative failure sources and groups them into two broad categories: (1) structural and change-related sources and (2) dynamic and behavioral sources. These categories describe where resilience problems originate or first become observable. They are not mutually exclusive. A single incident may involve multiple sources, and its impact may propagate through shared dependencies, protocol interactions, traffic redistribution, resource contention, or automated recovery actions.

Network Resilience Problem Space

Failure Sources

Physical Interruption Physical interruption occurs when links, ports, devices, line cards, or supporting infrastructure become unavailable. Mechanisms such as BFD and fast reroute can provide rapid protection against many anticipated and relatively well-bounded failures. However, protection may become ineffective when multiple components fail concurrently, backup resources are insufficient for the rerouted load, or the actual dependency relationship between primary and backup resources is not fully known. In such cases, recovery actions may lead to congestion, traffic blackholing, or secondary service degradation.

Configuration Errors Configuration errors occur when incorrect or inconsistent configuration is introduced during provisioning, maintenance, or policy changes. The central problem is a mismatch among intended service behavior, routing and forwarding policy, and the configuration actually deployed in the network. Such mismatches can affect reachability, isolation, path selection, traffic engineering, or Quality of Service (QoS).

Typical Scenarios: Misconfigured routing policies, incorrect VPN parameters, inconsistent network-slicing configuration, and incomplete or partially applied changes.

Defects in Network Planning and Specification Design Unlike transient configuration errors, planning and design defects are latent weaknesses in topology design, network specifications, protection schemes, dependency assumptions, or capacity planning. These weaknesses may remain invisible during normal operation and emerge only during protection switching, concurrent failures, or traffic surges.

Typical Scenarios: Hidden shared-risk link groups (SRLGs) between primary and backup paths, insufficient backup capacity, poorly designed escape paths, and unexpectedly large fault domains caused by incomplete dependency analysis or inconsistent design practices.
Impact Example: After a primary-path failure, traffic may be switched to a backup path whose capacity was not validated against the expected failover load. The backup path then becomes congested or unavailable, turning a localized failure into wider service degradation.

Control Plane and Protocol Anomalies These failures arise from abnormal behavior in routing, detection, or control protocols, including software defects, unstable protocol interactions, inconsistent state, and repeated state transitions that prevent the control plane from converging to a stable state.

Typical Scenarios: Intermittent BFD flapping, IGP route churn, BGP route oscillation, inconsistent control state, and unstable interaction between independently designed protection or traffic-engineering mechanisms.
Amplification Effect: A localized anomaly can trigger repeated route recalculation, unnecessary protection switching, route churn, transient forwarding loops, or traffic blackholing. Services with strict delay, jitter, or packet-loss objectives may be affected even when the network eventually converges.

Implicit Deterioration (Gray Failures) A gray failure is a condition in which a link, node, protocol session, or service component appears operational while the service experienced by some or all traffic has degraded beyond an acceptable level.

Typical Scenarios: Microbursts, intermittent packet loss, increased delay or jitter, asymmetric degradation, and degradation affecting only specific traffic classes or service flows.
Core Architectural Gaps:
- State Ambiguity: Periodic liveness mechanisms may not observe short-lived, flow-specific, path-specific, or performance-only degradation.
- Observation-to-Action Gap: IOAM defines data fields for collecting operational and telemetry information along a packet path, but automated diagnosis, policy selection, and mitigation are outside the scope of that specification. Without an integrated decision and response loop, affected traffic may remain on a degraded path and continue to violate service objectives.

Security Incidents Security incidents can reduce network resilience by compromising the trustworthiness of control or management systems, manipulating routing or policy state, or consuming network resources through malicious traffic.

Typical Scenarios: DDoS attacks, route hijacking, unauthorized configuration changes, falsified operational data, and manipulation of control or policy inputs.
Impact Mechanisms: A rapid traffic surge may exhaust link bandwidth, forwarding resources, processing capacity, or queue resources. Compromised control or management inputs may also trigger incorrect routing, protection, or recovery actions.

Cross-cutting Resilience Challenges The failure sources above describe where resilience problems originate or first become observable. However, the severity of a resilience incident is often determined by properties that cut across multiple failure sources. In particular, three challenges make complex incidents difficult to handle using mechanisms designed primarily for isolated and anticipated failures.

Correlated and Cascading Failures Failures that appear independent at the logical network layer may share dependencies in physical infrastructure, control and management systems, software components, or external services. A single underlying event can therefore affect multiple nominally redundant components at the same time. Failures may also propagate through traffic redistribution, protocol interactions, shared-resource contention, and automated recovery actions. For example, a physical failure may trigger traffic rerouting, overload a backup path, cause packet loss and protocol instability, and subsequently trigger additional routing changes. In such cases, recovery mechanisms that operate correctly in isolation may interact in ways that amplify the original failure. The challenge is therefore not limited to detecting individual failed components. The network also needs sufficient dependency awareness and failure-context correlation to identify common causes, estimate the potential propagation scope, and avoid mitigation actions that expand the affected fault domain.

Resource Exhaustion and Loss of Recovery Capacity Network failures and traffic surges may exhaust data-plane, control-plane, or management-plane resources. Resource exhaustion can result from malicious traffic, legitimate traffic surges, failure-induced traffic redistribution, excessive protocol churn, telemetry storms, or repeated recovery actions. A particular resilience concern is that resources required for diagnosis and recovery may themselves become unavailable during a large-scale incident. For example, control-plane CPU exhaustion may delay routing convergence, while management-system or telemetry-processing overload may delay fault correlation, decision making, and policy execution. Consequently, a logically valid protection path or mitigation action may fail when activated under actual incident load. The resilience problem is therefore not only whether backup resources exist, but whether sufficient forwarding, control, and management capacity remains available throughout the mitigation and recovery process.

Recovery-System Survivability Resilience mechanisms increasingly depend on telemetry systems, controllers, management connectivity, orchestration platforms, policy engines, and other automation components. These systems may themselves become degraded, unreachable, overloaded, or inconsistent during the same incident they are expected to mitigate. Examples include telemetry pipeline failures, stale topology or resource information, inconsistent controller state, management connectivity loss, orchestration backlog, partially applied transactions, conflicting automated policies, and failed rollback operations. The resulting problem is a loss of recovery capability: the network may still have usable forwarding resources, but the systems required to observe the incident, select a mitigation strategy, execute the action, or verify the result may no longer operate correctly. A resilience architecture therefore needs to consider the survivability and degraded-mode behavior of its own sensing, decision, control, and recovery mechanisms. The failure sources and cross-cutting challenges described above explain where resilience incidents originate and why their impact may exceed the assumptions of conventional protection mechanisms. The following section analyzes where resilience capabilities are insufficient across the pre-event, in-event, and post-event stages of the failure lifecycle.

Resilience Gap Analysis Across the Failure Lifecycle The failure sources and cross-cutting challenges described above explain where resilience incidents originate and why they may propagate or exceed the assumptions of conventional protection mechanisms. This section analyzes the corresponding capability gaps across three stages of the failure lifecycle: pre-event, in-event, and post-event.

Resilience Gaps Across the Failure Lifecycle

Pre-event: Insufficient Risk Discovery and Validation Before an incident occurs, resilience is weakened when latent dependencies, risky changes, and insufficient recovery resources cannot be identified or validated in advance.

Incomplete Dependency Awareness: Logical topology information alone may not reveal shared physical infrastructure, common control or management dependencies, software commonality, or other hidden relationships among nominally redundant resources. As a result, failures that are assumed to be independent may actually share a common cause.
Limited Change and Policy Validation: Configuration, routing-policy, protection, and service changes may be syntactically valid while producing unintended behavior when combined with existing topology, policy, and resource constraints. Existing operational processes do not always provide sufficiently comprehensive pre-deployment validation across multi-vendor devices, multiple protocol layers, and service dependencies.
Limited Failure Simulation and Stress Validation: Protection paths and recovery strategies are often evaluated against predefined failure assumptions. They may not be validated under correlated failures, concurrent traffic surges, protocol instability, or partial management-system degradation. Consequently, backup capacity and recovery-system resources that appear sufficient under normal conditions may become inadequate during an actual incident.

The pre-event gap is therefore not merely the presence of configuration or design errors. The deeper problem is the inability to discover hidden dependencies, predict compound failure effects, and validate whether protection and recovery mechanisms remain effective under realistic incident conditions.

In-event: Insufficient Situation Awareness, Containment, and Adaptive Response During an incident, successful mitigation depends on the ability to understand the evolving situation, identify the affected scope, preserve sufficient recovery capacity, and coordinate actions across multiple mechanisms and layers. Current networks may lack these capabilities when failures are correlated, rapidly changing, or resource-constrained.

Fragmented Failure Context: Telemetry, alarms, protocol state, topology information, resource utilization, and service-impact information may be collected by separate systems. Without timely correlation, multiple symptoms of a common underlying failure may be treated as independent events, delaying diagnosis and causing inconsistent mitigation decisions.
Insufficient Dependency and Propagation Awareness: When failures propagate across physical infrastructure, logical topology, protocols, and services, the network may not have sufficient information to determine the common cause or estimate the potential propagation scope. This limits the ability to contain the failure before additional services or network domains are affected.
Resource Blindness During Mitigation: A logically available protection or escape path may not have sufficient forwarding capacity, queue resources, control-plane processing capacity, or management-system capacity under actual incident load. Recovery actions based on incomplete or stale resource information may therefore move traffic or control workload into another bottleneck.
Interaction Among Independent Recovery Mechanisms: Routing convergence, fast reroute, traffic engineering, load balancing, admission control, and service-level recovery mechanisms may operate independently. Although each mechanism may behave correctly in isolation, their combined actions may result in oscillation, repeated traffic movement, transient loops, resource contention, or expansion of the affected fault domain.
Insufficient Recovery-System Survivability: Telemetry systems, controllers, management connectivity, orchestration platforms, and policy engines may themselves become overloaded, unreachable, or inconsistent during a major incident. The network may therefore lose part of its observation, decision, execution, or verification capability at the time when these capabilities are most needed.

The in-event gap is consequently not limited to detection speed or protection-switching time. It also includes the ability to construct a coherent incident view, contain propagation, preserve recovery capacity, and coordinate adaptive responses without amplifying the original failure.

Post-event: Insufficient Recovery Assurance and Adaptive Improvement After an incident has been contained, resilience still depends on whether service recovery is complete, whether temporary mitigation can be safely removed, and whether operational evidence is converted into validated improvements. Service Assurance for Intent-Based Networking (SAIN) provides an architecture for assessing whether service instances and their dependent subservices are operating as expected. Building on such assurance information, resilience enhancement requires stronger integration among diagnosis, recovery orchestration, verification, and policy improvement.

Dependence on Manual Recovery: Complex incidents often require manual diagnosis, traffic adjustment, configuration repair, capacity reassessment, or restoration of control and management systems. Heavy dependence on expert intervention increases recovery time and may produce inconsistent outcomes.
Insufficient Recovery Verification: The removal of an alarm or restoration of protocol adjacency does not necessarily indicate that the affected service has returned to its required performance level. Recovery processes may lack end-to-end verification of reachability, delay, loss, isolation, policy compliance, and service-level objectives.
Limited Incident-to-Policy Feedback: Incident evidence, root-cause findings, propagation paths, resource bottlenecks, and recovery outcomes are not always converted into updates to risk models, detection thresholds, dependency information, protection policies, capacity assumptions, or validation rules.
Insufficient Validation of Learned Changes: Changes derived from operational experience may introduce new interactions or risks if they are deployed directly. Closed-loop improvement therefore requires not only learning from incidents, but also validating proposed policy or configuration changes before they are applied to the production network.

The post-event gap is therefore broader than repair automation. A resilient network needs to verify that recovery objectives have actually been achieved and to convert incident evidence into validated improvements that reduce the probability or impact of similar failures in the future.

Technical Requirements for Resilience Enhancement The problem space and lifecycle gap analysis above imply that a resilience-enhancement architecture needs to provide at least the following capabilities:

Proactive Risk Awareness and Validation: Detect latent design, configuration, dependency, and degradation risks by correlating multi-dimensional telemetry, configuration state, topology, policy, and service intent. The architecture should also support pre-deployment validation or simulation of high-impact changes, compound failures, and stress scenarios.
Resource-aware Elastic Protection: Maintain sufficiently current awareness of available path and resource conditions so that protection and traffic-steering actions do not move traffic onto paths that cannot sustain the expected load. The network should be able to absorb short-lived shocks through resource buffering, traffic isolation, admission control, load shedding, or elastic scheduling where applicable.
Coordinated Fault Containment and Adaptive Response: Correlate evidence across protocol, network, infrastructure, management, and service layers; identify common causes and affected scope; estimate potential propagation; and coordinate mitigation actions so that local failures are contained rather than amplified by independent recovery mechanisms.
Bounded Recovery and Service Continuity: Restore critical service performance within defined recovery objectives while maintaining a minimum acceptable service level during disruption whenever feasible. Recovery actions should be verifiable and should avoid repeated oscillation between unstable states. The sensing, decision, control, and management functions required for recovery should remain available, or provide defined degraded-mode behavior, under the failure conditions they are expected to handle.
Closed-loop Learning and Adaptation: Use incident evidence and recovery outcomes to improve detection logic, dependency information, risk models, protection policies, capacity assumptions, and validation rules. Policy or configuration changes derived from operational feedback should themselves be validated before deployment.

Conclusion Current network reliability mechanisms remain necessary, but they are not sufficient for failure scenarios that are correlated, non-binary, dynamic, resource-constrained, or difficult to anticipate. The problem space described in this document shows that resilience gaps arise not only from individual failure sources, but also from hidden dependencies, cascading propagation, resource exhaustion, interaction among recovery mechanisms, and degradation of the systems required for diagnosis and recovery. A resilient network architecture therefore requires a lifecycle approach: reduce risk before failures occur, construct an accurate incident view while failures are evolving, contain propagation, preserve sufficient recovery capacity, restore services within defined objectives, verify the recovery result, and convert operational evidence into validated improvements. The technical capabilities identified in this document provide a basis for further architectural and protocol work on network resilience.

Security Considerations Resilience mechanisms can introduce new attack surfaces. For example, an attacker could inject or manipulate telemetry, topology, dependency, resource-state, or policy information to trigger unnecessary traffic movement, oscillation, incorrect containment decisions, or resource exhaustion. A resilience framework therefore needs to protect the authenticity, integrity, authorization, and freshness of sensing data and control inputs, and to apply appropriate access control to policy updates and automated recovery actions. Automated mitigation can also amplify incorrect decisions. Implementations should therefore support policy constraints, scoped actions, rollback or fail-safe behavior, and post-action verification. The security and availability of telemetry systems, controllers, orchestration platforms, and management connectivity are themselves part of the resilience problem and should be protected accordingly.

IANA Considerations This document makes no requests of IANA. ---back