BGP Extensions to Enable BGP FlowSpec based IFIT

Border Gateway Protocol (BGP) Flow Specification defined in [RFC8955] and [RFC8956] (FlowSpec) is an extension to BGP that supports the dissemination of traffic flow specifications and resulting actions to be taken on packets in a specified flow. It leverages the BGP Control Plane to simplify the distribution of ACLs (Access Control Lists). Using the Flow Specification extension, new filter rules can be injected to all BGP peers simultaneously without changing router configuration. BGP Flow Specification [RFC8955] and [RFC8956] define some BGP Network Layer Reachability Information (NLRI) formats used to distribute traffic flow specification rules. The NLRI for (AFI=1, SAFI=133) specifies IPv4 unicast filtering and the NLRI for (AFI=1, SAFI=134) specifies IPv4 BGP/MPLS VPN filtering. The NLRI for (AFI=2, SAFI=133) specifies IPv6 unicast filtering and the NLRI for (AFI=2, SAFI=134) specifies IPv6 BGP/MPLS VPN filtering. The Flow Specification match part defined in [RFC8955]and[RFC8956]include L3/L4 information like IPv4/6 source/destination prefix, protocol, source/destination port. In-situ Flow Information Telemetry (IFIT) denotes a family of flow- oriented on-path telemetry techniques,which can provide high-precision flow insight and real-time network issue notification, such as Path tracking, packet loss localization,queue depth/congestion level, as well as SLA compliance verification. so on. In particular, IFIT refers to network Operations, Administration, and Maintenance (OAM) data plane on-path telemetry techniques, including In-situ OAM (IOAM) [RFC9197] and Alternate Marking [RFC9341]. It can provide flow information on the entire forwarding path on a per-packet basis in real time. With the evolution of IP carried networks towards the intent-based and autonomous networks,flexible deployment of IFIT based on network dynamics and service requirements is getting a must. [I-D.ietf-idr-sr-policy-ifit] defines BGP extensions to distribute SR policy carrying IFIT information so that IFIT behavior can be enabled automatically when the SR policy is applied. [I-D.ietf-idr-bgp-ifit-capabilities] defines a new Characteristic to advertise the IFIT capabilities from the endpoint node to the headend node, so that the headend can determine whether a particular IFIT Option type can be encapsulated in data packets. More generally, this document defines the BGP extensions to distribute BGP FlowSpec based traffic filtering together with IFIT information. So the IFIT behavior can be applied to the matched flow, and be automatically enabled and running. In particular, this document defines a new BGP path attribute - IFIT attribute, attached to the BGP UPDATE message. It is is worth noting that this IFIT attribute is general-purpose BGP path attribute, and could also applied to any other BGP address families.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Abbreviations used in this document: ACL: Access Control List AFI: Address Family Identifier AS: Autonomous System DEX: Direct Exporting IFIT: In-situ Flow Information Telemetry IOAM: In situ Operation, Administration, and Maintenance NLRI: Network Layer Reachability Information OAM: Operation, Administration, and Maintenance SAFI: Subsequent Address Family Identifier

IFIT attribute is an optional non-transitive BGP path attribute. IANA is requested to allocate a value as the code of the attribute in the "BGP Path Attributes" registry [IANA-BGP-PARAMS]. The attribute is composed of a set of Type-Length-Value (TLV) encodings. Each TLV contains information corresponding to a particular IFIT type. An IFIT TLV is structured as shown in Figure 1.

IFIT Type (2 octets): Identifies a type of IFIT. This document defines two types of IFIT as follows. When the IFIT Type is 1, it is the IOAM Type [RFC9197], [RFC9326]. When the IFIT Type is 2, it is the AltMark Type [RFC9343]. Length (2 octets): The total number of octets of the Value field. Value (variable): Comprised of one or multiple sub-TLVs. Each sub-TLV consists of three fields: A 1-octet type, a 1-octet length, and zero or more octets of value. A sub-TLV is structured as shown in Figure 2.

Sub-TLV Type (1 octet): Each sub-TLV type defines a certain OAM option about the IFIT TLV that contains this sub-TLV. Sub-TLV Length (1 octet): The total number of octets of the Sub-TLV Value field. Sub-TLV Value (variable): Encoding of the Value field depends on the sub-TLV type. The following subsections define the encoding in detail.

This section specifies a number of sub-TLVs. These sub-TLVs can be included in two TLVs of the IFIT attribute. For IFIT Type 1, namely the IOAM Type, four sub-TLVs are defined in this document as follows: IOAM Pre-allocated Trace Option [RFC9197] Sub-TLV, Type=1. IOAM Incremental Trace Option [RFC9197] Sub-TLV, Type=2. IOAM Directly Export (DEX) Option [RFC9326] Sub-TLV, Type=3. IOAM Edge-to-Edge Option [RFC9197] Sub-TLV, Type=4. For IFIT Type 2, namely the AltMark Type, two sub-TLVs are defined in this document as follows: Alternate Marking [RFC9343] Sub-TLV, Type=1. Enhanced Alternate Marking sub-TLV, Type=2.

IOAM Sub-TLVs include four sub-TLVs, and every sub-TLV structure is defined in the following subsections.

The IOAM tracing data is expected to be collected at every node that a packet traverses to ensure visibility into the entire path a packet takes within an IOAM domain. The preallocated tracing option will create pre-allocated space for each node to populate its information. The structure of IOAM pre-allocated trace option sub-TLV is defined as follows:

Type: 1 (to be assigned by IANA). Length: 6, the total number of octets of the Sub-TLV Value field. Namespace ID: A 16-bit identifier of an IOAM-Namespace. The definition is described in Section 4.4 of [RFC9197]. IOAM Trace Type: A 24-bit identifier which specifies which data types are used in the node data list. The definition is described in Section 4.4 of [RFC9197]. Flags: A 4-bit field. The definition is described in [RFC9322] and section 4.4 of [RFC9197]. Rsvd: A 4-bit field reserved for further usage. It MUST be zero on transmission and ignored on receipt.

The incremental tracing option contains a variable node data fields where each node allocates and pushes its node data immediately following the option header. The structure of IOAM incremental trace option sub-TLV is defined as follows:

Type: 2 (to be assigned by IANA). Length: 6, the total number of octets of the Sub-TLV Value field. The definition of other fields is the same as the pre-allocated trace option sub-TLV defined in Section 4.1.1.

IOAM DEX option is used as a trigger for IOAM data to be directly exported to a collector without being pushed into in-flight data packets. The structure of IOAM DEX sub-TLV is defined as follows:

Type: 3 (to be assigned by IANA). Length: 12, the total number of octets of the Sub-TLV Value field. Namespace ID: A 16-bit identifier of an IOAM-Namespace. The definition is described in Section 4.4 of [RFC9197]. Flags: A 8-bit field. The definition is described in Section 3.2 of [RFC9326]. Extension-Flags: A 8-bit field. The definition is described in Section 3.2 of [RFC9326]. Every bit in the Extension-Flag field that is set to 1 indicates the existence of a corresponding optional 4-octet field. Bit 0 (the most significant bit) is defined as Flow ID and bit 1 as Sequence Number. Flow ID may be uniquely assigned by the collector. In this document, when bit 1 is set to 1, Sequence Number MUST be sequentially assigned by the headend device (encapsulating node). IOAM Trace Type: A 24-bit identifier that specifies which IOAM-Data-Fields should be exported, as defined in Section 3.2 of [RFC9326]. Rsvd: A 4-bit field reserved for further usage. It MUST be zero on transmission and ignored on receipt. Flow ID: A 32-bit flow identifier. The definition is described in Section 3.2 of [RFC9326]. Flow ID may be uniquely assigned by the collector.

The IOAM edge-to-edge option is to carry data that is added by the IOAM encapsulating node and interpreted by IOAM decapsulating node. The structure of IOAM edge-to-edge option sub-TLV is defined as follows:

Type: 4 (to be assigned by IANA). Length: 4, the total number of octets of the Sub-TLV Value field. Namespace ID: A 16-bit identifier of an IOAM-Namespace. The definition is described in Section 4.4 of [RFC9197]. IOAM E2E Type: A 16-bit identifier which specifies which data types are used in the E2E option data. The definition is described in Section 4.6 of [RFC9197].

AltMark Sub-TLVs include two sub-TLVs, and every sub-TLV structure is defined in the following subsections.

The structure of Alternate Marking sub-TLV is defined as follows:

Type: 1 (to be assigned by IANA). Length: 4, the total number of octets of the Sub-TLV Value field. FlowMonID: A 20-bit identifier to uniquely identify a monitored flow within the measurement domain. The definition is described in Section 5.3 of [RFC9343]. L: 1-bit Loss flag set to 1 indicating Packet Loss Measurement as described in Section 5.1 of [RFC9343]. D: 1-bit Delay flag set to 1 indicating Packet Delay Measurement as described in Section 5.2 of [RFC9343]. H: 1-bit flag set to 1 indicating that the measurement is Hop-by-Hop mode. E: 1-bit flag set to 1 indicating that the measurement is End-to-End mode. Rsvd: 8-bit field reserved for further usage. It MUST be zero on transmission and ignored on receipt.

The structure of Enhanced Alternate Marking sub-TLV is defined as follows:

Type: 2 (to be assigned by IANA). Length: 6, the total number of octets of the Sub-TLV Value field. Period: 4-bit field used for encoding at most 16 measuement periods. The definition of its value and the corresponding measuement period is out of this document. L: 1-bit Loss flag set to 1 indicating Packet Loss Measurement as described in Section 5.1 of [RFC9343]. D: 1-bit Delay flag set to 1 indicating Delay Measurement as described in Section 5.2 of [RFC9343]. H: 1-bit flag set to 1 indicating that the measurement is Hop-by-Hop mode. E: 1-bit flag set to 1 indicating that the measurement is End-to-End mode. F: 1-bit flag set to 1 indicating an optional 32-bit Flow ID, which uniquely identify a monitored flow within the measurement domain. The definition and usage is described in section 7 of [I-D.draft-he-ippm-ioam-dex-extensions-incorporating-am]. In the centralized way, Flow ID is uniquely assigned by the controller; in the distributed way, Flow ID is locally assigned by the headend device (encapsulating node). S: 1-bit flag set to 1 indicating an optional 32-bit Sequence Number, starting from 0 and incremented by 1 for each packet from the same flow at the encapsulating node. The Sequence Number field is set at the encapsulating node and exported to the receiving entity by the forwarding nodes. The Sequence Number, when combined with the Flow ID, provides a convenient approach to correlate the exported data from the same user packet.In this document, when bit S is set to 1, Sequence Number MUST be sequentially assigned by the headend device (encapsulating node). M: 1-bit flag set to 1 indicating an optional 32-bit Measurement Period Number(MPN), starting from 0 and incremented by 1 for the specified flow with the same Flow ID. The field is set at the encapsulating node and exported to the receiving entity by the forwarding nodes. The MPN, when combined with the Flow ID, provides a convenient approach to correlate the exported data of the same flow during the same measurement period from multiple nodes. In this document, when bit M is set to 1, MPN MUST be sequentially assigned by the headend device (encapsulating node). Rsvd: 5-bit field reserved for further usage. It MUST be zero on transmission and ignored on receipt.

IFIT may be applied on all the traffic flow or a subset of the traffic. For the IOAM Type, take IOAM trace monitoring as example, when an IOAM encapsulating node incorporates the IOAM Pre-allocated Trace Option type (passport mode) or the DEX Option type (postcard mode) into all packets of the user traffic it forwards, more bandwidth and processing resources are required. So it is preferable for an IOAM encapsulating node to apply the IOAM functionality to the selected subset of the traffic. But for the AltMark Type, it is more perferable for an encapsulating node to color all the traffic of interest it forwards, not a subset of the traffic such that the fidelity of performance measurement for user traffic flow (e.g., packet loss) can be ensured. This document defines a new Traffic Sampling Action that it standardizes as BGP Extended Communities [RFC4360].

The structure of the traffic sampling Extended Community is defined as follows:

Type: 1-octet, BGP Transitive Extended Community Type, set to 0x80. Sub-Type: 1-octet, TBA (to be assigned by IANA). ASN: 2-octet AS number, which can be assigned from a 2-octet AS number. When a 4-octet AS number is locally present, the 2 least significant octets of such an AS number can be used. This value is purely informational and SHOULD NOT be interpreted by the implementation. Sampling Rate: 2-octet unsigned integer as a percentage in packets, restricted to [0x0000,0xffff], with value 0x0000 (0) indicating a sampling rate of 0, and value 0xffff (65535) indicating a sampling rate of 100%. The resolution is 0.0015%. A sampling rate of 0 should result in no packet for the particular flow to be applied to IFIT, and a sampling rate of 100% should result in all traffic for the particular flow to be be applied to IFIT. Reserved: 2-octet reserved field, It MUST be zero on transmission and ignored on receipt.

A Flow Specification is an n-tuple consisting of several matching criteria that can be applied to IP traffic flow. A given IP packet is said to match the defined Flow Specification if it matches all the specified criteria. This n-tuple is encoded into a BGP NLRI. Flow Specifications can be seen as more specific routing entries to a unicast prefix, and the routing system can take advantage of the ACL (Access Control List) capabilities in the router's forwarding path. Typically, in operator network, the centralized controller determines the particular user traffic to be monitored according to service requirements; at the same time, the centralized controller also need to determine th IFIT type applied to the specified traffic flow. Based on BGP FlowSpec, the centralized controller sends the BGP FlowSpec UPDATE message to the headend device (i.e., IOAM encapsulating node), carrying NLRI and IFIT attributes along with the traffic sampling Extended Community(if present). The BGP FlowSpec update message is used to instruct the headend device to perform IFIT automatic configuration on the monitored traffic flows. The headend device automatically generates ACLs according to the received traffic filter rules, and encapsulates IFIT for the incoming specified traffic flow packets, achieving the automated configuration for the monitored traffic flows. Once the centralized controller determines to terminate monitoring the specified traffic flow, it can withdraw the corresponding NRLI routes, indicating that the headend device will remove the ACLs related to the traffic filter rules and stop encapsulating IFIT for the incoming specified traffic flow packets.

)- |Endpoint| +-------+ ( BGP/MPLS VPN Network ) +--------+ ( ) (-----------------)]]>

Flow Specifications with IFIT attribute received from a BGP peer that are accepted in the respective Adj-RIB-In are used as input to the route selection process. The validation procedure is the same as specified in [RFC8955],[RFC8956], and [RFC9117]. Additionally, the IFIT attribute (if present) MUST be attached to the BGP UPDATE message and MUST have an IFIT TLV whose IFIT Type sets to the IOAM Type (1) or the AltMark Type (2). The validation of the TLVs/sub-TLVs introduced in this document and defined in their respective sub-sections of Section 4 MUST be performed to determine if they are malformed or invalid. In case of any error detected, either at the attribute or its TLV/sub-TLV level, the "treat-as-withdraw" strategy MUST be applied. This is because a BGP update without a valid IFIT attribute (comprising of all valid TLVs/sub-TLVs) is not usable. A router that receives a BGP update that is invalid according to these criteria MUST treat the update as malformed. A BGP update that is determined to be invalid, and therefore malformed, MUST be handled by the "treat-as-withdraw" strategy. When an IFIT TLV includes any valid sub-TLV that is but unrecognized or unsupported, the BGP update SHOULD NOT be considered unusable. An implementation MAY provide an option for ignoring unsupported sub-TLVs. An implementation SHOULD log any errors found during the above validation for further analysis.

IANA is requested to allocate a value as the code of the attribute in the "BGP Path Attributes" registry [IANA-BGP-PARAMS].

IANA is requested to create a new registry grouping name "In-situ Flow Information Telemetry (IFIT). IANA is requested to create the following new registries under the "In-situ Flow Information Telemetry (IFIT) registry group. The "IFIT Type" registry. The "IOAM Type Sub-TLVs" registry. The "AltMark Type Sub-TLVs" registry.

IANA is requested to allocate the following values as the type code of the IFIT Type TLVs in the "IFIT Type" registry.

IANA is requested to allocate the following values as the type code of the IOAM Type Sub-TLVs in the "IOAM Type Sub-TLVs" registry.

IANA is requested to allocate the following values as the type code of the AltMark Type Sub-TLVs in the "AltMark Type Sub-TLVs" registry.

IANA is requested to allocate a value as the Sub-type code of Traffic Sampling Extended Community in the registry entitled "Generic Transitive Extended Community Sub-Types".

The security mechanisms of the base BGP security model apply to the extensions described in this document as well. See the Security Considerations section of [RFC4271] for a discussion of BGP security. The BGP extensions specified in this document enable IFIT within an controlled domain, as defined in [RFC9378] and [I.D.draft-ietf-ippm-alt-mark-deployment]. IFIT operates within a controlled domain and its security considerations also apply to BGP sessions when carrying IFIT information. The IFIT configurations distributed by BGP are expected to be used entirely within this trusted IFIT domain which comprises a single AS or multiple ASes/ domains within a single provider network. Therefore, precaution is necessary to ensure that the IFIT information advertised via BGP sessions is limited to nodes in a secure manner within this trusted IFIT domain. Flow Specification BGP speakers (e.g., the centralized controller) also need to be cautious for sending BGP updates. For example, sending updates at a high rate, or generating a high number of Flow Specifications may stress the receiving systems (the Headend devices), or exceed their capabilities. Another major concern is that enabling IFIT on all traffic flows may have some impact on network forwarding performance, thus the traffic sampling action may be needed for protecting network resources. In particular, settting the appropriate traffic sampling rate for the IOAM trace monitoring is also necessary.

The authors would like to thank Jeffrey Haas for the valuable comments and suggestions.