BGP Extension for 5G Edge Service Metadata

3.1. Edge Metadata Influenced BGP Path Selection

When an ingress router receives BGP UPDATEs for the same IP prefix from multiple egress routers, all these egress routers' loopback addresses are considered as the next hops for the IP prefix. For the selected low latency edge services, the ingress router BGP engine would call an edge service Management function that can select paths based on the edge service Metadata received. Section 5.1 has an exemplary algorithm to compute the weighted path cost based on the edge service Metadata carried by the Sub-TLV(s) specified in this document.¶

Section 5 has the detailed description of the edge service Metadata influenced optimal path selection.¶

3.2. Ingress Router Forwarding Behavior

When the ingress router receives a packet and does a lookup on the route in the FIB, it determines the destination prefix's entire path including the optimal egress node. The ingress router encapsulates the packet destined towards the optimal egress router. For routes that carry the Metadata Path Attribute but lack the Tunnel Encapsulation Path Attribute [RFC9012], it is recommended that the ingress router encapsulate the original packet using an IP-in-IP header. This encapsulation ensures that intermediate nodes not supporting the Metadata Path Attribute do not forward the packet to unintended destinations. The outer header SHOULD set the destination address to the optimal egress router and the source address to the ingress router.¶

For routes without the Metadata Path Attribute, no changes are required. Packets are forwarded according to existing behavior: encapsulation is applied when Tunnel Attributes are present, and parkets are forwarded without encapsulation when they are not.¶

For subsequent packets belonging to the same flow, the ingress router needs to forward them to the same egress router unless the selected egress router is no longer reachable. Forwarding packets for a particular flow to the same egress router, also known as Flow Affinity, is supported by many commercial routers. Most registered EC services have relatively short-lived flows.¶

How Flow Affinity is implemented is out of the scope for this document.¶

3.3. Forwarding Behavior when UEs Move

When a UE moves to a new 5G gNB which is anchored to the same UPF, the packets from the UE traverse to the same ingress router. Path selection and forwarding behavior are same as before.¶

If the UE maintains the same IP address when anchored to a new UPF, the directly connected ingress router might use the information passed from a neighboring router to derive the optimal BGP Next Hop for this route. The detailed algorithm is out of the scope of this document.¶

4.1. Edge Metadata Path Attribute

The Edge Metadata Path Attribute is an optional non-transitive BGP Path attribute that carries metadata associated with edge services attached to the egress router. The attribute consists of one or more Edge Metadata Sub-TLVs, where each Sub-TLV encodes one specific metadata item associated with the advertised route or service.¶

The Edge Metadata Path Attribute MAY be included in a BGP UPDATE together with other BGP Path Attributes, such as Communities, NEXT_HOP, Tunnel Encapsulation Path Attribute, and other applicable attributes. The choice of which routes carry the Edge Metadata Path Attribute, and which Sub-TLVs are included for those routes, is determined by local policy. The fields within the Edge Metadata Path Attribute and all included Sub-TLVs MUST use network byte order.¶

Boundary filtering SHOULD be applied at the administrative boundary to prevent the Edge Metadata Path Attribute from being distributed beyond its intended scope.¶

4.2. The Site Preference Index Sub-TLV

Different services might have different preference index values configured for the same site. For example, Service-A requires high computing power, Service-B requires high bandwidth among its microservices, and Service-C requires high volume storage capacity. For a DC with relatively low storage capacity but high bisectional bandwidth, its preference index value for Service-B is higher and lower for Service-C. Site Preference Index can also be used to achieve stickiness for some services.¶

It is out of the scope of this document how the preference index is determined or configured.¶

The Site Preference Index Sub-TLV has the following format:¶

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |Site-Preference-Index Sub-Type | Length        | Reserved      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                Site Preference Index value                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

-

Site-Preference-Index Sub-Type (16 bits): 1 (specified in this document).¶

-

Length (8 bits): Specifies the total length in octets of the value field (not including the Type and Length fields). For the Site-Preference-Index Sub-Type, the length SHOULD be set to 5.¶

-

Reserved: Reserved for future use. In this version of the document, the Reserved field MUST be set to zero and MUST be ignored upon receipt. Received values MUST be propagated without change.¶

-

Site Preference Index value: 1 .. (2^32-1); the higher the value, the more preference for the site. Site Preference Index value == 0 is reserved, and the Site-Preference-Index Sub-TLV SHOULD be ignored when 0 is received..¶

4.3. Site Physical Availability Index Metadata Sub-TLV

The Site Physical Availability Index indicates the percentage of impact on a group of routes associated with a common physical characteristic, for example, a pod, a row of server racks, a floor, or an entire DC. The purpose is to use one UPDATE message to indicate a group of routes of different NLRIs impacted by a physical event. For example, a power outage to a pod can cause the Site Physical Availability Index to be 0% for all the routes in the pod. Partial fiber cut to a row of shelves can cause the Site Physical Availability Index to be 50% for all the routes in those shelves. The value is 0-100, with 100% indicating the site is fully functional, 0% indicating the site is entirely out of service, and 50% indicating the site is 50% degraded.¶

It is recommended to assign each route with one Site-ID. When a route is associated with multiple Site-IDs, the latest BGP UPDATE will override any previous associations. For example, one DC can use POD number as Site-ID, another DC can use Row of Shelves as the Site-ID.¶

Cloud Site/Pod failures and degradation include, but are not limited to, a site degradation or an entire site going down caused by a variety of reasons. Examples include fiber cuts impacting a site or among pods, cooling failures, insufficient backup power, cyber threats attacks, too many changes outside of the maintenance window, etc. Fiber-cuts are not uncommon within a Cloud site or between sites.¶

When a physical failure occurs at an edge site (or a pod), many instances can be affected, and the associated routes (i.e., IP addresses) may not be easily aggregated. Instead of sending numerous BGP UPDATE messages to ingress routers for each impacted instance, the egress router can send a single BGP UPDATE to indicate the site's physical capacity availability. Based on this update, ingress routers can decide to reroute all or some of the affected instances, depending on the extent of the site's degradation. This approach significantly improves efficiency, particularly when fault detection within an edge site relies on proprietary or deployment-specific mechanisms.¶

The BGP UPDATE for the individual instances (i.e., the routes) can include the Capacity Availability Index solely for ingress routers to associate the routes with the Side-ID. The actual Capacity Availability Index value, i.e., the percentage for all the routes associated with the Side-ID, is generated by the egress routers with the egress routers' loopback address as the NLRI.¶

The Site Physical Availability Index Sub-TLV has fixed length of 8 Octets, including the Type field.¶

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|      PhyAvailIdx Sub-Type     |     Length    |I|   Reserved  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        Site-ID (2 octets)     | Site Availability Percentage  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- PhyAvailIdx Sub-Type (16 bits):

Indicates teh Site-Physical-Availability-Index Sub-Type=2 (Specified in this document).¶

-

Length (8 bits): Specifies the total length in octets of the value field (not including the Type and Length fields). For the PhyAvailIdx Sub-Type, the length SHOULD be set to 5.¶

Route-Flag I (1 bit):

is a flag bit. When set to 1, the Site Availability Index is for BGP speakers (receivers) to associate the routes with the Site-ID. The Site Availability Percentage value is ignored. When set to 0, the BGP speakers (receivers) SHOULD apply the Site Availability Index value to all the routes associated with the Site-ID.¶

Reserved (7 bits):

Reserved for future use. The bits are set to zero upon transmission, and ignored upon reception.¶

- Site ID (16 bits):

is an identifier for a group of routes associated with a common physical characteristic, for example, a pod, a row of server racks, a floor, or an entire DC. The purpose is to use one UPDATE message to indicate a group of routes impacted by a physical event. Those routes might be from different address families or NLRIs. There could be multiple sites connected to one egress router (a.k.a. Edge DC GW).¶

- Site Availability Percentage (16 bits):

When the RouteFlag-I is 1, the Site Availability Percentage is ignored by the Ingress routers. When the RouteFlag I is set to 0, the Site Availability Percentage represents the percentage of the site availability for all the routes associated with the Site-ID; e.g., 100%, 50%, or 0%. When a site goes dark, the Index is set to 0. 50 means 50% functioning. When the value is outside the 0-100% range, the value carried in this Sub-TLV is ignored.¶

4.4. Service Delay Prediction Sub-TLV

It is desirable for an ingress router to select a site with the shortest processing time for an ultra-low latency service. However, it is not easy to predict which site has "the fastest processing time" or "the shortest processing delay" for an incoming service request because:¶

-

The given service instance shares the same physical infrastructure with many other applications and service instances. Service requests by other applications, UEs, or applications running behavior can impact the processing time for the given service instance.¶

-

The given service instance can be served by a cluster of servers behind a Load Balancer. To the network, the service is identified by one service ID.¶

-

The service complexity is different. One service may call many microservices, need to access multiple backend databases, and need to go through sophisticated security scrubbing functions, etc. Another service can be processed by a few simple steps. Without the application internal logic, it is not easy to estimate the processing time for future service requests.¶

Even though utilization measurements, like those below, are collected by most data centers, they cannot indicate which site has the shortest processing time. A service request might be processed faster on Site-A even if Site-A is overutilized.¶

-

Server utilization for the server where the instance is instantiated.¶

-

The network utilization for the links to the server where the instance is instantiated.¶

-

The number of databases that the service instance will access.¶

-

The memory utilization of the databases.¶

The remaining available resource at a site is a more reasonable indication of process delay for future service requests.¶

-

The remaining available Server resources.¶

-

The remaining available network utilization for the links to the server where the instance is instantiated.¶

-

The number of databases that the service instance will access.¶

-

The remaining storage available for the databases.¶

The Service Delay Prediction Index is a value that predicts processing delays at the site for future service requests. The higher the value, the longer of the delay.¶

While out of scope, we assume there is an algorithm that can derive the Service Delay Prediction Index that can be assigned to the egress router. When the Service Delay Prediction value is updated, which can be triggered by the available resources change, etc., the egress router can attach the updated Service Delay Predication value in a Sub-TLV under the Edge Metadata Path Attribute of the BGP Route UPDATE message to the ingress routers.¶

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ServiceDelayPredict Sub-Type  |   Length      |F|L|Reserved   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Service Delay Predication Value                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- ServiceDelayPredict Sub-Type (16 bits):

3 (specified in this document).¶

- Length (8 bits):

specifies the total length in octets of the value field, not including the sub-Type and Length field. The value of Length can be 5 or 9 depends on what format the Service Delay Prediction Vlaue uses.¶

- Flag (F) (1 bit):

Indicates whether the Service Delay is a timer value (F=0) or a relative value (F=1) where a higher value represents a longer delay¶

- Flag (L) (1 bit):

Indicates the unit of measurement for the Service Delay Prediction Value. When the F-flag is set to 0, L=0 specifies the 64-bit NTP Timestamp format, and L=1 indicates milliseconds. If the F-flag is set to 1, the L-flag value is ignored.¶

- Reserved (6 bits):

These bits are reserved for future use and MUST be set to zero. Future documents may specify different uses for these bits.¶

- Service Delay Predication Value (when the Flag bit is set to 1):

an integer in the range of 0-100, with 0 indicating that the service delay is negligible and 100 indicating that the site has the most significant delay compared to all other sites for the same service. When the value is outside the 0-100 range, the value carried in this Sub-TLV is ignored.¶

- Service Delay Predication Value (when the Flag bit is set to 0):

the estimated delay time encoded in the NTP Format as defined in [RFC5905]. When the L-flag is 1, then it is a 64-bit format, otherwise it is a 32-bit short format.¶

4.5. Raw Measurement Sub-TLV

When ingress routers have embedded analytics tool relying on the raw measurements, it is useful for the egress router to send the raw measurement.¶

Raw Measurement Sub-TLV has the following format:¶

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Raw-Measurement Sub-Type      | Length        |  Reserved     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           Value                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- Raw-Measurement Sub-Type (16 bits): 4 (specified in this document). Indicating raw measurements Metadata associated with the edge service address.¶

- Length (8 bits): specifies the total length, in octets, of the value field, excluding the Sub-Type and the Length fields. For the Raw-Measurement Sub-Type, the length is determined by the Value field, which carries one or more types of raw measurement.¶

- Reserved (8 bits): These bits are reserved for future use and MUST be set to zero. Future documents may specify different uses for these bits.¶

- Value: The value filed can contain multiple types of raw measurements, each represented as a Sub-Sub-TLV.¶

One example of a raw measurement Metadata Sub-sub-TLV is defined below to convey the total number of packets or bytes transmitted over a specified period for a particular edge service address. When a Data DC GW router cannot directly access the internal state of an edge service, the volume of incoming traffic can be a reliable indicator of its load. A sudden increase in packets or bytes can signal a surge in requests, potentially leading to performance issues or resource constraints on the service side.¶

To differentiate this measurement from others that may be defined in the future, this document assigns a Sub-sub-Type value of 1 to represent the total packets or bytes transmitted to an edge service address.¶

Future documents may define additional Sub-sub-types of raw measurement metadata. Each type of raw measurement will have a unique Sub-sub-type value assigned at the time of its specification.¶

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |RawPacketsMeasure Sub-sub-Type | Length        |B|Reserved     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                   Measurement Period                          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   total number of packets (or bytes) to the Edge Service      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   total number of packets (or bytes) from the Edge Service    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- RawPacketsMeasure Sub-sub-Type (8 bits): 1 (specified in this document). Indicating raw measurements of packets or bits transmitted to or from the edge service address.¶

- Length (8 bits): specifies the total length in octets of the value field, excluding the Sub-sub-Type and the Length fields. For the raw measurements of packets transmitted to or from the edge service address Sub-sub-Type, the length SHOULD be 22.¶

- B flag (1 bit): If set to 0, the raw measurement is the number of packets. If set to 1, the raw measurement is the number of bytes.¶

- Reserved (7 bits): These bits are reserved for future use and MUST be set to zero.¶

- Measurement Period: BGP Update period in Seconds or user-specified period.¶

- Total number of packets to the Edge Service (32 bits): This field specifies the total number of packets transmitted to the edge service address over the specified measurement period.¶

- Total number of packets from the Edge Service (32 bits): This field specifies the total number of packets from the edge service address over the specified measurement period.¶

The receiver nodes can compute the needed metrics, such as the Service Delay Prediction, for the service based on the raw measurements sent from the egress router and preconfigured algorithms.¶

4.6. Service-Oriented Capability Sub-TLV

The service-oriented capability Sub-TLV is for distributing information regarding the capabilities of a specific service in a deployment environment. Depending on the deployment, a deployment environment can be an edge site or other types of environments. This information provides ingress routers or controllers with the available resources for the specific service in each deployment environment. It enables them to make well-informed decisions for the optimal paths to the selected deployment environment.¶

Currently, the Sub-TLV only has an abstract value derived from various metrics, although the specifics of this derivation are beyond the scope of this document. Importantly, this value is significant only when comparing multiple data center sites for the same service. This value is not meaningful when comparing different services, meaning the capability value relevant to Service A cannot be directly compared with that for Service B. Future enhancements may expand this sub-TLV to include more types of metrics or even raw data that represents direct metrics. This information is important in 5G network environments where efficient resource utilization is crucial for enhancing performance and service quality.¶

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ServiceOriented Cap Sub-Type  |   Length      | Res   |  MT   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       SO-CapValue                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- ServiceOriented Cap Sub-Type (16 bits):

5 (specified in this document).¶

- Length (8 bits):

Specifies the total length in octets, excluding the sub-Type and Length fields. For the ServiceOriented Cap Sub-Type, the Length SHOULD be 5.¶

- Res (4 bits):

These bits are reserved for future use and MUST be set to zero.¶

- MT (Metric Type)(4 bits):

An unsigned 4 bits integer. When the MT value is set to 0, it indicates the SoCapValue field contains a normalized metric derived from multiple metric types. The rules for deriving this normalized metric are out of scope of this document and defined by per-service. Additional metric types may be defined in future documents.¶

- SO-CapValue (32 bits):

The Service-Oriented Capability Abstract Value is an integer between 0 and 2^32-1. A larger number means higher capability, and a value of 0 indicates the site has the lowest relative capability for the service. The method used to derive this value is beyond the scope of this document.¶

Multiple Service-Oriented Capability Sub-TLVs with different metric types can be encoded in a Edge Metadata Path Attribute, indicating that multiple metrics are carried. However, if more than one Service-Oriented Capability Sub-TLVs with the same metric type are encoded in a Edge Metadata Path Attribute, only the first one will be processed and the others will be ignored in processing.¶

4.7. Service-Oriented Available Resource Sub-TLV

The "Service-Oriented Available Resource Sub-TLV" is for distributing a metric that measures the real-time avaiable resources allocated for processing specific services or applications at an edge site. This Sub-TLV complements the "Service-Oriented Capability Sub-TLV" described in Section 4.6, which addresses the static resource capability of a site for a service. While the Capability Abstract Value provides a baseline understanding of a site's potential to handle a service, the Available Resource metric offers a dynamic perspective by quantifying how much of this capacity is currently available. This distinction is crucial for managing resource efficiency and responsiveness in network operations, ensuring that capabilities are not only available but also optimally used to meet the actual service demands.¶

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|ServiceOriented Avail Sub-Type |   Length      |P| Res |  MT   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         SO-AvailRes                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- ServiceOriented Avail (Service-Oriented Available Resource) Sub-Type:

6 (specified in this document).¶

- Length (8 bits)

Specifies the total length in octets, excluding the sub-Type and the length field. For the ServiceOriented Available Resource Sub-Type, the Length SHOULD be 5.¶

- Flag (P):

Is a single-bit Percentage flag. When it is set to 1, it indicates the value is the Service-Oriented Available Resource in percentage. When the "P" flag is set to 0, the value in this Sub-TLV is the abstract value of the available resource.¶

- Res (3 bits):

These bits are reserved for future use and MUST be set to zero.¶

- MT (4 bits)

Metric Type. This document defines a default metric type as value 0, indicating this is the normalized metric derived by multiple type of metrics. The rules to derive the normalized metric are out of scope of this document and defined by the service. Other Metric Types could be defined by other documents in the future.¶

- SO-AvailRes (32 bits):

When the P-Flag bit is set to 1, Service-Oriented Available Resource Value is a percentage (0-100), with 0 indicating that 0% of the capability is available and 100 indicating that 100% of the capability is available. When the value is outside the 0-100 range, the value carried in this Sub-TLV is ignored. For example, Capacity value is 50 and the SO-AvailRes is 50 when P-flag is set, it means 50% of 50 unit of resource is available, while 25 unit of resource is available in this site for the service. When the P-flag is 0, then the value of this filed is the abstract value of the available resource. For example, When the capacity value is 50, and the SO-AvailRes is 50, it means all the resource is available.¶

Multiple Service-Oriented Available Resource Sub-TLVs with different metric types can be encoded in a Edge Metadata Path Attribute, indicating that multiple metrics are carried. However, if more than one Service-Oriented Available Resource Sub-TLVs with the same metric type are encoded in a Edge Metadata Path Attribute, only the first one will be processed and the others will be ignored in processing.¶

6.1. AS-Scope SubTLV

To address the potential issue where the NO-ADVERTISE well-known community of the BGP UPDATE message can be dropped by some routers, a new AS-Scope Sub-TLV can be included in the Metadata Path Attribute to prevent the Metadata Path Attribute from being leaked to unintended Autonomous Systems (ASes). The AS-Scope Sub-TLV will enforce stricter control over the propagation of the metadata by associating it with specific AS numbers.¶

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        AS-Scope Sub-Type      |   Length      | Reserved      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         In-Scope AS-Value                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- AS-Scope Sub-Type (16 bits):

7 (specified in this document).¶

- Length (8 bits)

Specifies the total length in octets, excluding the sub-Type and the length field. For the AS-Scope Sub-Type, the Length SHOULD be 6.¶

- Reserved (8 bits):

These bits are reserved for future use and MUST be set to zero.¶

- In-Scope AS-Value (32 bits):

AS value that is recognized by the BGP speaker in the domain.¶

7.1. Policy Application Order

To remain consistent with Section 9.1.1 of [RFC4271], metadata-aware policy evaluation MUST be applied after LOCAL_PREF has been set for iBGP routes, or after equivalent inbound policy has been applied for eBGP routes.¶

The use of Edge Metadata does not replace existing BGP routing policy. Rather, the Edge Metadata Path Attribute provides additional inputs that local policy MAY use when comparing candidate routes for selected services.¶

7.2. Metadata Selection by Local Policy

A deployment MAY use only a subset of the metadata attributes carried in the Edge Metadata Path Attribute. Which metadata attributes are considered, and for which services they are considered, is determined by local policy.¶

For example, one deployment may consider only the Service Delay Prediction Sub-TLV for latency-sensitive services, while another deployment may consider only availability-related or service-capability-related Sub-TLVs. A route that carries additional recognized metadata does not require all such metadata to be used in route selection.¶

If none of the recognized metadata carried by a route are selected by local policy for preference computation, the route is evaluated using ordinary BGP policy and tie-breaking procedures.¶

7.3. Policy-Based Preference Computation

For services configured for metadata-aware route selection, local policy MAY use one or more recognized metadata values carried in the Edge Metadata Path Attribute, together with other routing attributes, to derive a preference for each candidate route.¶

The procedure for combining recognized metadata with traditional BGP attributes is deployment specific and outside the scope of this document. The preference computation MAY be performed at a Route Reflector (RR), at an ingress node, or at another policy decision point within the same administrative domain.¶

When metadata-aware policy is applied to a set of candidate routes, the route with the most preferred policy outcome is selected. If two or more routes remain equally preferred after metadata-aware policy evaluation, the normal BGP tie-breaking procedures defined in [RFC4271] apply.¶

7.4. Policy Treatment of Routes with Degraded Metadata

Local policy MAY define threshold conditions for one or more metadata types. When the recognized metadata associated with a route indicates that such a threshold has been crossed, local policy MAY reduce the preference of that route or MAY treat the route as ineligible for metadata-aware service steering.¶

This document does not mandate a specific action for degraded metadata values. The action taken, if any, is determined by local policy. For example, local policy may de-prefer a route whose Service Delay Prediction exceeds a configured threshold, or a route whose availability-related metadata falls below a configured level.¶

If local policy excludes a route from metadata-aware service steering, the route MAY still remain valid for ordinary BGP reachability unless separate policy removes or suppresses that route.¶

7.5. Tie Breaking and ECMP

After metadata-aware policy evaluation, if multiple candidate routes remain equally preferred, BGP tie-breaking proceeds according to [RFC4271].¶

If the decision process results in multiple equally preferred paths and the deployment permits Equal Cost Multi Path (ECMP), those paths MAY be installed in the forwarding plane according to existing BGP procedures and platform capabilities.¶

12.1. Edge Metadata Path Attribute

IANA has done early allocation [RFC7120] of the codepoint 42 to the "Edge Metadata Path Attribute" in the "BGP Path Attributes" registry in the BGP Parameters registry group. The reference for this assignment is [this document].¶

    +=======+======================================+=================+
    | Value |             Description              |    Reference    |
    +=======+======================================+=================+
    |  42   |   Edge Metadata Path Attribute       | [this document] |
    +-------+--------------------------------------+-----------------+

12.2. Edge Metadata Capability Code

IANA has assigned a Capability Code of 78 from the "BGP Capability Codes" registry in "Capability Codes registry group" for the Edge Metadata Capability in the BGP OPEN message.¶

    +=======+======================================+=================+
    | Value |             Description              |    Reference    |
    +=======+======================================+=================+
    |  78   |     Edge Metadata Capability         | [This document] |
    +-------+--------------------------------------+-----------------+

12.3. Edge Metadata Path Attribute Sub-Types

IANA is requested to create a new sub-registry under the Edge Metadata Path Attribute registry as follows:¶

Name:

Sub-TLVs under the "Edge Metadata Path Attribute"¶

Registration Procedure:

Expert Review [RFC8126].¶

Detailed Expert Review procedure will be added per [RFC8126].¶

Reference:

[this document]¶

+========+=============================+===================+
|Sub-Type|   Description               | Reference         |
+========+=============================+===================+
|      0 |reserved                     |[this document ]   |
+--------+-----------------------------+-------------------+
|      1 |Site Preference Index        |[this document:4.3]|
+--------+-----------------------------+-------------------+
|      2 |Site Physical Avail Index    |[this document:4.4]|
+--------+-----------------------------+-------------------+
|      3 |Service Delay Predication    |[this document:4.5]|
+--------+-----------------------------+-------------------+
|      4 |Raw Measurement              |[this document:4.6]|
+--------+-----------------------------+-------------------+
|      5 |Service-Oriented Capability  |[this document:4.7]|
+--------+-----------------------------+-------------------+
|      6 |Service-Oriented Available   |                   |
|        |Resource                     |[this document:4.8]|
+--------+-----------------------------+-------------------+
|      7 |AS-Scope                     |[this document:5.1]|
+--------+-----------------------------+-------------------+
|8-65534 | unassigned                  |                   |
+--------+-----------------------------+-------------------+
|  65535 | reserved                    |[this document]    |
+--------+-----------------------------+-------------------+

15.1. Normative References

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC4271]

Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, January 2006, <https://www.rfc-editor.org/info/rfc4271>.

[RFC4360]

Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, February 2006, <https://www.rfc-editor.org/info/rfc4360>.

[RFC4364]

Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 2006, <https://www.rfc-editor.org/info/rfc4364>.

[RFC4760]

Bates, T., Chandra, R., Katz, D., and Y. Rekhter, "Multiprotocol Extensions for BGP-4", RFC 4760, DOI 10.17487/RFC4760, January 2007, <https://www.rfc-editor.org/info/rfc4760>.

[RFC4761]

Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007, <https://www.rfc-editor.org/info/rfc4761>.

[RFC4786]

Abley, J. and K. Lindqvist, "Operation of Anycast Services", BCP 126, RFC 4786, DOI 10.17487/RFC4786, December 2006, <https://www.rfc-editor.org/info/rfc4786>.

[RFC5291]

Chen, E. and Y. Rekhter, "Outbound Route Filtering Capability for BGP-4", RFC 5291, DOI 10.17487/RFC5291, August 2008, <https://www.rfc-editor.org/info/rfc5291>.

[RFC5492]

Scudder, J. and R. Chandra, "Capabilities Advertisement with BGP-4", RFC 5492, DOI 10.17487/RFC5492, February 2009, <https://www.rfc-editor.org/info/rfc5492>.

[RFC5905]

Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, <https://www.rfc-editor.org/info/rfc5905>.

[RFC6513]

Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February 2012, <https://www.rfc-editor.org/info/rfc6513>.

[RFC7120]

Cotton, M., "Early IANA Allocation of Standards Track Code Points", BCP 100, RFC 7120, DOI 10.17487/RFC7120, January 2014, <https://www.rfc-editor.org/info/rfc7120>.

[RFC7432]

Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, <https://www.rfc-editor.org/info/rfc7432>.

[RFC8126]

Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, <https://www.rfc-editor.org/info/rfc8126>.

[RFC8277]

Rosen, E., "Using BGP to Bind MPLS Labels to Address Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017, <https://www.rfc-editor.org/info/rfc8277>.

[RFC9012]

Patel, K., Van de Velde, G., Sangli, S., and J. Scudder, "The BGP Tunnel Encapsulation Attribute", RFC 9012, DOI 10.17487/RFC9012, April 2021, <https://www.rfc-editor.org/info/rfc9012>.

15.2. Informative References

[ATTRIBUTE-ESCAPE]

J. Haas, "BGP Attribute Escape", July 2023, <https://datatracker.ietf.org/doc/draft-haas-idr-bgp-attribute-escape/>.

[IANA-BGP-PARAMS]

IANA, "BGP Path Attributes", BGP Path Attributes https://www.iana.org/assignments/bgp-parameters/.

[RFC1136]

Hares, S. and D. Katz, "Administrative Domains and Routing Domains: A model for routing in the Internet", RFC 1136, DOI 10.17487/RFC1136, December 1989, <https://www.rfc-editor.org/info/rfc1136>.

[RFC4456]

Bates, T., Chen, E., and R. Chandra, "BGP Route Reflection: An Alternative to Full Mesh Internal BGP (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, <https://www.rfc-editor.org/info/rfc4456>.

[RFC8174]

Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8799]

Carpenter, B. and B. Liu, "Limited Domains and Internet Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, <https://www.rfc-editor.org/info/rfc8799>.

[TS.23.501-3GPP]

3rd Generation Partnership Project (3GPP), "System Architecture for 5G System; Stage 2, 3GPP TS 23.501 v2.0.1", December 2017.

B.1. Egress Router Behavior

Multiple instances of the same service could be attached to one egress router. When all instances of the same service are grouped behind one application layer load balancer, they appear as one single route to the egress router, i.e., the application loader balancer's prefix. Under this scenario, the compute metrics for all those instances behind one application layer balancer are aggregated under the application load balancer's prefix. In this case, the compute metrics aggregated by the Load Balancer are visible to the egress router as associated with the Load Balancer's prefix. However, how the application layer Load Balancers distribute the traffic among different instances is out of the scope of this document. When multiple instances of the same service have different paths or links reachable from the egress router, multiple groups of metrics from respective paths could be exposed to the egress router. The egress router can have preconfigured policies on aggregating various metrics from different paths and the corresponding policies in selecting a path for forwarding the packets received from ingress routers. The aggregated metrics can be carried in the BGP UPDATE messages instead of detailed measurements to reduce the entries advertised by the control plane and dampen the routes update in the forwarding plane. Upon receiving packets from ingress routers, the egress router can use its policies to choose an optimal path to one service instance. It is out of the scope of this document how the measurements are aggregated on egress routers and how ingress routers are configured with the algorithms to integrate the aggregated metrics with network layer metrics.¶

Many measurements could impact and correspondingly reflect service performance. In order to simplify an optimal selection process, egress routers can have preconfigured policies or algorithms to aggregate multiple metrics into one simple one to ingress routers. Though out of the scope of this document, an egress router can also have an algorithm to convert multiple metrics to network metrics, an IGP cost for each instance, to pass to ingress nodes. This decision-making process integrates network metrics computed by traditional IGP/BGP and the service delay metrics from egress routers to achieve a well-informed and adaptive routing approach. This intelligent orchestration at the edge enhances the service's overall performance and optimizes resource utilization across the distributed infrastructure. When the egress has merged the compute metrics from the local sites behind it, it can include one or more aggregated compute metrics in the Metadata Path Attribute in the BGP UPDATE to the Ingress. Also, an identifier or flag can be carried to indicate that the metrics are merged ones. After receiving the routes for the Service ID with the identifier, the ingress would do the route selection based on pre-configured algorithms (see Section 3 of this document).¶

B.2. Integrating Network Delay with the Service Metrics

As the service metrics and network delays are in different units, here is an exemplary algorithm for an ingress router to compare the cost to reach the service instances at Site-i or Site-j.¶

                ServD-i * CP-j               Pref-j * NetD-i
Cost-i=min(w *(----------------) + (1-w) *(------------------))
                ServD-j * CP-i               Pref-i * NetD-j

CP-i:

Capacity Availability Index at Site-i. A higher value means higher capacity available.¶

NetD-i:

Network latency measurement (RTT) to the Egress Router at the site-i.¶

Pref-i:

Preference Index for Site-i, a higher value means higher preference.¶

ServD-i:

Service Delay Predication Index at Site-i for the service, i.e., the ANYCAST address [RFC4786] for the service.¶

w:

Weight is a value between 0 and 1. If smaller than 0.5, Network latency and the site Preference have more influence; otherwise, Service Delay and capacity availability have more influence.¶

When a set of service Metadata is converted to a simple metric, a decision process is determined by the metric semantics and deployment situations. The goal is to integrate the conventional network decision process with the service Metadata into a unified decision-making process for path selection.¶

B.3. Integrating with BGP Route Selection

Not all metadata attributes specified in this document are intended for use in every deployment. Each deployment may choose to consider only a subset of the available metadata attributes based on its specific service requirements.¶

- Deployment-Specific Attribute Selection:¶

A deployment may prioritize only certain metadata attributes relevant to its operational needs. For example, one deployment might only use the Service Delay Prediction Index for latency-sensitive applications, while another might focus solely on the Capacity Availability Index to manage resource availability.¶

- Influence on BGP Decision Process:¶

The edge service Metadata influences next-hop selection differently from traditional BGP metrics (e.g., Local Preference, MED). Unlike a general next-hop metric that can affect many routes, edge service Metadata selectively impacts optimal next-hop selection for specific routes configured to consider these service-specific attributes. This targeted influence allows for optimized path selection without disrupting broader route decisions.¶

- Handling Degraded Metrics (Policy-Based):¶

If a service-specific metric degrades beyond a configured threshold (e.g., the Service Delay Prediction Index exceeds an acceptable delay threshold or the Capacity Availability Index drops below a required level), the ingress router will treat that route as ineligible for traffic steering. This is similar to a BGP route withdrawal, where the degraded route is deprioritized or ignored, even if traditional BGP attributes would otherwise favor it. This ensures that traffic is directed only to service instances that meet the defined performance criteria.¶

- Fallback to Non-Metadata Routes:¶

If no suitable routes with the required metadata are available, the BGP decision process defaults to traditional attribute evaluation [RFC4271], ensuring consistent routing even when metadata-specific paths are absent.¶

This approach provides flexibility and adaptability in routing decisions, allowing each deployment to apply relevant metadata attributes and enforce performance thresholds for improved service quality.¶