QP-based SRv6 Load Balancing Deployment

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 30 August 2026.¶

Copyright Notice

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

The proliferation of RDMA technology in Intelligent Computing Data Center (DC) fabrics has revolutionized high-performance computing, distributed storage, and machine learning workloads.¶

These workloads generate large, predictable flows that demand ultra-low latency, high bandwidth, and precise congestion control to ensure optimal performance. Traditional networking methods, like hash-based Equal-Cost Multi-Path (ECMP) load balancing, struggle with insufficient entropy due to the low diversity of RDMA (specifically RDMA over Converged Ethernet v2, abbreviated as RoCEv2) [IBTA-SPEC] flow identifiers. This often results in fabric hotspots, network congestion, and performance degradation.¶

The transmission process of RoCEv2 messages in intelligent computing Wide Area Network (WAN) used for Data Center Interconnection (DCI) is the same as inside the DC, and it will also generate elephant streams, which leads to fabric hotspots, network congestion, and performance degradation.¶

Segment Routing over IPv6 (SRv6) [RFC8986] provides flexible traffic engineering by supporting policy-based programmability and explicit path steering. SRv6 policy enables deterministic path steering and fine-grained traffic control for RoCEv2 flows, ensuring predictable performance.¶

This document details SRv6 path selection based on Queue Pair (QP) to optimize load balancing for predictable RoCEv2 flows in intelligent computing WAN by ensuring all packets within a QP follow the same path.¶

2. Terminology

The following terms are used in this document:¶

• QP (Queue Pair): A communication endpoint in RDMA architecture, identified by a 24-bit or 32-bit value.¶
• BTH (Base Transport Header): The RDMA transport header containing QP information.¶
• SRv6 Policy: An ordered list of segments (SIDs) representing a path through the SRv6 network.¶
• SL (Segment List): An ordered list of SIDs in an Segment Routing Header (SRH) [RFC8754].¶
• ECMP (Equal-Cost Multi-Path): A routing technique for load-balancing traffic across multiple best-path routes.¶

3. Problem Statement

Traditional ECMP load balancing faces several challenges with RoCEv2 flow:¶

• Insufficient Entropy: The relatively static 5-tuple of RoCEv2 flows provides limited entropy, which leads to poor load balancing of elephant flows.¶
• Persistent Hotspots: When multiple elephant flows hash to the same path, they create persistent hotspots that cannot be resolved without manual intervention or flow termination.¶
• Poor Failure Convergence: When a link fails, all flows previously using that link are rehashed to remaining paths. This sudden influx of elephant flows can overwhelm the remaining links, causing secondary congestion.¶
• Lack of Application Awareness: ECMP operates purely on packet header fields without understanding the application-level semantics of QPs.¶

4. QP-based SRv6 Path Selection

By encoding an ordered list of segments in the packet header, SRv6 (Policy) allows the ingress device to directly steer RoCEv2 workload traffic through the fabric.¶

FlowSpec, as a traffic scheduling tool, can guide RoCEv2 flows to different SRv6 policies based on their characteristics (such as Dest QP), and forward them along different paths. QP-based FlowSpec protocol extensions are beyond the scope of this document.¶

QP-to-SRv6 Policy Mapping:¶

• Upon ingress, the RoCEv2 packet is parsed to extract the destination QP identifier from its Base Transport Header (BTH).¶
• When multiple SRv6 policies exist, the destination QP is mapped to a corresponding SRv6 Policy via a pre-configured mapping table. The mapping table can rely on local configuration or the flowspec mechanism.¶

Enhanced Hash-Based Segment List (SL) Scheduling:¶

• For the selected SRv6 Policy (which may contain multiple SLs), an enhanced hash algorithm, using the QP as a key input, deterministically selects one specific SL for use.¶
• The chosen SRv6 SL is applied to the RoCEv2 packet, which is then forwarded accordingly.¶

5. Deployment Illustration

A typical WAN topology for DCI is shown in the figure below.¶

               +-------------+
               |     DC1     |
               |             |
               |  QP1 ~ QP4  |
               +------+------+
+---------------------|---------------------+
|                +----+----+          WAN   |
|                |   PE1   |                |
|                +----+----+                |
|                     |                     |
|     +---------+-----+-----+---------+     |
|     |         |           |         |     |
|  +--+--+   +--+--+     +--+--+   +--+--+  |
|  |  P1 |   |  P2 |     |  P3 |   |  P4 |  |
|  +--+--+   +--+--+     +--+--+   +--+--+  |
|     |         |           |         |     |
|     +---------+-----+-----+---------+     |
|                     |                     |
|                +----+----+                |
|                |   PE2   |                |
|                +----+----+                |
+---------------------|---------------------+
               +------+------+
               |     DC2     |
               |             |
               |  QP1 ~ QP4  |
               +-------------+

The topology consists of two Provider Edge (PE) devices, and each of the PEs is connected to four Provider (P) devices and one DC.¶

In this example, there are 2 DCs, in which four QPs can be established to transmit RoCEv2 workloads.¶

In the above topology, there are four paths that pass through WAN from DC1 to DC2.¶

all paths is below:¶

• *Path1*: PE1 -> P1 -> PE2¶
• *Path2*: PE1 -> P2 -> PE2¶
• *Path3*: PE1 -> P3 -> PE2¶
• *Path4*: PE1 -> P4 -> PE2¶

6. Operational Considerations

In ingress device, the control plane must support QP range to SRv6 Policy mapping by protocol extension or local configuration. For non-RoCEv2 traffic, the system MUST revert to the standard five-tuple hash for SL selection.¶

The ingress devices require deep packet inspection capability to parse BTH headers, programmable hash engines with configurable input fields, sufficient TCAM/SRAM for QP classification mapping tables, and support for multiple active SRv6 policies with multiple SLs.¶

When network congestion or failure occurs, operators can flexibly configure QP range to SRv6 Policy mapping strategies on the ingress device to guide RoCEv2 flows to the appropriate path.¶

7. Security Considerations

Malicious actors could spoof QP values to bypass mapping policies, cause hash collisions, or exhaust specific network paths. Mitigations may include cryptographic validation of RoCEv2 packets, and QP whitelisting/blacklisting.¶

QP values may reveal application-level information, so QP values SHOULD be anonymized or encrypted.¶

The additional packet processing (such as parsing BTH headers) could be exploited for Denial of Service (DoS) attacks; therefore, implementations MUST support graceful degradation mechanisms (such as rate limiting) under attack.¶

8. IANA Considerations

This document has no IANA actions.¶

Authors' Addresses