Precise Priority-based Flow Control Notification

Introduction Driven by the rapid development of big data and artificial intelligence (AI) technologies, the demand for high-performance and low-latency data transmission has become critical across various industries. Data transfers typically rely on transport layer protocols such as the Transmission Control Protocol (TCP), Quick UDP Internet Connections (QUIC), or Remote Direct Memory Access over Converged Ethernet (RoCEv2). These protocols primarily employ end-to-end congestion control algorithms, which adjust sending rates based on network status feedback by manipulating congestion windows. Traditional end-to-end congestion control mechanisms depend on signals such as:

Explicit Congestion Notification (ECN), where packets are marked by congested queues to signal the sender to slow down.
Measurements of delay and bandwidth (e.g., Round-trip Time (RTT), One-Way Delay (OWD)), used to infer network status and available capacity.
In-band Network Telemetry (INT), which provides fine-grained per-packet queuing information.

A fundamental limitation of these end-to-end approaches is that the congestion feedback path is coupled with the data return path. When congestion occurs, the notification must traverse the reverse path to the sender, incurring at least one RTT of delay. In long-distance or highly dynamic networks, this delay prevents timely rate adjustment, leading to prolonged congestion, increased latency, and packet loss.

Motivation and Overview Precise Priority-based Flow Control (PPFC) can address this limitation by enabling per-flow control to instantly pause the traffic sources based on explicit notifications from the network. The proposed PPFC mechanism operates as follows: When a network node (e.g., a switch or router) detects congestion it can generate a PPFC Notification message. This message is sent to the ingress gateway(s) from which the congested flows originated. The notification contains key information such as:

The congested destination identifier (e.g., IP address).
A recommended flow control action (e.g., PAUSE) and its duration.
Optional context about the congestion severity.

This document specifies the notification mechanism for PPFC, defining the message formats and network actions for communicating congestion information. The PPFC notification enables rapid congestion signaling, allowing per-flow flow control for traffic from source.

Conventions Used in This Document

Abbreviations RTT: Round-Trip Time TCP: Transfer Control Protocol RDMA: Remote Direct Memory Access Round-Trip Time QUIC: Quick UDP Internet Connections ECN: Explicit Congestion Notification PFC: Priority-based Flow Control PPFC: Precise Priority-based Flow Control RoCEv2: RDMA over Converged Ethernet version 2 QP: Queue Pair

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

PPFC Notification When the per-flow congestion occurs at Node Y, it will trigger the PPFC notification in ICMP or UDP message to the ingress or the gateway or the client in RoCEv2 message. This document defines the message formats and network actions for communicating congestion information to the ingress or the gateway. |Gateway|<--->|Node X |<--->|Node Y |<--->|Server | +--------+ +----->+-------+ +-------+ +-------+ +-------+ Traffic| Congestion +--------+ | Occurs |Client B+<-----+ +--------+ Figure 1: PPFC Notification ]]>

PPFC Notification Message Formats

ICMP Message Format The PPFC Notification message sent from gateway may be a ICMP message which is formatted as Figure 2.

Type and Code: TBD 1 (8bits) and TBD2 (8bits), the fields indicate the PPFC notification type and code.
PT (2bits): indicates the action type of the PPFC notification for the ingress to take actions of traffic. It can be set to 00 "stop", 01 "resume", 10 "alarm", 11 "hold".
Priority (8bits): Corresponds to IP DSCP or flow priority.
Router Identifier (variable): indicates IPv4 or IPv6 address of the congested node.
Congestion Port (16bits): Identifier of the congested port.
Pause Duration (16bits): Recommended pausing interval, in microseconds.
Source Address (variable): indicates the IPv4 or IPv6 address of the sender.
Destination Address (variable): indicates the IPv4 or IPv6 address of the receiver.
Flow Identifier (variable): It indicates the IP 5 tuples when transmitting TCP or QUIC data and it also can be mapped at ingress to source and destination QP when transmitting RDMA data.

UDP Message Format The PPFC Notification message sent from gateway may be a UDP message which is formatted as Figure 3. Where:

UDP Header: The UDP header as specified in includes the UDP source port, UDP destination port, UDP length, and UDP checksum.
UDP Destination port: TBD3, indicates a new well-known UDP destination port needs to be allocated for this PPFC notification message.
The other fields are identical to the fields defined in Section 4.1.

Security Considerations To be discussed in future versions of this document.

IANA Considerations This document requests IANA to allocate a new ICMP message type and UDP port.