fann Z. Han, Ed. Internet-Draft R. Pang Intended status: Standards Track Y. Yue Expires: 6 January 2027 China Unicom J. Dong Huawei Technologies Z. Ruan China Unicom Q. Xiong ZTE Corporation 5 July 2026 Use cases and Requirement for Flow Control Collaboration Across DCNs and WAN draft-han-fann-codeployment-pfc-fgfc-00 Abstract The demand for lossless network transmission and the application of flow control mechanisms have expanded from DCNs (Data Center Networks) to WANs(Wide Area Networks). To mitigate PFC - related issues in WANs, the fine - grained flow control is proposed. This mechanism aims to achieve precise control at flow / tenant levels, limits flow control to specified paths and slices, and provides intelligent congestion backpressure. As current DCN already adopts PFC mechanisms, the fine-grained flow control in WANs needs to work with PFC in DCNs to achieve end-to-end flow control. This document describes the use cases and requirements for the collaboration of flow control mechanisms across DCNs and WANs. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 6 January 2027. Han, et al. Expires 6 January 2027 [Page 1] Internet-Draft Req of PFC and fine-grained flow control July 2026 Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction and Background {#intro and backg} . . . . . . . 2 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Collaboration deployment scenarios of PFC and fine-grained Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Interworking between PFC and fine-Grained Flow Control . . . 5 4.1. PFC to fine-grained flow control . . . . . . . . . . . . 5 4.2. Fine-grained flow control to PFC . . . . . . . . . . . . 6 5. Requirement of collaboration deployment . . . . . . . . . . . 8 6. Security Considerations . . . . . . . . . . . . . . . . . . . 8 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 8. Appendix A. Buffer Requirement for Fine-Grained Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . 8 8.1. A.1. Principles of Fine-Grained Flow Control . . . . . . 9 8.2. A.2. Buffer Requirement Formula . . . . . . . . . . . . 9 8.3. A.3. Example . . . . . . . . . . . . . . . . . . . . . . 10 9. Informative References . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction and Background {#intro and backg} DCNs are typically characterized by a limited network scale, short path and predictable traffic patterns, so flow control mechanisms like PFC (Priority Flow Control) and ECN (Explicit Congestion Notification) operate effectively. With the growth of AI LLM distributed training and inference, lossless transmission of massive data between geographically separated data centers is required [I-D.hs-rtgwg-wan-lossless-uc], and the flow control mechanisms need to be extended from DCNs to WANs. Unlike DCNs, WANs are large-scale with complex topologies, long paths, and diverse traffic type. PFC based on port-level feedback ensures lossless transmission of RDMA protocol, by pausing/resuming specific priority queues to prevent congestion. When using it in the WANs, the backpressure from PFC Han, et al. Expires 6 January 2027 [Page 2] Internet-Draft Req of PFC and fine-grained flow control July 2026 will cause head-of-line blocking, deadlocks, and congestion spreading, which degrade network throughput. To mitigate these issues in [I-D.ietf-rtgwg-net-notif-ps], the fine - grained flow control is required for WANs. Fine-grained flow control improves upon the coarse-grained port-based PFC mechanism. It enables precise control at the flow, tenant, or other granular levels, limits flow control to specified paths and slices, and provides intelligent congestion backpressure with granular parameters (pausing time, backpressure bandwidth, flow identifier etc.). These capabilities collectively contribute to achieving efficient and refined flow control in WANs [I-D.han-rtgwg-fine-grained-backpressure]. This draft focuses on the scenarios where PFC is employed in DCNs and the fine-grained flow control is adopted in WANs. Such cross-domain coordination relies on fast notifications triggered by congestion events, with data-plane optimized processing to meet the millisecond- level response time. This document describes the use cases and requirements for interworking of the two flow control mechanisms, enabling end-to-end lossless transmission through coordination and policy mapping of flow control information between DCNs and WANs.This work also offers practical insights for rapid, scalable network notifications in DCI environments, where timely reaction to local congestion events is critical for application performance and network stability. 2. Terminology PFC: Priority-based Flow Control DCN: Data Center Network WAN: Wide Area Network RDMA: Remote Direct Memory Access RoCE: RDMA over Converged Ethernet 3. Collaboration deployment scenarios of PFC and fine-grained Flow Control Han, et al. Expires 6 January 2027 [Page 3] Internet-Draft Req of PFC and fine-grained flow control July 2026 +----------+ +----------+ -- | Data | | Data |-- ^ | center A | | center B | ^ | +----------+ +----------+ | | | | | |PFC | | PFC| | v v | v +----+ ----> +----+ ----> +----+ ----> +----+ ----> +----+ v -- | R1 | | R2 | | R3 | | R4 | | R5 | -- +----+ +----+ +----+ +----+ +----+ | | |-------------------------------------------------->| fine-grained flow control over WAN Figure 1: Codeployment of PFC and fine-grained flow control As shown in Figure 1, there are two data centers, A and B.The internal nodes of data center A and data center B employ the PFC mechanism. Because most DCN NICs today are optimized for legacy protocols (e.g., Ethernet, DCB) and lack SRv6 processing capabilities. This limitation prevents the direct extension for refined flow control. Hardware/firmware upgrades are needed to enable fine-grained flow control deployment. All or some of the WAN nodes R1-R5 support fine-grained flow control [I-D.han-rtgwg-fine-grained-backpressure] to mitigate PFC backpressure issues, enabling flow/tenant-level congestion handling with granular parameters for precise and intelligent backpressure. These nodes also support HQoS (Hierarchical Quality of Service) queuing mechanisms and slicing. Edge nodes R1 and R5 support both PFC and fine-grained flow control[I-D.han-rtgwg-fine-grained-backpressure] , interworking DCN and WAN flow control mechanisms and ensuring seamless end-to-end flow control. The NNI ports of edge nodes R5 and R1 can establish multiple slices, each corresponding to a tenant and supporting 1-8 queues. Based on factors such as distance, number of users, network topology and node capabilities in WAN, the interworking and collaboration scenarios of PFC and fine-grained flow control can be classified into the following two categories. 1) Single-hop direct interconnection without intermediate node participation Han, et al. Expires 6 January 2027 [Page 4] Internet-Draft Req of PFC and fine-grained flow control July 2026 In this scenario, Data Center A and Data Center B are directly connected through edge node R1 and R5 with a single hop, without any intermediate devices in between, or intermediate nodes (R2, R3, R4) are legacy network devices that whithout flow control capability. In this case, intermediate nodes do not participate in the flow control process, and a tunnel is established between R1 and R5 to transmit fine-grained flow control packet. 2) Multi-hop interconnection scenario with intermediate node participation In this scenario, Data Center A and Data Center B are connected by WAN via nodes R1 -> R2 -> R3 -> R4 -> R5.Intermediate nodes R2, R3, and R4 (or a subset of them) support fine-grained flow control capabilities. These intermediate nodes actively participate in the flow control process. The WAN nodes can adopt either hop-by-hop backpressure or cross-hop Backpressure mechanisms [I-D.ruan-spring-priority-flow-control-sid] to handle congestion. 4. Interworking between PFC and fine-Grained Flow Control 4.1. PFC to fine-grained flow control tenant traffic |------------> +--------------+ | Slice ID = 1 | +--------------+ Congestion Occurs | | | | v v ----> +--- + -2/0/0 1/0/0- +----+ -2/0/0 3/0/0- +----------+ | Rx | --------------> | R5 | --------------> | Data | | | | | | center B | +----+ +----+ +----------+ <- - - - - - - -| <- - - - - - - -| fine-grained flow control PFC backpressure backpressure packet frame ^ | | +--------------+ | Slice ID = 1 | +--------------+ +--------------+ | Slice ID = N | +--------------+ Han, et al. Expires 6 January 2027 [Page 5] Internet-Draft Req of PFC and fine-grained flow control July 2026 Figure 2: PFC to fine-grained flow control Edge node R5 responds to the PFC frame sent by the data center and transmits fine - grained flow control packet [I-D.han-rtgwg-fine-grained-backpressure] to the WAN. The process follows these steps: 1) When congestion occurs at the incoming port 3/0/0 of data center B. 2) The data center B sends a PFC backpressure frame to the 2/0/0 port of edge node R5. The PFC frame carries the queue priority of the traffic to be backpressured, which is af1. 3) Edge node R5 needs to support responding to the PFC frame and buffers the traffic with the priority af1 through the 2/0/0 physical port. 4) The 1/0/0 port of edge device R5 has multiple slices. When the buffer queue corresponding to the 2/0/0 port of edge device R5 reaches the buffer threshold. 5) According to the port, tenant traffic, and slice mapping relationship, the 1/0/0 port of edge device R5 sends a fine - grained flow control backpressure packet to the network node Rx. Rx is the upstream network node with fine-grained flow control capability, it can be the intermediate node or the edge node R1 in WAN for different deployment scenarios mentioned in clause 3. The packet carries the tenant traffic information to be backpressured, with the queue priority af1, sliceID, and pause time, etc. 6) Based on the congestion handling situation, if the RX node fails to resolve the congestion: * For multi-hop scenario mentioned in clause 3 , the RX node sends fine-grained flow control packets to upstream WAN nodes as needed; * For the single-hop scenario mentioned in clause 3, where RX is edge node R1, the RX node sends PFC frames to the DCN as needed. 4.2. Fine-grained flow control to PFC Han, et al. Expires 6 January 2027 [Page 6] Internet-Draft Req of PFC and fine-grained flow control July 2026 +--------------+ | Slice ID = 1 | +--------------+ | Congestion Occurs | | v | tenant traffic v |---------------------------------------------> +----------+ -3/0/0 2/0/0- +----+ -1/0/0- +----+ | Data | --------------> | R1 | ---------> | Rx | | Center A | | | | | +----------+ +----+ +----+ <- - - - - - - -| <- - - - -| PFC backpressure frame fine-grained flow control backpressure ^ | | +--------------+ | Slice ID = 1 | +--------------+ Figure 3: fine-grained flow control to PFC Edge node R1 responds to fine - grained flow control packet[I-D.han-rtgwg-fine-grained-backpressure] from WAN, then sends PFC frame to the data center. The process follows these steps: 1) When congestion occurs in the traffic of queue af1 with sliceID = 1 at the egress port of network node R2. 2) Network node Rx sends a fine - grained flow control backpressure packet to edge node R1. This packet carries the tenant traffic information to be backpressured, with the queue priority af1, sliceID = 1, and the pause timed, etc. 3) Edge node R1 performs traffic control and buffers the tenant traffic with priority af1 and sliceID = 1. 4) When the buffer queue corresponding to port 1/0/0 of edge device R1 reaches the buffer threshold, port 2/0/0 of edge node R1 sends backpressure to the data center according to the standard PFC packet. 5) Data center A performs standard PFC backpressure and stops all traffic with priority af1 destined for port 3/0/0. Han, et al. Expires 6 January 2027 [Page 7] Internet-Draft Req of PFC and fine-grained flow control July 2026 5. Requirement of collaboration deployment Requirement 1: Edge node needs support the coordination and bidirectional translation between the fine-grained flow control mechanism in the WAN and the PFC mechanism in the DCN, enabling seamless end-to-end flow control across WAN and DCN domains. Requirement 2: Edge node needs to respond to PFC frames from the DCN. It includes the following capabilities: 1) Learn task flow-to-port mappings to identify affected traffic; 2) Configure appropriate buffer thresholds; 3) Generate and send fine-grained flow control messages to WAN nodes with granular parameters. Requirement 3: Edge nodes needs to respond to fine-grained flow control messages from the WAN. It includes the following capabilities: 1) Use established flow-to-port mappings to determine target DCN ports; 2) Configure appropriate buffer thresholds; 3) Generate and send standard PFC frames to corresponding DCN ports. 6. Security Considerations This document does not introduce any new security considerations. 7. IANA Considerations This document has no IANA actions. 8. Appendix A. Buffer Requirement for Fine-Grained Flow Control This appendix provides supplementary guidelines for buffer requirement on network devices in the context of fine-grained flow control in WANs. Han, et al. Expires 6 January 2027 [Page 8] Internet-Draft Req of PFC and fine-grained flow control July 2026 8.1. A.1. Principles of Fine-Grained Flow Control The fine-grained flow control mechanism operates by creating a tenant queue for each tenant on the network device, with a configurable backpressure threshold for each queue. The backpressure threshold may be implemented using either a single-watermark mechanism T or a dual-watermark mechanism (T-OFF and T-ON). In the dual-watermark mechanism, T-OFF is the backpressure trigger threshold, and T-ON is the recovery threshold. When congestion is detected on a particular queue (i.e., the occupied buffer exceeds threshold T/T-OFF), it sends a precise congestion notification message to the upstream device,instructing it to stop or reduce the sending rate for that tenant's traffic. When the occupied buffer falls back below threshold T/T-ON, the congestion is relieved and the upstream device resumes normal transmission. The backpressure threshold T/T-OFF shall be set to a value greater than or equal to (RTT + Congestion_Detection_Time) × PIR (Peak Information Rate) of the tenant.The recovery threshold T-ON is typically set based on operational experience, with a value lower than the backpressure threshold T-OFF. 8.2. A.2. Buffer Requirement Formula After the backpressure threshold T/T-OFF is triggered, the device needs to reserve dedicated buffer space (Headroom) to accommodate the packets that continue to arrive before the upstream device responds and stops sending. To ensure lossless operation, a larger Headroom valu is preferred. The Headroom Shall be greater than (RTT + Congestion_Detection_Time) × PIR. Therefore, the total buffer requirement for a network device is: Buffer > T（T-OFF) + Headroom That is: Buffer > 2 × (RTT + Congestion_Detection_Time) × PIR where: - RTT is the round-trip time between the local device and the upstream device that supports the flow control capability. For stable links and network environments, a fixed value may be configured, or dynamic measurement mechanisms such as TWAMP may be used for periodic acquisition. - Congestion_Detection_Time is the time from when the device locally detects tenant congestion to when it sends the precise congestion notification message. Han, et al. Expires 6 January 2027 [Page 9] Internet-Draft Req of PFC and fine-grained flow control July 2026 8.3. A.3. Example Assuming both R1 and R2 support fine-grained flow control capability, the optical fiber distance between them adds 1 ms of RTT per 100 km, the congestion detection time is 3 ms, and the tenant PIR is 10 Gbps. +----------+------------------+-------------+-------------------+-------------+ | Scenario | Tenant PIR | WAN Distance| RTT + Detection | Required | | | | | Time | Buffer | +----------+------------------+-------------+-------------------+-------------+ | 1 | 10 Gbps | 100 km | 1 ms + 3 ms = 4ms | 10 MB | +----------+------------------+-------------+-------------------+-------------+ | 2 | 10 Gbps | 200 km | 2 ms + 3 ms = 5ms | 12.5 MB | +----------+------------------+-------------+-------------------+-------------+ | 3 | 10 Gbps | 300 km | 3 ms + 3 ms = 6ms | 15 MB | +----------+------------------+-------------+-------------------+-------------+ | 4 | 10 Gbps | 400 km | 4 ms + 3 ms = 7ms | 17.5 MB | +----------+------------------+-------------+-------------------+-------------+ | 5 | 10 Gbps | 500 km | 5 ms + 3 ms = 8ms | 20 MB | +----------+------------------+-------------+-------------------+-------------+ Figure 4: Buffer requirements for fine-grained flow control at 10 Gbps PIR over varying WAN distances 9. Informative References [I-D.hs-rtgwg-wan-lossless-uc] Zhengxin, H., He, T., Shi, H., and T. Zhou, "Use Cases and Requirements for Implementing Lossless Techniques in Wide Area Networks", Work in Progress, Internet-Draft, draft- hs-rtgwg-wan-lossless-uc-01, 2 July 2025, . [I-D.ietf-rtgwg-net-notif-ps] Dong, J., McBride, M., Clad, F., Zhang, Z. J., Zhu, Y., Xu, X., Zhuang, R., Pang, R., Lu, H., Liu, Y., Contreras, L. M., Mehmet, D., and R. Rahman, "Fast Network Notifications Problem Statement", Work in Progress, Internet-Draft, draft-ietf-rtgwg-net-notif-ps-02, 7 May 2026, . Han, et al. Expires 6 January 2027 [Page 10] Internet-Draft Req of PFC and fine-grained flow control July 2026 [I-D.han-rtgwg-fine-grained-backpressure] Zhengxin, H., Ruan, Z., Pang, R., Yue, Y., Yao, J., and Q. Xiong, "Fine-Grained Flow Control Backpressure Mechanism for Wide Area Networks", Work in Progress, Internet-Draft, draft-han-rtgwg-fine-grained-backpressure-02, 7 June 2026, . [I-D.ruan-spring-priority-flow-control-sid] Ruan, Z., Liu, Y., Han, M., Zhengxin, H., and Ying, "SRv6 behavior extention for Flow Control in WAN", Work in Progress, Internet-Draft, draft-ruan-spring-priority-flow- control-sid-03, 27 February 2026, . Authors' Addresses Zhengxin Han (editor) China Unicom Beijing China Email: hanzx21@chinaunicom.cn Ran Pang China Unicom Beijing China Email: pangran@chinaunicom.cn Yi Yue China Unicom Beijing China Email: yuey80@chinaunicom.cn Jie Dong Huawei Technologies Email: jie.dong@huawei.com Zheng Ruan China Unicom Email: ruanz6@chinaunicom.cn Han, et al. Expires 6 January 2027 [Page 11] Internet-Draft Req of PFC and fine-grained flow control July 2026 Quan Xiong ZTE Corporation Email: xiong.quan@zte.com.cn Han, et al. Expires 6 January 2027 [Page 12]