BGP PORT EC for AIDC

Introduction With the rapid development of Artificial Intelligence (AI) and Machine Learning (ML), AI tasks often generate large traffic due to the characteristics of large language model computation (LLM). If the link bandwidth is insufficient, packet loss may occur. AI computation has very high reliability requirements and extremely low tolerance for packet loss and latency. When there is link congestion in the network that leads to packet loss or excessive latency, it will have a significant impact on the computational efficiency of AI tasks. In data centers used for AI and machine learning, BGP is often used as the routing protocol . In some implementations, sufficient bandwidth between the destination server and its connected leaf switches must be ensured before sending traffic for AI tasks. On the network side, specifically the area comprised of the Leaf and Spine switches, there are numerous ECMP links. Techniques such as packet spraying can be used to minimize congestion and packet loss. However, on the computing side, specifically the last hop between the Leaf switches and the server, congestion can easily lead to packet loss, significantly reducing the efficiency of AI tasks. To minimize or eliminate packet loss on the last hop, BGP needs to be extended to include port information on the destination leaf switch. This allows the sender to negotiate based on this information before sending traffic, ensuring sufficient bandwidth is available in the last hop and preventing congestion and packet loss due to insufficient bandwidth. To reduce the stress caused by full-mesh connections, Leaf switches do not establish neighbors with each other. demonstrates two common deployment scenarios in AIDC. In both scenarios, it is necessary to advertise the corresponding ports along with the routes.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

Specification When an advertisement is sent to a server or a route learned from the core, the Leaf switch or PoD edge switch will carry the extended community attribute newly defined in this document. This newly added extended community attribute "Route Port ID extended community" and "Aggregate PORT Route Port ID extended community" will be abbreviated as PORT EC and AGG PORT EC in the following text.

PORT EC advertisement within PoD Scenario 1 in describes a scenario within a single Point of Destiny (PoD). Before sending traffic, each Leaf, in addition to announcing its own loopback route, also announces routes to connected GPUs/NICs, carrying the extended community attributes defined in this document. The Global Administrator in the PORT EC is set to the Leaf's own loopback address, and the Local Administrator in the PORT EC is set to the port number connected to the GPU/NIC. Routes are forwarded through the Spine device, allowing each Leaf to learn routes to other GPUs within its PoD, including connected Leaf devices and their ports. As illustrated in the example, when GPU1 wants to send traffic to GPU20, it needs to ensure that the interface between Leaf3 and NIC20 does not experience incasting, meaning the link between Leaf3 and NIC20 has sufficient bandwidth to receive the traffic. According to the authorization mechanism mentioned in , before GPU1 sends traffic, it needs to request authorization from GPU20. This authorization includes a bandwidth request. If GPU20 determines that the link bandwidth with Leaf3 is sufficient, it will send a successful authorization message back to GPU1. GPU1 then begins sending traffic. Upon receiving the route carrying the PORT EC, the leaf switch checks if the address carried in the PORT EC is reachable. If unreachable, the extended community is ignored. If reachable, the address and port information are stored locally or sent to the server. This storing or sending process is outside the scope of this draft. When the GPU or NIC does not support the authorization mechanism, this task can also be performed by the Leaf device. Leaf1 initiates a authorization request to Leaf3, specifying the required bandwidth and the link between Leaf3 and GPU20/NIC20. If Leaf3 determines that the link bandwidth with GPU20/NIC20 is sufficient, it will send a successful authorization message back to Leaf1, and only then will Leaf1 begin forwarding traffic from GPU1. This scenario also requires negotiation of traffic transmission request information between GPU1/NIC1 and Leaf1, which is out the scope of this document. After traffic enters Leaf1, it undergoes GSE header encapsulation which described in as it travels through the Spine device to Leaf3. During this process, the header is used for packet spraying on ECMP routing. Upon reaching Leaf3, Leaf3 then forwards the packet to NIC20. When the NIC supports both authorization mechanisms and GSE header encapsulation, the Leaf device only needs to forward packets based on the GSE header. When the NIC does not support authorization mechanisms or GSE encapsulation, the Leaf device performs the GSE header encapsulation, and upon reaching Leaf3, Leaf3 decapsulates the GSE header and rearranges any out-of-order packets before sending them to NIC20.

AGG PORT EC advertisement Scenario 2 in describes a traffic interconnection scenario between multiple PoDs, where traffic needs to be sent to the GPUs of other PoDs. The Core device in the diagram is used to interconnect the SSpine devices of different PoDs; the SSpine device is a device within a PoD used to connect to the Core device for cross-PoD interconnection. In some deployment scenarios, the SSpine layer device may be omitted, and the Spine device directly interconnects with the Core device. As seen in Scenario 2 in , there may be multiple ECMP paths from Spine to SSpine. When the same advertised route may have multiple different PORT EC attributes, only one route will be selected as the optimal route according to the EBGP forwarding rules within the data center, and then forwarded to other nodes. This does not make good use of resources. For example, in Scenario 2, Spine1 selects the optimal route received from SSpine switches to other PoDs. Though not only SSpine1 can be used as the next hop, but other SSpine switch can also be used as ECMP next hops. However, after Spine switch select the best route with the same prefix, they may only advertise one corresponding PORT EC attribute to Leaf swtiches. In this case, Leaf swithes can only negotiate with the link between Spine1 and SSpine1, and cannot use the ECMP link between Spine1 and other SSpine switches, which will result in a waste of resources. To avoid attribute loss after BGP route selection, one approach is to enable the ADD-PATH function defined in in the PoD. However, this function may cause a route advertisement storm, severely impacting the efficiency of route advertisement and affecting normal forwarding traffic. Another approach is to use the AGG PORT EC attributes defined in this document. This helps establish more comprehensive ECMP entries. Suppose GPU1 needs to send traffic to GPU5000 in another PoD, such as PoD2. Within PoD1, where GPU1 resides, a similar approach to Scenario 1 can be used to ensure that traffic is not out of order or lost before being sent to the Core device. According to BGP route advertisements, routes from other PoDs, such as the route for GPU5000, will be advertised to the SSpine device through multiple Core devices, such as Core1 through Core8. The SSpine device will then advertise these routes to the Spine device, which in turn advertises them to the Leaf device. During this advertisement process, the SSpine device, acting as the interface between its PoD and the Core layer devices, will include its own address and ports when advertising routes obtained from other PoDs. In this example, SSpine1 receives routes from GPU5000 from Core1 through Core8. When advertising these routes to the Spine device, it will include the AGG PORT EC attributes defined in this document. Because there are multiple ports connected to the Core, they need to be carried in an aggregated list format. When Spine advertises routes received from SSpine to Leaf devices, it needs to further aggregate different AGG PORT ECs for the same route. For example, Spine1 might receive routes from SSpine1 to SSpine8 advertising GPU5000. Spine1 will then use AGG PORT ECs to further aggregate these advertising devices and ports before advertising them to Leaf devices. In this way, the GPU5000 routes learned by Leaf devices will include multiple devices from SSpine to SSpine8, and each SSpine will also have a set of ports. To avoid incasting between the SSpine device and the Core device, a similar approach to Scenario 1 is adopted: before Leaf1 forwards traffic, it negotiates authorization with the SSpine device. Leaf1 has multiple selectable SSpine devices, each with multiple interfaces connected to the Core. Leaf1 can send authorization requests based on the received SSpine device and port information. For example, in this case, Leaf1 can send an authorization request to SSpine1, specifying the interfaces between SSpine1 and Core1. If the authorization request fails, it can then send an authorization request to the interfaces between SSpine1 and Core2. When all interfaces on SSpine1 connected to the Core cannot meet the requirements, it sends an authorization request to SSpine2, specifying the interfaces between SSpine2 and Core1, and so on, until an SSpine device and port capable of handling the bandwidth are found. Of course, in the implementation, the method of finding the target SSpine device and its port can be optimized, not necessarily starting from the first port of the first device each time, to improve efficiency. During the entire forwarding process from Leaf to SSpine, the traffic also needs to be encapsulated with the GSE header described in . The fields in the header will guide the traffic packets to be sprayed onto the ECMP path. When the packet reaches the selected SSpine device, the SSpine device will preserve the order of the packet and then send it to the Core device to ensure reliable forwarding of the traffic within this PoD.