EVPN control plane for Geneve

The Network Virtualization over Layer 3 (NVO3) solutions for network virtualization in data center (DC) environment are based on an IP-based underlay. An NVO3 solution provides layer 2 and/or layer 3 overlay services for virtual networks enabling multi-tenancy and workload mobility. This document describes how the EVPN control plane defined in can signal Geneve encapsulation type in the BGP Tunnel Encapsulation Extended Community defined in . In addition, this document defines how to communicate the Geneve tunnel option types using BGP Tunnel Encapsulation Attribute sub-TLV. The Geneve tunnel options are encapsulated as TLVs after the Geneve base header in the Geneve packet as described in . recommends that a control plane determine how Network Virtualization Edges (NVEs) use the Geneve option TLVs when sending/receiving packets. In particular, the control plane negotiates the subset of option TLVs supported, their order and the total number of option TLVs allowed in the packets. This negotiation capability allows, for example, interoperability with hardware-based NVEs that can process fewer options than software-based NVEs. This EVPN control plane extension will allow an NVE to express what Geneve option TLV types it is capable of receiving, or sending over the Geneve tunnel with its peers. In the datapath, a transmitting NVE MUST NOT encapsulate a packet destined to another NVE with any option TLV(s) the receiving NVE is not capable of processing. Furthermore, the document defines a new ethernet option TLV to handle BUM traffic, etree root and leaf indication, and split horizon.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

NVO3: Network Virtualization Overlays over Layer 3 Geneve: Generic Network Virtualization Encapsulation. NVE: Network Virtualization Edge. VNI: Virtual Network Identifier. MAC: Media Access Control. OAM: Operations, Administration and Maintenance. PE: Provide Edge Node. CE: Customer Edge device e.g., host or router or switch. EVPN: Ethernet VPN. ES: Ethernet segment. ESI: Ethernet Segment Identifier. EVI: An EVPN instance spanning the Provider Edge (PE) devices participating in that EVPN. MAC-VRF: A Virtual Routing and Forwarding table for Media Access Control (MAC) addresses on a PE.

This document adds an extension to the encapsulation that is relevant to the operation of EVPN.

describes when an ingress NVE uses ingress replication to flood unknown unicast traffic to the egress NVEs, the ingress NVE needs to indicate to the egress NVE that the Encapsulated packet is a BUM packet. This is required to avoid transient packet duplication in all-active multi-homing scenarios. For Geneve, we need a bit for this purpose. uses an MPLS label for leaf indication of BUM traffic originated from a leaf attachment circuit (AC) in an ingress NVE so that the egress NVEs can filter BUM traffic toward their leaf ACs. For Geneve, we need a bit for this purpose. Although the default mechanism for split-horizon filtering of BUM traffic on an Ethernet segment for IP-based encapsulations such as VxLAN, GPE, NVGRE, and Geneve, is local-bias as defined in section 8.3.1 of , there can be an incentive to leverage the same split-horizon filtering mechanism of that uses a 20-bit MPLS label so that a) the a single filtering mechanism is used for all encapsulation types and b) the same PE can participate in a mix of MPLS and IP encapsulations. For this purpose a 20-bit label field MAY be defined for Geneve encapsulation. The support for this label is OPTIONAL. If an NVE wants to use local-bias procedure, then it sends the new option TLV with ESI-label=0

If an NVE wants to use ESI-label, then it sends the new option TLV with a non zero ESI-label

Where: - Option Class is set to Ethernet (new Option Class requested to IANA) - Type is set to EVPN-OPTION with value = 0, and C bit must be set. - B bit is set to 1 for BUM traffic. - L bit is set to 1 for Leaf-Indication. - H bit is set to 1 for Root-Indication. - Source-ID is a 24-bit value that encodes the ESI-label value signaled on the EVPN Autodiscovery per-ES routes, as described in for multi-homing and for leaf-to-leaf BUM filtering. The ESI-label value is encoded in the high-order 20 bits of the Source-ID field. The egress NVEs that make use of ESIs in the data path because they have a local multi-homed ES or support SHOULD advertise their Ethernet A-D per-ES routes along with the Geneve tunnel sub-TLV in addition to the ESI-label Extended Community. The ingress NVE can then use the Ethernet option-TLV when sending Geneve packets based on the and procedures. The egress NVE will use the Source-ID field in the received packets to make filtering decisions. Note that modifies the split-horizon procedures for NVO3 tunnels using the "local-bias" procedure. "Local-bias" relies on tunnel IP source address checks (instead of ESI-labels) to determine whether a packet can be forwarded to a local ES. While "local-bias" MUST be supported along with Geneve encapsulation, the use of the Ethernet option-TLV is RECOMMENDED to follow the same procedures used by EVPN MPLS. An ingress NVE using ingress replication to flood BUM traffic MUST send B=1 in all the Geneve packets that encapsulate BUM frames. An egress NVE SHOULD determine whether a received packet encapsulates a BUM frame based on the B bit. The use of the B bit is only relevant to Geneve packets with Protocol Type 0x6558 (Bridged Ethernet).

As per the BGP Encapsulation extended community defined in is included with all EVPN routes advertised by an egress NVE. This document uses the Geneve Encapsulation BGP Tunnel Encapsulation Type from the IANA BGP Tunnel Encapsulation Types registry, Value = 19.

The Geneve tunnel option types is a new BGP Tunnel Encapsulation Attribute Sub-TLV.

The Sub-TLV Type field contains a value in the range from 192-252. To be allocated by IANA. Sub-TLV value MUST match exactly the first 4-octets of the option TLV format. For instance, if we need to signal support for two option TLVs:

An NVE informs its peers which Geneve option TLVs it can receive by including the first 4 bytes of each option TLV in the Geneve Tunnel Option Types sub-TLV. The peers MUST send Geneve packets to this NVE with only the option TLVs that it has specified here, following the same order. The above sub-TLV(s) MAY be included with Ethernet A-D per-ES routes and MUST NOT be included with other routes.

The following figure shows an example of an NVO3 deployment with EVPN.

<---- DC2 ------>| Figure 5: Data Center Interconnect with ASBR ]]> iBGP sessions are established between NVE1, NVE2, ASBR1, possibly via a BGP route-reflector. Similarly, iBGP sessions are established between NVE3, NVE4, ASBR2. eBGP sessions are established among ASBR1 and ASBR2. All NVEs and ASBRs are enabled for the EVPN SAFI and exchange EVPN routes. For inter-AS option B, the ASBRs re-advertise these routes with NEXT_HOP attribute set to their IP addresses as per . NVE1 sets the BGP Encapsulation extended community defined in all EVPN routes advertised. NVE1 sets the BGP Tunnel Encapsulation Attribute Tunnel Type to Geneve tunnel encapsulation, and sets the Tunnel Encapsulation Attribute Tunnel sub-TLV for the Geneve tunnel option types with all the Geneve option types it can transmit and receive. All other NVE(s) learn what Geneve option types are supported by NVE1 through the EVPN control plane. In the datapath, NVE2, NVE3 and NVE4 MUST only encapsulate overlay packets with the Geneve option TLV(s) that NVE1 is capable of receiving, and in case more than one option TLV is being used, they MUST be in the order specified by NVE1. A PE advertises the BGP Encapsulation extended community defined in if it supports any of the encapsulations defined in . A PE advertises the BGP Tunnel Encapsulation Attribute defined in if it supports Geneve encapsulation, setting the type to Geneve Encapsulation.

The mechanisms in this document uses EVPN control plane as defined in . Security considerations described in are equally applicable. This document uses IP-based tunnel technologies to support data plane transport. Security considerations described in and in are equally applicable.

The authors wish to thank T. Sridhar, for his input, feedback, and helpful suggestions.