]> HPE

1701 East Mossy Oaks Road Spring TX 77389 United States of America jordan.head@hpe.com

HPE

1701 East Mossy Oaks Road Spring TX 77389 United States of America antoni.przygienda@hpe.com

RTG rift rift kv The Routing in Fat Trees (RIFT) protocol allows for key/value pairs to be advertised within Key-Value Topology Information Elements (KV TIEs). The data contained within these KV TIEs can be used for any imaginable purpose. This document specifies behavior for the various Key Types (i.e., Well-Known, Organizationally Unique Identifier (OUI), and Experimental) and a method to structure corresponding values. It also defines a Well-Known Key Sub-Type used for testing tie-breaking behavior.

Introduction The Routing in Fat Trees (RIFT) protocol allows for key/value pairs to be advertised within Key-Value Topology Information Elements (KV TIEs). There are no restrictions placed on the data that is contained in KV TIEs nor what the data is used for. For example, it might be beneficial to advertise overlay protocol state from leaf nodes to the Top-of-Fabric (ToF) nodes. This would make it possible to view the critical state of a fabric-wide service from a single ToF node rather than retrieving and reconciling the same state from multiple leaf nodes.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Key-Value Structure This section describes the generic key structure and semantics; further illustrates these components. specifies the use of Thrift to define the protocol's packet structure. While no explicit restrictions are placed on Key-Value data or what it is used for, it is RECOMMENDED that a serialized Thrift model also be used to define a KV TIE structure for simpler interoperability. For example, describes this type of implementation.

Generic Key-Value Structure

where:

Key Type:

A 1-byte value that identifies the Key Type. Key Type values are taken from the "RIFTCommonKVTypes" registry defined in . The range of valid values is 1 - 255 (2⁸-1). 0 is an illegal value and MUST NOT be allocated to or used by any implementation. KV TIEs received with this value MUST be discarded and logged at least once.

Key Identifier:

A 3-byte value that identifies the specific key and describes the semantics of any contained values. It SHOULD be unique within the context of the given Key Type. The range of valid values is 1 - 16777215 (2²⁴-1). 0 is an illegal value and MUST NOT be allocated to or used by any implementation. KV TIEs received with this value MUST be discarded and logged at least once.

Values:

A variable length value that contains data associated with the Key Identifier. It SHOULD contain 1 or more elements. The semantics (i.e., existence, order, duplication, etc.) of any contained values is governed by the particular key's specification.

Key Sub-Type The Key Sub-Type is a mechanism to further describe the key's semantics. This is illustrated by . The Key Sub-Type MUST be used when the Key Type is either Well-Known or Experimental in order to avoid interoperability issues but is OPTIONAL for other Key Types.

Generic Key-Value Structure with Key Sub-Type

where:

Key Sub-Type:

A 1-byte value that identifies the Key Sub-Type that describes the key and its semantics. The range of valid values is 1 - 255 (2⁸-1). 0 is an illegal value and MUST NOT be allocated to or used by any implementation. KV TIEs received with this value MUST be discarded and logged at least once.

Key Sub-Identifier:

A 2-byte value that identifies the specific key and describes the semantics of any contained values. It SHOULD be unique within the context of the given Key Sub-Type. The range of valid values is 1 - 65535 (2¹⁶-1). 0 is an illegal value and MUST NOT be allocated to or used by any implementation. KV TIEs received with this value MUST be discarded and logged at least once.

Experimental Key Type This section describes the Experimental Key Type. As shown in , the Key Type is set to 1, which identifies the Key Type as Experimental. The Experimental Key Type MUST support the use of a Key Sub-Type. The Key Sub-Identifier will be used to identify the specific key and the semantics of any contained values.

Experimental Key Type

Well-Known Key Type This section describes the Well-Known Key Type. As shown in , the Key Type is set to 2, which identifies the Key Type as Well-Known. The Well-Known Key Type MUST support the use of a Key Sub-Type. The Key Sub-Identifier will be used to identify the specific key and the semantics of any contained values.

Well-Known Key Type

OUI Key Type This section describes the OUI (vendor-specific) Key Type that an implementation MAY support. As shown in , the Key Type is set to 3, which identifies the Key Type as OUI. The Key Identifier MUST use the implementing organization's reserved OUI space to indicate the key and the semantics of any contained values.

OUI Key Type

Design Considerations While no explicit restrictions are placed on how Key-Value elements are used, it is of critical importance to consider these details. For example, they should not be used to carry topology information used by RIFT itself to perform distributed computations as it would likely lead to race conditions in convergence, oscillations, and/or other suboptimal behaviors. It is possible that deployments may have nodes that support a given KV TIE and others that do not. In this scenario, nodes that receive KV TIEs that they don't recognize (e.g., an unknown Key Type) will flood them normally as specified in . New Key Types offer 3 bytes of key identification space, and new Well-Known Key Sub-Types offer 2 bytes. When defining how key identification space is used, it is important to consider how much space is actually necessary in order to help ensure efficient use of available registry values.

Tie-Breaking Considerations In cases where KV TIEs are flooded southbound, mechanisms SHOULD be implemented in order to avoid network-wide flooding where possible. Key Targets (defined in ) are one such mechanism. specifies that only one KV TIE is selected when identical keys are received from multiple northbound neighbors. Therefore, it is RECOMMENDED that implementations ensure that nodes determine Values within KV TIEs independently in a consistent fashion in order to prevent scenarios where multiple ToFs advertise KV TIEs with identical keys but differing Values. In such scenarios, node(s) will select the KV TIE with the highest System ID, which may lead to unintended effects. Even with a robust implementation, operators should also consider that this may still happen under failure conditions, for example, multiple ToFs becoming split-brained.

Southbound KV TIE Tie-Break Sub-Type This section reserves a Key Sub-Type from the "RIFT Well-Known Key Sub-Types" registry. This Key-Value pair contains information that allows implementations to test and verify proper tie-breaking behavior for the Southbound Keystore. All implementations MUST support this Sub-Type. All implementations SHOULD use the Thrift model defined in .

Southbound Tie-Break Sub-Type

where:

System ID:

A REQUIRED value indicating the node's unique System ID.

Level:

A RECOMMENDED value indicating the node's level.

Thrift Models This section contains the normative Thrift model to support testing southbound Key-Value tie-breaking based on System ID. Per , all signed values MUST be interpreted as unsigned values.

RIFT Common Schema for Southbound Key-Value Tie-Break Key Sub-Type

Key Target The Key Target is an OPTIONAL 64-bit value that identifies group(s) of node(s) that are intended to receive a given KV TIE. Key Targets have a valid range of 0 - 18446744073709551615 (2⁶⁴-1). The Thrift model defined in SHOULD be used for Key Target implementation. illustrates the format.

Key Target Format

A value of all 0s indicates that every node is intended to receive this KV TIE and MUST NOT be used for any other reason. A value of all 1s indicates that all leaf nodes are intended to receive this KV TIE and MUST NOT be used for any other reason. Any other value MUST be derived from the following normative algorithm. Note that while the algorithm is shown using example code written in , this document does not mandate the use of any particular language for implementation.

Key Target Standard Algorithm KeyValueTargetType { (0 as usize .. 3) .map(|s| { let rot = (target ^ RANDOMSEEDS[s]).rotate_left(s as _); rot.to_ne_bytes().iter().fold(0, |v: u8, nv| v.rotate_right(4) ^ *nv) % 64 }) .fold(0, |v, nv| v | (1 << nv)) } ]]>

Key Target Processing Nodes that support the processing of Key Targets MUST only do so on KV TIEs in the southbound direction. Key Targets MUST NOT be present on KV TIEs in the northbound direction and are ignored and logged at least once. Nodes that do not support the processing of Key Targets MUST continue to send KV TIEs to all nodes in the appropriate direction. Additionally, Key Targets MUST be preserved when KV TIEs are re-originated in the southbound direction.

Purging/Rollover There are several reasons a node may select a different KV TIE. For example, the KV TIE is considered newer due to the sequence number incrementing, a change in the original tie-breaking result between multiple KV TIEs, or a loss of northbound connectivity to the node that advertised the previously selected KV TIE. Consider a case where Leaf-1, Leaf-2, and Leaf-3 are members of a group of nodes represented by Key Target KT1. If Leaf-2 is removed from that group and a newer instance of the KV TIE needs to be flooded, Leaf-2 will have to maintain the older KV TIE in the Link State Database (LSDB) until the lifetime expires. This could lead to suboptimal behavior in the fabric. If the new KV TIE being flooded does not include the previous Key Target value, then implementations SHOULD flood the newer instance of the KV TIE with a very short lifetime to nodes that belonged to the previous Key Target but not the new Key Target.

IANA Considerations Per , IANA has created the "RIFT Well-Known Key Sub-Types" registry in the "Routing in Fat Trees (RIFT)" registry group at . IANA has updated the "RIFTCommonKVTypes" registry based on values defined in of this document, and this document has been added as a reference. Experts reviewing requests for new values to either the "RIFTCommonKVTypes" registry or the "RIFT Well-Known Key Sub-Types" registry MUST consider the items in "Expert Review Guidance" .

RIFT Well-Known Key Sub-Types IANA has created the following registry:

Registry Name:

RIFT Well-Known Key Sub-Types

Registration Procedures:

Expert Review

Description:

Well-Known Key Sub-Types registry for the RIFT protocol.

Reference:

RFC 9992

RIFT Well-Known Key Sub-Types Entries IANA has registered the following values in the "RIFT Well-Known Key Sub-Types" registry. RIFT Well-Known Key Sub-Types Entries

Value	Name	Description	Reference
0	Illegal	Not allowed.	RFC 9992
1-126	Unassigned
127	Southbound Tie-Break Sub-Type	Used for testing/verifying Southbound Keystore tie-breaking behavior.	RFC 9992
128-255	Unassigned

Expert Review Guidance Experts reviewing requests for values from the "RIFTCommonKVTypes" registry or the "RIFT Well-Known Key Sub-Types" registry are responsible for the following:

1. Ensuring that the supporting documentation accompanying the request properly defines how Key Identifiers and/or Key Sub-Identifiers are used (e.g., as a boolean, an explicit value, an auto-derived value, etc.).
2. Ensuring that the supporting documentation provides normative Thrift model(s) (if applicable).
3. Ensuring that any work originating outside the IETF does not conflict with any work that is already published or in active pursuit of being published.

Security Considerations This document introduces no new security concerns to RIFT or other specifications referenced in this document given that the KV TIEs are already extensively secured by the RIFT protocol specification itself.

References Normative References Apache Software Foundation commit 66d8976 IEEE Rust Foundation Informative References

Acknowledgements Thanks to for his very thoughtful review that yielded an improved spec.