Swisscom

Binzring 17 Zurich 8045 Switzerland thomas.graf@swisscom.com

Huawei

benoit@everything-ops.net

INSA-Lyon

Lyon France alex.huang-feng@insa-lyon.fr

OPS opsawg Flow Record Performance Metric On-Path Hybrid Type I OAM This document specifies new IP Flow Information Export (IPFIX) Information Elements to export the On-Path delay at each Operations, Administration, and Maintenance (OAM) transit and decapsulating nodes. The On-Path delay is defined as the delay between the OAM header encapsulating node and each OAM header transit and OAM header decapsulating nodes. This delay measurement is computed by an On-Path Telemetry protocol and is exported by the IPFIX process.

Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .

Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents

• .  Introduction
• .  Terminology
• .  Solution
• .  Performance Metrics
  • .  IP One-Way Delay Hybrid Type I Performance Metrics
    • .  Summary
    • .  Description
    • .  Reference
    • .  Change Controller
    • .  Version of Registry Format
  • .  Metric Definition
    • .  Reference Definition
    • .  Fixed Parameters
  • .  Method of Measurement
    • .  Reference Methods
    • .  Packet Stream Generation
    • .  Traffic Filtering (Observation) Details
    • .  Sampling Distribution
    • .  Runtime Parameters and Data Format
    • .  Roles
  • .  Output
    • .  Type
    • .  Reference Definition
    • .  Administrative Items
    • .  Comments and Remarks
• .  Use Cases
• .  IANA Considerations
  • .  Performance Metrics
  • .  IPFIX Entities
    • .  pathDelayMeanDeltaMicroseconds
    • .  pathDelayMinDeltaMicroseconds
    • .  pathDelayMaxDeltaMicroseconds
    • .  pathDelaySumDeltaMicroseconds
• .  Operational Considerations
  • .  Time Accuracy
  • .  Mean Delay
  • .  Reduced-Size Encoding
  • .  Measurement Interval
  • .  In-Packet OAM Application
• .  Security Considerations
• .  References
  • .  Normative References
  • .  Informative References
• .  IPFIX Encoding Examples
  • .  Aggregated On-Path Delay Examples
    • .  Template Record and Data Set with Mean Delta
    • .  Template Record and Data Set with Sum Delta
• Acknowledgements
• Authors' Addresses

Introduction Network operators usually maintain statistical views of delay across their networks to support diagnostics and performance analysis. These views assist in identifying the location, extent, and potential causes of abnormal delay affecting specific customer traffic or services. To achieve this, delay-related metrics need to be reported from devices covering both data and control planes. Further, in order to understand which customers are affected, delay-related metrics need to be reported in the context of the customer data plane. This correlation enables the detection of changes in forwarding paths, such as updated intermediate hops or interfaces, and of the resulting impact on delay experienced by customer traffic. Delay measurements in the network are computed using an On-Path Telemetry protocol, which inserts metadata into the data-plane packet when entering the monitored domain . To compute delay measurements, the On-Path Telemetry protocol inserts a timestamp reference when entering the OAM encapsulating node. Implementation examples are In situ Operations, Administration, and Maintenance (IOAM) or Enhanced Alternate Marking . Two modes of On-Path Telemetry are generally recognized: passport mode, in which only the OAM header decapsulating node of the OAM domain reports metrics; and postcard mode, in which OAM header transit nodes also export On-Path Telemetry data. Both modes enable exposure of per-hop performance metrics, including delay accumulation. The approach defined in this document is primarily applicable to postcard mode. To enable the export of the delay-related metrics via IPFIX , this document defines four new IPFIX Information Elements (IEs), exposing the On-Path delay on OAM header transit and decapsulating nodes, following the principles of postcard mode. This enables the computation of delay metrics (minimum, maximum, and mean) directly on the OAM header transit and decapsulating node, allowing aggregation within the Flow Record. As these IEs represent performance metrics, they are also registered in the IANA "Performance Metrics Registry" in accordance with .

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document defines the following terms:

OAM Header Encapsulating Node:

Receives the IP packets, encapsulates the packets with an OAM header, and adds the timestamp into the OAM header.

OAM Header Transit Node:

Receives the IP packets, then measures the delay between the timestamp in the packet and the timestamp when the packet was received.

OAM Header Decapsulating Node:

Receives the IP packets, computes the delay between the timestamp in the packet and the timestamp when the packet was received, and removes the OAM header from the packet.

This document makes use of the terms defined in , and . The following terms are used as defined in :

• Collector
• Exporter
• Flow
• Flow Record
• IPFIX
• IPFIX Information Elements (IEs)
• Observation Point

The following terms are used as defined in :

• Performance Metric
• Performance Metrics Registry
• Registered Performance Metric

The following term is used as defined in :

• Hybrid Type I

Solution In line with the guidelines for Registered Performance Metric requesters and reviewers , each metric is specified with its required characteristics (e.g., Identifier, Name, URI, Status, Requester, Revision, Description) to ensure comparability of measurement results across implementations and environments. These characteristics are registered in the IANA "Performance Metrics Registry". Metric naming follows the "MetricType_Method_SubTypeMethod_... Spec_Units_Output" convention defined in . This document defines the following performance metrics and IPFIX Information Elements: Mapping Between IPFIX IEs and Performance Metrics

Performance Metric	IPFIX Information Element
OWDelay_HybridType1_I P_RFC9951_Seconds_Mean (27)	pathDelayMeanDeltaMicroseconds (530)
OWDelay_HybridType1_I P_RFC9951_Seconds_Min (28)	pathDelayMinDeltaMicroseconds (531)
OWDelay_HybridType1_I P_RFC9951_Seconds_Max (29)	pathDelayMaxDeltaMicroseconds (532)
OWDelay_HybridType1_I P_RFC9951_Seconds_Sum (30)	pathDelaySumDeltaMicroseconds (533)

Assuming time synchronization on devices, the delay is measured by calculating the difference between the timestamp imposed with On-Path Telemetry in the packet at an OAM header encapsulating node and the timestamp exported in the IPFIX Flow Record from the OAM header transit and OAM header decapsulating nodes. The lowest, highest, mean, and the sum of measured path delay can be exported, thanks to the different IPFIX IE specifications.

Delay Use Case: Packets Flow from Host 1 to Host 2 On-Path Telemetry Domain ......................................... . . . D1 . . x-------> . . . . D2 . . x--------------------> . . . . D3 . . x----------------------------------> . . . (H1) -----> (R0) ------> (R1) ------> (R2) -------> (R3) -----> (H2) Host 1 Encapsulating Transit Transit Decapsulating Host 2 Node Node 1 Node 2 Node . . . . .........................................

In the use case shown in , using On-Path Telemetry to export the delay metrics, the node R1 exports the delay D1, the node R2 exports the delay D2, and the OAM header decapsulating node R3 exports the total delay D3 for the same flow using IPFIX. This solution enables the computation of delay metrics (minimum, maximum, and mean) directly on the OAM header transit and decapsulating node, allowing aggregation within the Flow Record. This reduces both export bandwidth and processing requirements on the Collector. To compute these metrics locally, the Exporter's Metering Process must perform per-packet caching and processing, particularly when computing mean delay under Flow Aggregation . A less computationally intensive alternative is to export the sum of delays, allowing the Collector to compute the mean via a simple division using the packet count. In contrast, if no delay processing occurs on the OAM header transit or decapsulating node, each packet must be exported as an individual Flow Record, including timestamp information, as specified in . The Collector must then compute the delay metrics and reconstruct the aggregated Flow Record accordingly.

Performance Metrics This section defines four new performance metrics following the template defined in .

IP One-Way Delay Hybrid Type I Performance Metrics This section specifies four performance metrics for the Hybrid Type I assessment of IP One-Way Delay; they have been registered in the IANA "Performance Metrics Registry". All column entries besides the Identifier, Name, URI, Description, Reference Description (Output only) categories are the same; thus, this section defines four closely related performance metrics. As a result, IANA has assigned corresponding URIs to each of the four registered performance metrics.

Summary This category includes multiple indexes of the registered performance metrics: the Identifier and Metric Name.

ID (Identifier) IANA has allocated the numeric Identifiers 27, 28, 29, and 30 for the four Named Metric Entries in the following section.

Name

27:

OWDelay_HybridType1_IP_RFC9951_Seconds_Mean

28:

OWDelay_HybridType1_IP_RFC9951_Seconds_Min

29:

OWDelay_HybridType1_IP_RFC9951_Seconds_Max

30:

OWDelay_HybridType1_IP_RFC9951_Seconds_Sum

URI

URI:

URI:

URI:

URI:

Description

OWDelay_HybridType1_IP_RFC9951_Seconds_Mean:

This metric assesses the mean of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay is based on a single Observation Point somewhere in the network.

OWDelay_HybridType1_IP_RFC9951_Seconds_Min:

This metric assesses the minimum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay is based on a single Observation Point somewhere in the network.

OWDelay_HybridType1_IP_RFC9951_Seconds_Max:

This metric assesses the maximum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay is based on a single Observation Point somewhere in the network.

OWDelay_HybridType1_IP_RFC9951_Seconds_Sum:

This metric assesses the sum of one-way delays of all successfully forwarded IP packets constituting a single Flow. The measurement of one-way delay is based on a single Observation Point somewhere in the network.

Reference RFC 9951

Change Controller IETF

Version of Registry Format 1.0

Metric Definition This category includes columns to prompt the entry of all necessary details related to the metric definition, including the immutable document reference and values of input factors, called "Fixed Parameters".

Reference Definition See and in the Normative References (). provides the reference definition of the singleton (single value) one-way delay metric. provides the reference definition expanded to cover a multi-value sample. Note that terms such as "singleton" and "sample" are defined in . With the Observation Point typically located between the hosts participating in the IP Flow, the one-way delay metric requires one individual measurement between the Observation Point and sourcing host, such that the Spatial Composition of the measurements yields a one-way delay singleton. This document specifies how to export the performance metric using IPFIX.

Fixed Parameters None

Method of Measurement This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous method for implementations.

Reference Methods The foundational methodology for this metric is defined in using the Timestamps option with modifications that allow application at a mid-path Observation Point .

Packet Stream Generation This is the time when the packet is being received at the OAM header encapsulating node. The timestamp format depends on the On-Path Telemetry implementation. For IOAM, describes the supported timestamps. Sections and of describe where the timestamp is being inserted. For the Enhanced Alternate Marking Method, and define the supported timestamp encodings and granularity.

Traffic Filtering (Observation) Details Runtime Parameters (in the following sections) may be used for Traffic Filtering.

Sampling Distribution This metric requires a partial sample of all packets that qualify according to the Traffic Filter criteria.

Runtime Parameters and Data Format Runtime Parameters are input factors that must be determined, configured into a measurement system, and reported with the results for the context to be complete. The Hybrid Type I metering parameters must be reported to provide the complete measurement context. As an example, if the IPFIX Metering Process is used, then the IPFIX Metering Process parameters (IPFIX Template Record, potential traffic filters, and potential sampling method and parameters) that generate the Flow Records must be reported to provide the complete measurement context. At a minimum, the following fields are required:

Src:

The IP address of the host in the host A Role (format ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value for IPv6; see ).

Dst:

The IP address of the host in the host B Role (format ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value for IPv6; see ).

T0:

T time, the start of a measurement interval (format "date/time" as specified in ; see also "date-and-time" in ). The UTC Time Zone is required by . When T0 is "all-zeros", a start time is unspecified, and Tf is to be interpreted as the duration of the measurement interval. The start time is controlled through other means.

Tf:

A time, the end of a measurement interval (format "date/time" as specified in ; see also "date-and-time" in ). The UTC Time Zone is required by . When T0 is "all-zeros", an ending time and date are ignored, and Tf is interpreted as the duration of the measurement interval.

Roles

Host A:

Launches an IP packet to start the Flow.

Host B:

Receives the IP packet to start the Flow.

OAM Header Encapsulating Node:

Receives the IP packets, encapsulates the packets with the OAM header, and adds the timestamp into the OAM header.

OAM Header Transit Node:

Receives the IP packets, measures the delay between the timestamp in the packet and the timestamp when the packet was received.

OAM Header Decapsulating Node:

Receives the IP packets, computes the delay between the timestamp in the packet and the timestamp when the packet was received, and removes the OAM header from the packet.

Output This category specifies all details of the output of measurements using the metric.

Type OWDelay Types are discussed in the subsections below.

Reference Definition For all output types:

OWDelay_HybridType1_IP:

The one-way delay of one IP packet is a singleton.

For each <statistic> singleton, one of the following subsections applies.

OWDelay_HybridType1_IP_RFC9951_Seconds_Mean Similar to , the mean SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analytic choice. See for details on calculating this statistic; see also .

Mean:

The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC9951_Seconds_Min Similar to , the minimum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analytic choice. See for details on calculating this statistic; see also .

Min:

The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC9951_Seconds_Max Similar to , the maximum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analytic choice. See for a closely related method for calculating this statistic; see also . The formula is as follows: Max = (FiniteDelay[j]) such that for some index, j, where 1 <= j <= N FiniteDelay[j] >= FiniteDelay[n] for all n where all packets n = 1 through N have finite singleton delays.

Max:

The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

OWDelay_HybridType1_IP_RFC9951_Seconds_Sum The sum SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay (undefined delays are excluded) -- a single value, as follows: See for details on the conditional distribution to exclude undefined values of delay, and see for background on this analytic choice. See for details on calculating this statistic; however, in this case, FiniteDelay or MaxDelay MAY be used.

Sum:

The time value of the result is expressed in units of microseconds, as a positive value of type decimal64 with fraction digits = 9 (similar to decimal64 in YANG, ) with a resolution of 0.001 microseconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per .

Metric Units

• Mean
• Min
• Max
• Sum

The one-way delay of the IP Flow singleton is expressed in microseconds.

Calibration A clock synchronization between the nodes of the monitored OAM domain is needed to compute representative delay measurements at the OAM header transit and decapsulating nodes. NTP, as defined in , can be used for synchronizing the clocks of the monitored nodes.

Administrative Items

Status Current

Requester RFC 9951

Revision 1.0

Revision Date 2026-04-02

Comments and Remarks None

Use Cases The measured On-Path delay can be aggregated with Flow Aggregation as defined in across the following device and control-plane dimensions to determine:

• With node id and egressInterface(14), on which node which logical egress interfaces have been contributing to how much delay.
• With node id and egressPhysicalInterface(253), on which node which physical egress interfaces have been contributing to how much delay.
• With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the forwarding path to which next-hop IP contributed to how much delay.
• With mplsTopLabelIPv4Address(47) or destinationIPv6Address and srhActiveSegmentIPv6(495), the forwarding path to which MPLS top-label IPv4 address or IPv6 destination address and Segment Routing over IPv6 (SRv6) active segment contributed to how much delay.
• BGP communities are often used for setting a path priority or service selection. With bgpDestinationExtendedCommunityList(488) or bgpDestinationCommunityList(485) or bgpDestinationLargeCommunityList(491), which group of prefixes accumulated at which node how much delay.
• With destinationIPv4Address(13), destinationTransportPort(11), protocolIdentifier(4), and sourceIPv4Address(8), or equivalent IPFIX IEs for IPv6, the forwarding path delay on each node from each IPv4 source address to a specific application in the network.

Let us consider as a topology example. shows the aggregated delay per each node, ingressInterface(10), egressInterface(14), destinationIPv6Address(28), and srhActiveSegmentIPv6(495) measured at ingress. Example Table of Measured Delay at Ingress, Ascending by Delay

ingress Interface	egress Interface	Node	destination IPv6Address	srhActive SegmentIPv6	Path Delay
271	276	R0			0 µs
301	312	R1	2001:db8::1	2001:db8::3	22 µs
22	27	R2	2001:db8::2	2001:db8::3	42 µs
852	854	R3	2001:db8::3	2001:db8::3	122 µs

IANA Considerations

Performance Metrics IANA has add four new performance metrics in the "Performance Metrics Registry" with the four templates defined in .

IPFIX Entities IANA has registered new IPFIX IEs (see ) in the "IPFIX Information Elements" registry in the "IP Flow Information Export (IPFIX) Entities" registry group and assigned the following code points. New IPFIX IEs in the "IPFIX Information Elements" Registry

ElementID	Name
530	pathDelayMeanDeltaMicroseconds
531	pathDelayMinDeltaMicroseconds
532	pathDelayMaxDeltaMicroseconds
533	pathDelaySumDeltaMicroseconds

pathDelayMeanDeltaMicroseconds

Name:

pathDelayMeanDeltaMicroseconds

ElementID:

530

Description:

This Information Element identifies the mean path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC9951_Seconds_Mean in the IANA "Performance Metrics Registry".

Abstract Data Type:

unsigned32

Data Type Semantics:

deltaCounter

Reference:

RFC 9951

Additional Information:

OWDelay_HybridType1_IP_RFC9951_Seconds_Mean in the IANA "Performance Metrics Registry".

pathDelayMinDeltaMicroseconds

Name:

pathDelayMinDeltaMicroseconds

ElementID:

531

Description:

This Information Element identifies the lowest path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to the OWDelay_HybridType1_IP_RFC9951_Seconds_Min in the IANA "Performance Metrics Registry".

Abstract Data Type:

unsigned32

Data Type Semantics:

deltaCounter

Reference:

RFC 9951

Additional Information:

OWDelay_HybridType1_IP_RFC9951_Seconds_Min in the IANA "Performance Metrics Registry".

pathDelayMaxDeltaMicroseconds

Name:

pathDelayMaxDeltaMicroseconds

ElementID:

532

Description:

This Information Element identifies the highest path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC9951_Seconds_Max in the IANA "Performance Metrics Registry".

Abstract Data Type:

unsigned32

Data Type Semantics:

deltaCounter

Reference:

RFC 9951

Additional Information:

OWDelay_HybridType1_IP_RFC9951_Seconds_Max in the IANA "Performance Metrics Registry".

pathDelaySumDeltaMicroseconds

Name:

pathDelaySumDeltaMicroseconds

ElementID:

533

Description:

This Information Element identifies the sum of the path delay of all packets in the Flow, in microseconds, between an OAM header encapsulating node and the local node with the OAM domain (either an OAM header transit node or an OAM header decapsulating node), according to OWDelay_HybridType1_IP_RFC9951_Seconds_Sum in the IANA "Performance Metrics Registry".

Abstract Data Type:

unsigned64

Data Type Semantics:

deltaCounter

Reference:

RFC 9951

Additional Information:

OWDelay_HybridType1_IP_RFC9951_Seconds_Sum in the IANA "Performance Metrics Registry".

Operational Considerations

Time Accuracy In terms of clock precision, the same recommendation as defined in for IPFIX applies to this document as well.

Mean Delay The mean (average) path delay can be calculated by dividing the pathDelaySumDeltaMicroseconds(533) by the packetDeltaCount(2) at the IPFIX data collection at the collection time instead of the IPFIX Exporter at the export time.

Reduced-Size Encoding Unsigned64 has been chosen as the type for pathDelaySumDeltaMicroseconds to support cases with large delay numbers and where many packets are being counted. As an example, a specific Flow Record with path delay of 100 milliseconds cannot observe more than 42949 packets without overflowing the unsigned32 counter. The procedure described in may be applied to reduce network bandwidth between the IPFIX Exporter and Collector if unsigned32 would be large enough without wrapping around.

Measurement Interval The delay metrics are computed for the Flow Record lifetime by comparing the OAM timestamps in each received packet with the timestamp when they were received. For a long-running Flow, the IPFIX Metering Process might miss the temporal distribution of the delay (for example, a longer delay only at the beginning of the Flow). If this is an operational problem, the IPFIX Metering Process might be configured with a smaller expiration timeout (see "Flow Expiration", ).

In-Packet OAM Application Multiple methods can be used to compute the delay performance metrics defined in this document. Some examples of such methods are IOAM and Enhanced Alternate Marking . For IOAM, these performance metrics can be computed using the Edge-to-Edge and the Direct Exporting Option-Type. IOAM Edge-to-Edge Option-Type, as described in , can use bits 2 and 3. In this case, timestamps are encoded as defined in Sections and of . This timestamp can be used to compute the delay between an OAM header encapsulating node and the decapsulating node. The IOAM Direct Exporting Option-Type, as described in , can use the Extension-Flag defined in to insert a timestamp in the OAM header encapsulating node. The timestamp is encoded as defined in Sections and of . This timestamp can be used to compute the delay between the inserted timestamp and the OAM header transit and decapsulating node. For the Enhanced Alternate Marking Method, and define that, within the metaInfo, a nanosecond timestamp can be encoded in an OAM header encapsulating node and be read at the OAM header intermediate and decapsulating nodes to calculate the On-path delay. defines how this can be applied to the IPv6 extensions header, and defines how this can be applied to the SRv6 Segment Routing Header . Given that the delay measurements are computed with the timestamp introduced on the OAM header encapsulating node, regardless of the approach, implementations should document at which point of the forwarding plane this timestamp is introduced (e.g., the time at which the packet was received by the node, the time at which the packet was transmitted by the node, etc.). Based on this information, different actions can be taken.

Security Considerations The IPFIX Information Elements introduced in this document do not directly introduce security issues. Rather, they define a set of performance metrics that may, for privacy or business issues, be considered sensitive information. For example, exporting delay metrics may make attacks possible by the receiver of this information; otherwise, this would only be possible for direct observers of the reported Flows along the data path. IPFIX collectors MUST ensure that IPFIX data originates from trusted sources. Accepting IPFIX data from unauthenticated sources could lead to data spoofing, policy misapplication, or denial of service. Therefore, the underlying protocol used to exchange the information described here must apply appropriate procedures to guarantee the integrity and confidentiality of the exported information. These protocols are defined in separate documents; specifically, see the IPFIX security considerations in . Implementations SHOULD also refer to for additional details.

References Normative References This document formally deprecates Transport Layer Security (TLS) versions 1.0 (RFC 2246) and 1.1 (RFC 4346). Accordingly, those documents have been moved to Historic status. These versions lack support for current and recommended cryptographic algorithms and mechanisms, and various government and industry profiles of applications using TLS now mandate avoiding these old TLS versions. TLS version 1.2 became the recommended version for IETF protocols in 2008 (subsequently being obsoleted by TLS version 1.3 in 2018), providing sufficient time to transition away from older versions. Removing support for older versions from implementations reduces the attack surface, reduces opportunity for misconfiguration, and streamlines library and product maintenance. This document also deprecates Datagram TLS (DTLS) version 1.0 (RFC 4347) but not DTLS version 1.2, and there is no DTLS version 1.1. This document updates many RFCs that normatively refer to TLS version 1.0 or TLS version 1.1, as described herein. This document also updates the best practices for TLS usage in RFC 7525; hence, it is part of BCP 195. Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) are used to protect data exchanged over a wide range of application protocols and can also form the basis for secure transport protocols. Over the years, the industry has witnessed several serious attacks on TLS and DTLS, including attacks on the most commonly used cipher suites and their modes of operation. This document provides the latest recommendations for ensuring the security of deployed services that use TLS and DTLS. These recommendations are applicable to the majority of use cases. RFC 7525, an earlier version of the TLS recommendations, was published when the industry was transitioning to TLS 1.2. Years later, this transition is largely complete, and TLS 1.3 is widely available. This document updates the guidance given the new environment and obsoletes RFC 7525. In addition, this document updates RFCs 5288 and 6066 in view of recent attacks. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document defines a date and time format for use in Internet protocols that is a profile of the ISO 8601 standard for representation of dates and times using the Gregorian calendar. The Network Time Protocol (NTP) is widely used to synchronize computer clocks in the Internet. This document describes NTP version 4 (NTPv4), which is backwards compatible with NTP version 3 (NTPv3), described in RFC 1305, as well as previous versions of the protocol. NTPv4 includes a modified protocol header to accommodate the Internet Protocol version 6 address family. NTPv4 includes fundamental improvements in the mitigation and discipline algorithms that extend the potential accuracy to the tens of microseconds with modern workstations and fast LANs. It includes a dynamic server discovery scheme, so that in many cases, specific server configuration is not required. It corrects certain errors in the NTPv3 design and implementation and includes an optional extension mechanism. This memo utilizes IP performance metrics that are applicable to both complete paths and sub-paths, and it defines relationships to compose a complete path metric from the sub-path metrics with some accuracy with regard to the actual metrics. This is called "spatial composition" in RFC 2330. The memo refers to the framework for metric composition, and provides background and motivation for combining metrics to derive others. The descriptions of several composed metrics and statistics follow. [STANDARDS-TRACK] This document specifies the IP Flow Information Export (IPFIX) protocol, which serves as a means for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information from an Exporting Process to a Collecting Process, a common representation of flow data and a standard means of communicating them are required. This document describes how the IPFIX Data and Template Records are carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. This document obsoletes RFC 5101. This document specifies a set of TCP extensions to improve performance over paths with a large bandwidth * delay product and to provide reliable operation over very high-speed paths. It defines the TCP Window Scale (WS) option and the TCP Timestamps (TS) option and their semantics. The Window Scale option is used to support larger receive windows, while the Timestamps option can be used for at least two distinct mechanisms, Protection Against Wrapped Sequences (PAWS) and Round-Trip Time Measurement (RTTM), that are also described herein. This document obsoletes RFC 1323 and describes changes from it. This memo defines a metric for one-way delay of packets across Internet paths. It builds on notions introduced and discussed in the IP Performance Metrics (IPPM) Framework document, RFC 2330; the reader is assumed to be familiar with that document. This memo makes RFC 2679 obsolete. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document defines the format for the IANA Registry of Performance Metrics. This document also gives a set of guidelines for Registered Performance Metric requesters and reviewers. This memo defines the set of initial entries for the IANA Registry of Performance Metrics. The set includes UDP Round-Trip Latency and Loss, Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson One-Way Delay and Loss, UDP Periodic One-Way Delay and Loss, ICMP Round-Trip Latency and Loss, and TCP Round-Trip Delay and Loss. Informative References Huawei Huawei China Mobile Telecom Italia China Unicom CITC Huawei Work in Progress IANA IANA INSA-Lyon INSA-Lyon Huawei Swisscom This document extends the In Situ Operations, Administration, and Maintenance (IOAM) Direct Export option type to support timestamping by adding and defining two optional timestamp fields and corresponding flags. Work in Progress Swisscom Huawei Huawei China Telecom This document specifies the IP Flow Information Export (IPFIX) Information Elements (IEs) to export Alternate Marking measurement data. Work in Progress This document describes an extension to BGP which may be used to pass additional information to both neighboring and remote BGP peers. [STANDARDS-TRACK] The purpose of this memo is to define a general framework for particular metrics to be developed by the IETF's IP Performance Metrics effort. This memo provides information for the Internet community. It does not specify an Internet standard of any kind. The IP Flow Information Export (IPFIX) protocol defines how IP Flow information can be exported from routers, measurement probes, or other devices. This document provides guidelines for the implementation and use of the IPFIX protocol. Several sets of guidelines address Template management, transport-specific issues, implementation of Exporting and Collecting Processes, and IPFIX implementation on middleboxes (such as firewalls, network address translators, tunnel endpoints, packet classifiers, etc.). This memo provides information for the Internet community. This memo defines the IP Flow Information eXport (IPFIX) architecture for the selective monitoring of IP Flows, and for the export of measured IP Flow information from an IPFIX Device to a Collector. This memo provides information for the Internet community. YANG is a data modeling language used to model configuration and state data manipulated by the Network Configuration Protocol (NETCONF), NETCONF remote procedure calls, and NETCONF notifications. [STANDARDS-TRACK] Consumers of IP network performance metrics have many different uses in mind. This memo provides "long-term" reporting considerations (e.g., hours, days, weeks, or months, as opposed to 10 seconds), based on analysis of the points of view of two key audiences. It describes how these audience categories affect the selection of metric parameters and options when seeking information that serves their needs. This document is not an Internet Standards Track specification; it is published for informational purposes. This document provides a common implementation-independent basis for the interoperable application of the IP Flow Information Export (IPFIX) protocol to the handling of Aggregated Flows, which are IPFIX Flows representing packets from multiple Original Flows sharing some set of common properties. It does this through a detailed terminology and a descriptive Intermediate Aggregation Process architecture, including a specification of methods for Original Flow counting and counter distribution across intervals. This memo provides clear definitions for Active and Passive performance assessment. The construction of Metrics and Methods can be described as either "Active" or "Passive". Some methods may use a subset of both Active and Passive attributes, and we refer to these as "Hybrid Methods". This memo also describes multiple dimensions to help evaluate new methods as they emerge. Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable. In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document discusses the data fields and associated data types for IOAM. IOAM-Data-Fields can be encapsulated into a variety of protocols, such as Network Service Header (NSH), Segment Routing, Generic Network Virtualization Encapsulation (Geneve), or IPv6. IOAM can be used to complement OAM mechanisms based on, e.g., ICMP or other types of probe packets. Network telemetry is a technology for gaining network insight and facilitating efficient and automated network management. It encompasses various techniques for remote data generation, collection, correlation, and consumption. This document describes an architectural framework for network telemetry, motivated by challenges that are encountered as part of the operation of networks and by the requirements that ensue. This document clarifies the terminology and classifies the modules and components of a network telemetry system from different perspectives. The framework and taxonomy help to set a common ground for the collection of related work and provide guidance for related technique and standard developments. In situ Operations, Administration, and Maintenance (IOAM) is used for recording and collecting operational and telemetry information. Specifically, IOAM allows telemetry data to be pushed into data packets while they traverse the network. This document introduces a new IOAM option type (denoted IOAM-Option-Type) called the "IOAM Direct Export (DEX) Option-Type". This Option-Type is used as a trigger for IOAM data to be directly exported or locally aggregated without being pushed into in-flight data packets. The exporting method and format are outside the scope of this document. This document describes how the Alternate-Marking Method can be used as a passive performance measurement tool in an IPv6 domain. It defines an Extension Header Option to encode Alternate-Marking information in both the Hop-by-Hop Options Header and Destination Options Header. This document defines a collection of common data types to be used with the YANG data modeling language. It includes several new type definitions and obsoletes RFC 6991. This document describes an alternative experimental approach for the application of the Alternate-Marking Method to Segment Routing for IPv6 (SRv6). It uses an experimental TLV in the Segment Routing Header (SRH); thus, participation in this experiment should be between coordinating parties in a controlled domain. This approach has potential scaling and simplification benefits over the technique described in RFC 9343, and the scope of the experiment is to determine whether those are significant and attractive to the community. This protocol extension has been developed outside the IETF as an alternative to the IETF's Standards Track specification RFC 9343 and it does not have IETF consensus. It is published here to guide experimental implementation and ensure interoperability among implementations to better determine the value of this approach. Researchers are invited to submit their evaluations of this work to the Independent Submissions Editor or to the IETF SPRING Working Group as Internet-Drafts.

IPFIX Encoding Examples This appendix represents two different encodings for the newly introduced IEs. Let's take as a topology example. shows the aggregated delay with ingressInterface, egressInterface, destinationIPv6Address, and srhActiveSegmentIPv6. Aggregated Delay with egressInterface and srhActiveSegmentIPv6

ingressInterface	271
egressInterface	276
destinationIPv6Address	2001:db8::3
srhActiveSegmentIPv6	2001:db8::2
packetDeltaCount	5
pathDelayMeanDeltaMicroseconds	36 µs
pathDelayMinDeltaMicroseconds	22 µs
pathDelayMaxDeltaMicroseconds	74 µs

Aggregated On-Path Delay Examples

Template Record and Data Set with Mean Delta With encoding in , the mean (average) path delay is calculated on the exporting node.

• Ingress interface => ingressInterface
• Egress interface => egressInterface
• IPv6 destination address => destinationIPv6Address
• Active SRv6 Segment => srhActiveSegmentIPv6
• Packet Delta Count => packetDeltaCount
• Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (531)
• Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (532)
• Mean One-Way Delay => pathDelayMeanDeltaMicroseconds (530)

Template Record for pathDelayMeanDeltaMicroseconds 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SET ID = 2 | Length = 40 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID = 256 | Field Count = 8 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| ingressInterface = 10 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| egressInterface = 14 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| destinationIPv6Address = 28 | Field Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| srhActiveSegmentIPv6 = 495 | Field Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| packetDeltaCount = 5 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelayMeanDelta.. = 530 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelayMinDelta.. = 531 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelayMaxDelta.. = 532 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The data set is represented as follows:

Data Set Encoding for pathDelayMeanDeltaMicroseconds 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SET ID = 256 | Length = 60 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ingressInterface = 271 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | egressInterface = 276 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | destinationIPv6Address = | | ... | | ... | | 2001:db8::2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | srhActiveSegmentIPv6 = ... | | ... | | ... | | 2001:db8::3 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | packetDeltaCount = 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelayMeanDeltaMicroseconds = 36 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelayMinDeltaMicroseconds = 22 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelayMaxDeltaMicroseconds = 74 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Template Record and Data Set with Sum Delta With encoding in , the mean (average) path delay is calculated on the IPFIX data collection.

• Ingress interface => ingressInterface
• Egress interface => egressInterface
• IPv6 destination address => destinationIPv6Address
• Active SRv6 Segment => srhActiveSegmentIPv6
• Packet Delta Count => packetDeltaCount
• Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (531)
• Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (532)
• Sum of One-Way Delay => pathDelaySumDeltaMicroseconds (533)

Template Record for pathDelaySumDeltaMicroseconds. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SET ID = 2 | Length = 40 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID = 257 | Field Count = 8 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| ingressInterface = 10 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| egressInterface = 14 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| destinationIPv6Address = 28 | Field Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| srhActiveSegmentIPv6 = 495 | Field Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| packetDeltaCount = 5 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelayMinDelta.. = 531 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelayMaxDelta.. = 532 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| pathDelaySumDelta.. = 533 | Field Length = 8 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The data set is represented as follows:

Data Set Encoding for pathDelaySumDeltaMicroseconds 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SET ID = 257 | Length = 64 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ingressInterface = 271 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | egressInterface = 276 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | destinationIPv6Address = | | ... | | ... | | 2001:db8::2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | srhActiveSegmentIPv6 = ... | | ... | | ... | | 2001:db8::3 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | packetDeltaCount = 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelayMinDeltaMicroseconds = 22 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelayMaxDeltaMicroseconds = 74 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | pathDelaySumDeltaMicroseconds = 180 | | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Acknowledgements The authors would like to thank (Rest in Peace, Al), , , , , , , , , , , , , and for their review and valuable comments. Special thanks to (as IPFIX Designated Expert), (as IP Performance Metrics Designated Expert), and to for their very detailed feedback.

Authors' Addresses Swisscom

Binzring 17 Zurich 8045 Switzerland thomas.graf@swisscom.com

Huawei

benoit@everything-ops.net

INSA-Lyon

Lyon France alex.huang-feng@insa-lyon.fr