]> Fastly

kazuhooku@gmail.com

ccwg internet-draft This document defines Rapid Start, a congestion-control startup algorithm. It starts by pacing first full flight over a full RTT, allowing an initial window up to 2× that of classic paced slow start at a comparable sending rate. It then grows the window by 3× per RTT until queue buildup is observed, after which it reverts to classic 2× slow start growth. When congestion is signaled, Rapid Start smoothly converges the window based on delivered data, avoiding bursts and underutilization, before handing over to ordinary congestion avoidance. Discussion Venues Discussion of this document takes place on the Congestion Control Working Group Working Group mailing list (ccwg@ietf.org), which is archived at . Source for this draft and an issue tracker can be found at .

Introduction New transport connections do not know the available bandwidth or the bandwidth-delay product (BDP) of the path, so TCP and QUIC start from an initial window and use an exponential startup (“slow start”; , ) to probe for the bottleneck, often paired with pacing to reduce sender-side burstiness. In practice, paced slow start can still leave performance on the table:

• The sender typically starts by pacing packets for half an RTT and then pausing. When the bottleneck bandwidth is higher than the paced rate, the bottleneck can remain idle for the other half of each RTT.
• Even when the bottleneck is being utilized, utilization remains below capacity until queueing begins.
• When the initial window is much smaller than the path BDP, many round-trips are required to ramp up.

These effects are particularly detrimental to short-lived flows, which may only have a few round-trips to send data and therefore suffer disproportionately from underutilization during the startup. Rapid Start retains the initial-window-based probing model but mitigates these issues. It paces the first full flight over a full estimated RTT, allowing an initial window up to 2× that of classic slow start at a comparable pacing rate. It then grows the congestion window by 3× per round-trip until queue buildup is observed, after which it reverts to classic 2× growth. When congestion is signaled, Rapid Start momentarily blocks sending to allow the bottleneck queue to drain slightly; it then resumes sending while reducing the window gradually in proportion to delivered and lost bytes. Doing so avoids burstiness as well as mitigating the risk of the bottleneck buffer becoming empty and the path becoming underutilized during recovery. After recovery, control is handed over to ordinary congestion avoidance, such as that of NewReno () and QUIC congestion control ().

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Algorithm This section describes the algorithm used by Rapid Start.

Full-RTT Pacing Rapid Start uses a more aggressive growth factor than classic slow start. When such growth is used, sending the initial congestion window as a short burst can make the sender observe a bottleneck overflow earlier than it would under evenly paced transmission. To ensure that Rapid Start observes the path's queueing behavior rather than sender-side burstiness, the sender SHOULD pace the packets over a full RTT, using the current RTT estimate, when it first sends more data than classic slow start with pacing would permit. By pacing the packets over a full RTT, Rapid Start can use an initial window up to 2× that of classic slow start with pacing; spreading the transmission over a full RTT (rather than half an RTT) yields a comparable pacing rate. If this more aggressive transmission overshoots and congestion is signaled, Rapid Start compensates by reducing the congestion window as specified in . Careful Resume provides a compatible way to realize these recommendations: it can defer entry to its Unvalidated Phase until the sender first sends more data than normal congestion control would permit, and it requires packets sent in that phase to be paced based on the current RTT.

Increasing the Congestion Window Like slow start, Rapid Start increases the congestion window as packets are acknowledged. The difference is that when the path appears not to be building a queue, the sender uses a more aggressive startup increase. The sender determines if the path is building a queue by comparing the recent minimum RTT (rtt_floor) against a calculated threshold (queue_buildup_thresh). Let:

• min_rtt be the minimum RTT for the connection so far;
• rtt_floor be the smallest RTT over a recent observation window of approximately one round-trip. For example, an implementation might use a sliding time window of length min_rtt, or simply use currentRoundMinRTT tracked for sequence-based rounds in HyStart++ ; and
• queue_buildup_thresh be min(min_rtt + 4 ms, min_rtt * 1.10), where the additive term (+4 ms) and the multiplicative term (×1.10) are RECOMMENDED defaults.

If rtt_floor is no greater than queue_buildup_thresh, the sender increases the congestion window (cwnd) by 2 bytes for every byte that is newly acknowledged, which results in a 3× growth of cwnd per round-trip. If rtt_floor is greater than queue_buildup_thresh, the sender SHOULD increase the congestion window as in classic slow start; i.e., by 1 byte for every byte that is newly acknowledged, which results in a 2× growth of cwnd per round-trip. The construction of queue_buildup_thresh follows HyStart++'s bounded RTT-inflation approach, but uses a tighter RECOMMENDED threshold because the threshold is used to enable a more aggressive startup increase when queue buildup is unlikely, whereas HyStart++ uses RTT inflation to reduce growth by exiting slow start. Consequently, HyStart++ can be used in conjunction with Rapid Start.

Congestion Handling When Rapid Start observes the first packet loss or an explicit congestion signal (e.g., ECN-CE), the sender enters the first recovery period (TCP: ; QUIC: ), but adjusts the congestion window in an alternative manner to smoothly converge after the more aggressive startup. Specifically, it briefly pauses sending to allow the bottleneck queue to drain slightly, then gradually reduces the congestion window during recovery. When entering the recovery period, the sender slightly scales down the current congestion window using a silence factor. As a result of this reduction, sending is momentarily blocked until bytes-in-flight is no greater than the reduced congestion window, allowing the bottleneck queue to be drained by a controlled amount. Then, for each ACK that results in an update of acknowledged or lost bytes while in the first recovery period, the sender reduces the congestion window in proportion to newly acknowledged or newly declared lost bytes: This approach is designed so that, upon exiting the recovery period, the congestion window becomes the full BDP (the sum of the idle BDP and the bottleneck queue size) multiplied by beta, assuming that packets are dropped only due to bottleneck queue overflow; see . At the same time, it limits the silence period so that no more is drained from the bottleneck queue than under congestion avoidance, reducing the risk of underutilizing the link even when the congestion window must be reduced significantly to compensate for the aggressive ramp-up. The gradual reduction based on newly acknowledged and newly declared lost bytes also avoids burstiness when transmission resumes. To avoid overly sharp reduction caused by losses other than those due to bottleneck queue overflow, the sender SHOULD NOT reduce the congestion window below where pre_recovery_cwnd is the congestion window immediately before entering the recovery period. This lower bound corresponds to the target window size under Rapid Start's most aggressive growth factor (3×), for which the pre-recovery congestion window is three times the full BDP. Separately, the sender MUST NOT reduce the congestion window below the minima specified by or . The sender MAY stop reducing the congestion window once it reaches the initial window multiplied by the window decrease factor. This allows the sender to keep the congestion window at least as large as classic slow start on paths with very small BDPs when transitioning to congestion avoidance. Upon exiting the first recovery period, Rapid Start ends; thereafter, the congestion window is governed by the underlying congestion controller's ordinary rules.

Reduction Factors Assuming that losses are caused only by overflow of the bottleneck queue, the recovery algorithm described in requires the following:

• The post-recovery congestion window becomes the full BDP multiplied by beta, independent of the loss ratio.
• The silence period drains at most 1 - beta times the full BDP from the bottleneck queue, matching congestion avoidance.

These requirements make Rapid Start match congestion avoidance in the post-recovery congestion window and in the maximum amount drained from the bottleneck queue. The first requirement implies: Taken together, these requirements yield: When beta is 0.5, the values are: When beta is 0.7 (i.e., that of CUBIC ), the values are: derives these formulas.

Interaction with ECN provides the rationale for the recovery behavior in terms of the full BDP (which, under loss-based detection, is estimated by probing until the bottleneck queue overflows and packets are dropped). However, when congestion happens on an ECN-capable path, it can be reported via CE marks without requiring packet loss. If Rapid Start enters a recovery period due to a CE mark but no packets are lost, then it exits recovery with a congestion window that is beta times its size immediately before entering recovery. If the growth factor in the last round-trip was 3×, the congestion window upon entering recovery can be larger than with 2×, and therefore the congestion window at the end of recovery (beta times the entry size) can also be larger. This makes the next recovery period start sooner, but otherwise does not change the flow's behavior under ECN-signaled congestion pressure. The other concern is buffer overflow before CE feedback is observed. Under 3× growth, the sender might build up a bottleneck queue that is twice as large as under 2× growth. However, even in the extreme case where a network’s buffering margin is tightly provisioned for a target maximum RTT under conventional slow start (i.e., 2× growth), this larger queue buildup under 3× growth simply halves the loss-free RTT range: only connections with RTTs above half of that target maximum would be affected. In practice, networks do not generally provision buffers that tightly; with ECN, they can signal congestion without relying on drop, so leaving extra buffering margin typically has little downside. For these reasons, this overflow risk is limited in practice. On loss-based paths, a more aggressive startup increases the likelihood of overflowing the bottleneck buffer and triggering packet drops, which delays delivery to the application due to retransmission. In contrast, on ECN-capable paths, congestion is typically signaled without relying on packet drops, so this loss-induced delivery delay mode is largely avoided. The benefits of faster growth of the congestion window are thus more reliable.

Considerations Rapid Start's startup and recovery behavior is driven by feedback from ACKs and loss detection. In practice, packet transmission and ACK reception can be affected by scheduling delays and buffering within the host network stack and along the path, which can make observed RTT signals noisier and reduce the smoothness of the algorithm's response compared to an idealized per-packet model.

Considerations for TCP Rapid Start's recovery behavior is based on the QUIC-style model of tracking newly delivered and newly declared lost bytes as ACKs are processed. In QUIC, these quantities can be computed directly from acknowledged packet ranges and loss declarations over packet numbers. TCP implementations vary in how delivery and loss information is represented and exposed to congestion control; loss may be declared in multiple waves as the SACK scoreboard evolves, and accurately accounting newly declared lost bytes can be implementation-dependent (e.g., avoiding double-counting across reordering and retransmission heuristics); RTO-driven recovery can further reduce the timeliness and fidelity of these signals. As a result, TCP implementations might not be able to produce a reliable estimate of delivered and newly declared lost bytes during the first recovery period, especially when loss is high. Therefore, it is up to each TCP implementation to determine whether and how the required delivered/lost byte accounting can be approximated robustly.

Security Considerations TODO Security

IANA Considerations This document has no IANA actions.

References Normative References This document defines TCP's four intertwined congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery. In addition, the document specifies how TCP should begin transmission after a relatively long idle period, as well as discussing various acknowledgment generation methods. This document obsoletes RFC 2581. [STANDARDS-TRACK] This document describes loss detection and congestion control mechanisms for QUIC. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References RFC 5681 documents the following four intertwined TCP congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery. RFC 5681 explicitly allows certain modifications of these algorithms, including modifications that use the TCP Selective Acknowledgment (SACK) option (RFC 2883), and modifications that respond to "partial acknowledgments" (ACKs that cover new data, but not all the data outstanding when loss was detected) in the absence of SACK. This document describes a specific algorithm for responding to partial acknowledgments, referred to as "NewReno". This response to partial acknowledgments was first proposed by Janey Hoe. This document obsoletes RFC 3782. [STANDARDS-TRACK] Thales Alenia Space Orange University of Aberdeen University of Aberdeen Private Octopus Inc. This document specifies a cautious method for Internet transports that enables fast startup of congestion control for a wide range of connections, known as "Careful Resume". It reuses a set of computed congestion control parameters that are based on previously observed path characteristics between the same pair of transport endpoints. These parameters are saved, allowing them to be later used to modify the congestion control behaviour of a subsequent connection. It describes assumptions and defines requirements for how a sender utilises these parameters to provide opportunities for a connection to more rapidly get up to speed and rapidly utilise available capacity. It discusses how use of this method impacts the capacity at a shared network bottleneck and the safe response that is needed after any indication that the new rate is inappropriate. This document describes HyStart++, a simple modification to the slow start phase of congestion control algorithms. Slow start can overshoot the ideal send rate in many cases, causing high packet loss and poor performance. HyStart++ uses increase in round-trip delay as a heuristic to find an exit point before possible overshoot. It also adds a mitigation to prevent jitter from causing premature slow start exit. CUBIC is a standard TCP congestion control algorithm that uses a cubic function instead of a linear congestion window increase function to improve scalability and stability over fast and long-distance networks. CUBIC has been adopted as the default TCP congestion control algorithm by the Linux, Windows, and Apple stacks. This document updates the specification of CUBIC to include algorithmic improvements based on these implementations and recent academic work. Based on the extensive deployment experience with CUBIC, this document also moves the specification to the Standards Track and obsoletes RFC 8312. This document also updates RFC 5681, to allow for CUBIC's occasionally more aggressive sending behavior. This document specifies a Standards Track version of the Proportional Rate Reduction (PRR) algorithm that obsoletes the Experimental version described in RFC 6937. PRR regulates the amount of data sent by TCP or other transport protocols during fast recovery. PRR accurately regulates the actual flight size through recovery such that at the end of recovery it will be as close as possible to the slow start threshold (ssthresh), as determined by the congestion control algorithm.

Deriving the Reduction Factors This appendix derives the reduction factors specified in and used by the recovery algorithm in . For the derivation, consider a tail-drop model in which packets are lost only due to overflow of the bottleneck queue, and assume that the sender is congestion-window-limited when entering recovery. Let full_bdp be the path's full bandwidth-delay product. Let bytes_acked and bytes_lost denote the number of bytes newly acknowledged and newly declared lost during a recovery period. Immediately before entering the recovery period, the congestion window equals the bytes in flight. By the end of the recovery period, those bytes are either newly acknowledged or newly declared lost. Therefore: The congestion window after processing all acknowledgements and losses in the recovery period is: The first requirement of is that the post-recovery congestion window becomes full_bdp * beta, independent of the loss ratio. Under the derivation assumptions, the bytes newly acknowledged during recovery equal one full BDP, i.e., bytes_acked = full_bdp. Substituting these relations into the expression for post_recovery_cwnd gives: For this expression to equal full_bdp * beta independent of the loss ratio, the coefficient of bytes_lost must be zero, and the coefficient of bytes_acked must equal beta. Therefore: or equivalently: To derive factors satisfying the second requirement — that the silence period drains at most 1 - beta times the full BDP from the bottleneck queue, matching congestion avoidance — it is sufficient to consider the case that yields the longest silence period. In the recovery algorithm of , the sender resumes transmission when the congestion window catches up with bytes in flight. Under this recovery rule, the silence period becomes longer as the loss ratio increases. Because Rapid Start uses at most a growth factor of 3, the largest possible loss ratio is 2/3, at which point the silence period is the longest. Because the loss ratio is 2/3, the congestion window immediately before entering recovery is three times the full BDP: Let x be the number of newly acknowledged bytes, normalized by the full BDP, measured from the start of the recovery period. Because two bytes are newly declared lost for every byte that is newly acknowledged, after x * full_bdp bytes are acknowledged, bytes in flight is: Over the same interval, the congestion window is: The sender resumes transmission once the congestion window catches up with bytes in flight. Satisfying the second requirement is therefore equivalent to requiring that transmission resumes when: At that point, cwnd = bytes_in_flight, therefore: Dividing by full_bdp and substituting x = 1 - beta: Substituting: into the previous equation yields: which simplifies to: Equivalently, using a constant K, the resulting formulas are:

Acknowledgments Rapid Start combines three ideas: (1) pacing the first full flight over a full RTT, (2) a more aggressive startup increase when queue buildup is not observed, and (3) a recovery behavior that smoothly converges the congestion window. Careful Resume provides a predecessor for (1): it paces the first flight over a full RTT, based on a current RTT estimate, to avoid bursts when (re)starting. Rapid Start applies the same full-RTT pacing principle when starting. "SUSS: Improving TCP Performance by Speeding Up Slow-Start" (Mahdi Arghavani et al.) advocates a similar approach for (2), built on top of HyStart that increases the congestion window by up to 4× per round-trip based on ACK dispersal and RTT. Compared to SUSS, Rapid Start bases the (2) decision solely on RTT-based queue buildup detection, making it easier to integrate with other mechanisms and specifications such as HyStart++ . For (3), Proportional Rate Reduction is related work in that it regulates sending during recovery to avoid bursts and underutilization. Rapid Start differs by defining a startup-specific recovery behavior, allowing the congestion window to smoothly converge before handing over to congestion avoidance.