| Internet-Draft | Scalable Quality Extension | June 2026 |
| Valin | Expires 14 December 2026 | [Page] |
This document updates [RFC6716] to add support for a scalable quality layer.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 14 December 2026.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
This document updates [RFC6716] to add support for a scalable quality extension layer. Implementations conforming to this document will be referred to as Opus HD.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The Opus codec was designed to operate at sampling frequencies up to 48 kHz, with an audio bandwidth up to 20 kHz. The CELT mode that is used for high bitrate coding uses vector quantization with a mostly implicit bit allocation system that is dictated by the bitstream definition. Opus can allocate up to 8 bits per MDCT bin in some of the bands.¶
While Opus capabilities listed above are sufficient to achieve perceptually transparent audio coding, there is a use for codecs that scale beyond those specs. That includes the current market for 24-bit/96 kHz codecs, but also any application where the intended recipient is not (only) a human being, e.g. ultra-sonic applications.¶
This document proposes a scalable quality extension layer that both increases the resolution of existing Opus quantizers below 20 kHz, and defines a way of coding audio above 20 kHz, with a sampling rate of 96 kHz. The extension is designed to be forward and backward compatible with [RFC6716]. All extra bits use the Opus extension mechanism defined in [opus-extension] and a 96 kHz decoder is designed to decode a regular 48 kHz [RFC6716] stream and vice versa.¶
The code implementing this draft is available on the main branch of the Opus repository at https://gitlab.xiph.org/xiph/opus/ and requires building with --enable-qext (or defining ENABLE_QEXT).¶
To reduce the coding error, we need to increase the resolution for 3 different quantizers: the fine energy quantizer (scalar), the band pyramid vector quantizer (PVQ), and the band splitting angle quantizer. We also introduce a new cubic quantizer that scales to higher bit depths than PVQ. To preserve compatibility, all of the bits extending the Opus resolution are stored in the extension payload.¶
As bands are split into sub-vectors (during stereo coupling or band partitioning), the extension bit budget is split equally between the two partitions. Because the base Opus layer already accounts for the relative magnitudes of the sub-vectors in its own allocation offset, a simple equal split is sufficient to maintain a uniform refinement depth across the two halves.¶
For each sub-vector of dimension N, the refinement bit depth is derived by dividing its budget by the degrees of freedom (N-1 for PVQ, or 2*N-1 for the angle quantizer). In both cases, the depth is clamped between 2 and 14 bits (or 0 if the budget is insufficient). For PVQ sub-vectors, the depth is also reduced if the remaining payload budget is too small to fit the worst-case refinement.¶
For each band we can increase the resolution of the fine energy quantizer by adding extra bits. The extra bits are added in the same way as the regular fine energy quantizer adds resolution on top of the coarse energy quantizer.¶
From a size-K PVQ codebook in N dimensions we can create an extended codebook of size u*K, where u is always odd and selected as 2^b-1, where b is the extra depth. Let y_i be the (integer) value for dimension i of the size-K codebook and z_i be the corresponding value for the size-u*K codebook. We define a refinement r_i = z_i - u*y_i where |r_i| < u. Only the refinement r_i needs to be coded since the regular Opus bitstream already includes y_i.¶
In the N=2 special case, only the single refinement r_0 is coded as a uniform integer in [-(u-1)/2, (u-1)/2]. If the base layer has y_1 != 0, the refined components are reconstructed as: z_0 = y_0 * u + r_0 * sgn(y_1) and z_1 = y_1 * u - r_0 * sgn(y_0). If the base layer has y_1 = 0, the sign of the refined component z_1 is implicit in r_0: z_0 = y_0 * u - |r_0| * sgn(y_0) and z_1 = -r_0 * sgn(y_0). No additional sign bits are coded for N=2.¶
For N>2, even though all |r_i| < u are allowed, smaller values of r_i are more likely, so we benefit from entropy coding r_i. We assume that the likelihood of |r_i| < (u+1)/2 is 7/8 and use that probability for decoding a "large" flag. If large=0, we decode b bits and subtract (u-1)/2 to get r_i. The maximum value 2^b-1 maps to r_i = (u+1)/2 and aliases with a large=1 case. If large=1, we decode a sign bit, followed by an integer with b-1 bits to which we add (u+1)/2 and apply the sign. To prevent exceeding the bit budget, the encoder and decoder calculate the worst-case size of the refinement, which is (N-1)*(b+3)+1 bits. If the remaining bit budget in the extension payload is less than or equal to this worst-case bound, the probability for the 'large' flag falls back to 1/2 (uncompressed uniform coding) to guarantee that the budget is respected. The last residual value r_{N-1} does not need to be coded since it's value can be inferred from the other values and the knowledge that the sum of the absolute values is u*K. The only exception for N > 2 is when y_{N-1}=0, in which case a single sign bit is coded, but the magnitude is still inferred.¶
When using mid-side stereo or when splitting a band, we code an angle representing the atan of two sub-vectors' magnitude ratio. The standard Opus encoder can code angles with up to 8 bits. In a similar way to how we define the PVQ refinement, we pick u = 2^b-1 where u is the number of (equidistant) extra quantization levels to be added between each of the original levels. We code a uint symbol between 0 and u-1, where 0 is almost mid-point to the previous (lower) quantization level, u-1 is almost mid-point to the next (higher) level, and (u-1)/2 perfectly lines up with with the originally selected quantization of the standard Opus layer.¶
The existing Opus PVQ only scales up to 32-bit codebooks. For cases where there is no PVQ in the base Opus layer, we define a new cubic quantizer. Whereas the PVQ codebook is defined as a reflected simplex warped onto the unit sphere, the cubic quantizer warps an N-dimensional cubic shell to the same unit sphere. Cubic codewords specify which face of the cube the vector lies on by coding the dimension of the largest component as a uniform range-coded integer from 0 to N-1, followed by a 1-bit sign of that component. The face of an N-dimensional hyper-cube shell is a full N-1-dimensional cube and can be coded with N-1 scalar values from 0 to Q-1. To minimize complexity, each scalar is encoded directly using b bits, consuming exactly (N-1)*b bits in the bitstream. We use even Q (Q=2^b) for non-transient bands (B==1) and odd Q (Q=2^b-1) for transient bands (B>1).¶
For transient bands, the maximum possible bitwise value (2^b-1) is simply left unused by the encoding. If a decoder encounters this unused value, it MUST process it normally without applying any clamping. In the cubic synthesis process, this evaluates to a coordinate slightly outside the bounds of the hyper-cube face. Because the entire reconstructed vector is subsequently L_2-normalized to the target band energy, this evaluates to a safe, bounded position on the unit sphere. Processing the value blindly prevents the need for branching logic in the decoder's critical path.¶
To extend the audio bandwidth, we define more frequency bands. Because psychoacoustics is no longer involved past 20 kHz, all new bands are defined to have a width of 2 kHz. Therefore, when encoding 48-kHz content we add 2 extra bands and when encoding 96-kHz content, we add 14 extra bands. A flag is encoded to specify whether 2 or 14 bands are added. The decoder uses that flag to know how many bands to decode, regardless of whether decoding at 48 or 96 kHz.¶
The exact band boundaries in terms of MDCT bin indices depend on the short MDCT block size (N_{short}):¶
100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240.¶
74, 82, 90, 98, 106, 114, 122, 130, 138, 146, 154, 162, 168, 174, 180.¶
For a 48 kHz stream, only the first 2 bands (up to index 120 or 90) are decoded.¶
For these extra bands, coarse energy, fine energy, and spectral coefficients are coded in a similar sequence to standard CELT bands in [RFC6716]:¶
When the cubic quantizer is selected for a band, if the block size is larger than 2.5 ms (where the block size index LM > 0) and the budget b (in bits) is greater than 2*N, the band is recursively split into two halves. An angle theta is coded to represent the energy distribution between the two halves. The resolution of theta is derived from the budget and the size of the band. The remaining budget (after coding theta) is split between the two halves: b_1 = min(b, max(0, floor((b - delta)/2))) and b_2 = b - b_1, where delta is a linear approximation of the standard CELT budget offset (described in Section 4.3.5 of [RFC6716]): delta = (N-1) * 2.875 * (theta / (pi/2) - 0.5). The two halves are then recursively quantized, with their gains scaled by cos(theta) and sin(theta) respectively.¶
All entropy decoding for these extra bands uses the extension decoder context.¶
The allocation of the extra bit depth b is explicitly signaled for each band at a time, using a resolution of 1/4 bit depth between 0 and a cap c=14. For band b_i, we use entropy coding to give a higher probability to three different cases: b_i=0, b_i=4*c, and b_i=b_{i-1}. In the case where b_{i-1} is either 0 or 4*c, we merge two of the probabilities. The ICDF for the general case is {120, 112, 70, 0}, where the first symbol means b_i=0, the second means b_i=4*c, the third means b_i=b_{i-1}, and the last symbol means that b_i is equal to 1 plus a uint value coded from 0 to 4*c-1. For b_{i-1} = 0, we use the ICDF {64, 50, 0} and for b_{i-1}=4*c, we use {110, 60, 0}, where the last symbol always means that a uint is coded. We start with b_{-1} = 0.¶
Given b_i, the number of extra energy bits is given by (b_i+3)/4. The number of 1/8 bits (BITRES) allocated for PVQ refinement and/or cubic codebook bits is given by ((W-1)*channels * b_i * 8 + 2)/4, where W is the number of bins in the band.¶
To guarantee decoder synchronization, both the encoder and decoder check the remaining bit budget in the extension payload before decoding the bit depth b_i of each band. If the remaining budget is less than 10 bits (80 in 1/8th bit units), the decoding of bit depths is terminated, and b_i is set to 0 for all remaining bands.¶
The dynamic bit sharing (re-balancing) mechanism for the extension payload follows the same algorithm as standard CELT (described in Section 4.3.3 of [RFC6716]), replacing the base stream variables with their extension payload equivalents.¶
CELT includes two time-domain filter pairs that require updating for 96 kHz: the preemphasis/deempahsis filters, as well as the pitch prefilter/postfilter. The CELT deemphasis filter is currently defined as D(z)=1/(1 - a1*z^-1) for a 48 kHz signal, where a1=27853/32768. To obtain approximately the same response in the 0-20 kHz range using a sampling rate of 96 kHz, we instead use D(z)=g*(1 - b1*z^-1)/(1 - a1*z^-1), where g=5415/8192, b1=7209/32768, a1=30245/32768.¶
For the pitch pre-filter/post-filter, we use zero-insertion upsampling of the 48 kHz filters, which results in the same frequency response below 24 kHz and a "folded" image above 24 kHz. The standard CELT postfilter in the Z-domain is defined as: P(z) = 1/(1 - G*(g_0*z^{-T} + g_1*(z^{-T+1} + z^{-T-1}) + g_2*(z^{-T+2} + z^{-T-2}))) where T is the pitch period, G is the global postfilter gain, and g_0, g_1, g_2 are the tapset gains. For the same pitch at 96 kHz, the filter becomes: P(z) = 1/(1 - G*(g_0*z^{-2T} + g_1*(z^{-2T+2} + z^{-2T-2}) + g_2*(z^{-2T+4} + z^{-2T-4}))) representing symmetric taps at delays (2T-2), (2T), and (2T+2) within the sub-streams.¶
The extension payload is entropy-coded in the following order¶
| Symbol(s) | PDF/Description |
|---|---|
| 96 kHz flag | {1, 1}/2 |
| Intensity stereo | uint |
| Dual stereo | {1, 1}/2, only if Intensity != 0 |
| Intra coarse energy | {7, 1}/8 |
| Coarse energy (high bands) | Section 2.2 |
| Bit allocation | Section 2.3 |
| Fine energy (low bands) | Section 2.1.1, Section 2.3 |
| PVQ refinement | Section 2.1.2, Section 2.1.3 |
| Fine energy (high bands) | Section 2.1.1, Section 2.3 |
| PVQ and cubic codebook (high bands) | Section 2.1.2, Section 2.1.4 |
This section defines some tests for evaluating Opus HD conformance. The evaluations are based on test vectors, along with a custom-made comparison tool named qext_compare and derived from the original opus_compare tool from [RFC6716].¶
For a decoder to conform to this specification, its output MUST be within the specified bounds for all testvectors when compared using qext_compare. Two sets of testvectors are provided. The first, qext_vector01.bit through qext_vector06.bit are high-quality 1024 kb/s bitstreams for which the decoder target files are qext_vector01.f32 through qext_vector06.f32.¶
The testvectors can be downloaded at https://media.xiph.org/opus/ietf/opushd_testvectors.tar.gz.¶
Using the reference decoder, a testvector can be decoded as:¶
% opus_demo -d 96000 2 -f32 qext_vector01.f32 qext_test01.f32¶
Then the output can be compared to the reference with specific thresholds:¶
% qext_compare -s -f32 -thresholds 0.05 0.1 0.1 \
qext_vector01dec.f32 qext_test01.f32
¶
which will output "Comparison PASSED" if the tested decoder is close enough to the target output.¶
The second set of testvectors are meant to test corner cases. The bitstream files are qext_vector01fuzz.bit through qext_vector06fuzz.bit, the corresponding target files qext_vector01fuzz.f32 through qext_vector06fuzz.f32. Those are decoded in the same way as the first set of testvectors, but the comparison thresholds are looser:¶
% qext_compare -s -f32 -thresholds 0.1 0.5 1.0 \
qext_vector01decfuzz.f32 qext_test01fuzz.f32
¶
For the tested decoder to be deemed compliant with this specification, all testvectors from both sets MUST pass.¶
It is RECOMMENDED, but not mandatory for an encoder to comply with the following criteria. Encoder testing involves encoding uncoded testvectors, decoding them with the reference decoder, and comparing to the original uncoded files. The test is meant to evaluate encoding at bitrates around 1 Mb/s. For example, encoding at 1024 kb/s can be done with (may be different for the encoder being tested):¶
% opus_demo -e audio 96000 2 1024000 -f32 -cbr -qext \
qext_vector01.f32 qext_test01.bit
¶
The resulting bitstream then needs to be decoded with the reference implementation with:¶
% opus_demo -d 96000 2 -f32 qext_test01.bit qext_test01enc.f32¶
The decoded output (from the tested encoder) can then be compared against the reference original uncoded PCM:¶
% qext_compare -s -f32 -skip <N> \
-thresholds 0.1 0.5 <RMS threshold> \
qext_vector01.f32 qext_test01enc.f32
¶
where <RMS threshold> for testvectors 1 through 6 are 5, 320, 20, 20, 40, and 5, respectively. The <N> value for the skip compensates for the encoder delay. For the "audio" encoding mode, N=624 samples. For "restricted-lowdelay", N=240 samples.¶
Because of the inherent limitations of objective quality evaluation metrics -- including the qext_compare tool -- it is also RECOMMENDED to perform a subjective evaluation of an encoder.¶
[Note: Until the IANA performs the actions described below, implementers should use 124 instead of 33 as the extension number.]¶
This document assigns ID 33 to the "Opus Extension IDs" registry created in [opus-extension] to implement the proposed scalable quality extension.¶
This document does not add security considerations beyond those already documented in [RFC6716].¶