]>

kixelated@gmail.com

wit moq Hang is a real-time conferencing protocol built on top of moq-lite. A room consists of multiple participants who publish media tracks. All updates are live, such as a change in participants or media tracks.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology Hang is built on top of moq-lite and uses much of the same terminology. A quick recap: Broadcast: A collection of Tracks from a single publisher. Track: A series of Groups, each of which can be delivered and decoded out-of-order. Group: A series of Frames, each of which must be delivered and decoded in-order. Frame: A sized payload of bytes representing a single moment in time. Hang introduces additional terminology: Room: A collection of participants, publishing under a common prefix. Participant: A moq-lite broadcaster that may produce any number of media tracks. Catalog: A JSON document that describes each available media track, supporting live updates. Container: A tiny header in front of each media payload containing the timestamp.

Discovery The first requirement for a real-time conferencing application is to discover other participants in the same room. Hang does this using moq-lite's ANNOUNCE capabilities. A room consists of a path. Any participants within the room MUST publish a broadcast with the room path as a prefix which SHOULD end with the .hang suffix. For example:

A participant issues an ANNOUNCE_PLEASE message to discover any other participants in the same room. The server (relay) will then respond with an ANNOUNCE message for any matching broadcasts, including their own. For example:

If a participant leaves or is disconnected, their broadcast is unannounced. Publishers and subscribers SHOULD terminate any subscriptions once a participant is unannounced.

Catalog The catalog describes the available media tracks for a single participant. It's a JSON document that extends the W3C WebCodecs specification . The catalog is published as a catalog.json track within the broadcast so it can be updated live as the participant's media tracks change. A participant MAY forgo publishing a catalog if it does not wish to publish any media tracks now and in the future. The catalog track consists of multiple groups, one for each update. Each group contains a single frame with UTF-8 JSON. A publisher MUST NOT write multiple frames to a group until a future specification includes a delta-encoding mechanism (via JSON Patch most likely).

Root The root of the catalog is a JSON document with the following schema:

Additional fields MAY be added based on the application. The catalog SHOULD be mostly static, delegating any dynamic content to other tracks. For example, a "chat" section should include the name of a chat track, not individual chat messages. This way catalog updates are rare and a client MAY choose to not subscribe. This specification currently only defines audio and video tracks.

Video A video track contains the necessary information to decode a video stream.

, "display": { "width": number, "height": number, } | undefined, "rotation": number | undefined, "flip": boolean | undefined, } ]]>

The renditions field contains a map of track names to video decoder configurations. See the WebCodecs specification for specifics and registered codecs. Any field carrying raw bytes, notably description, is a hex string (). The display field is the size to render the video at, in pixels. It is separate from a rendition's displayAspectWidth/displayAspectHeight because changing it does not require reinitializing the decoder. In addition to the WebCodecs fields, each rendition MAY carry the fields common to audio and video () plus:

displayAspectWidth and displayAspectHeight give the display aspect ratio of the media, stretching or shrinking the coded pixels. A consumer that understands neither field MUST assume square pixels, a 1:1 ratio. Both MUST be present together; a consumer that sees only one MUST ignore it. For example:

Audio An audio track contains the necessary information to decode an audio stream.

, } ]]>

The renditions field contains a map of track names to audio decoder configurations. See the WebCodecs specification for specifics and registered codecs. Any field carrying raw bytes, notably description, is a hex string (). In addition to the WebCodecs fields, each rendition MAY carry the fields common to audio and video ().

PCM Hang defines the "pcm" audio codec for uncompressed samples. The sampleRate and numberOfChannels fields MUST be present and greater than zero. The description field MUST NOT be present. If bitrate is present, it MUST equal sampleRate * numberOfChannels * 32. Each codec payload consists of interleaved IEEE 754 binary32 samples in little-endian byte order. Samples are ordered by sample frame, then by ascending channel index within each frame. The payload length MUST be a non-zero multiple of 4 * numberOfChannels. The frame timestamp identifies the presentation time of its first sample. The frame duration in seconds is the payload length divided by 4 * numberOfChannels * sampleRate. For example:

Binary Fields A decoder config field carrying raw bytes, notably description (an AllowSharedBufferSource in WebCodecs), is carried in the catalog as a hex string (). A publisher SHOULD emit lowercase hexadecimal characters and MUST NOT emit a 0x prefix or any separators. A consumer MUST accept either case. Note that this differs from the cmaf container's init field (), which is base64 (); the two alphabets overlap, so the encoding cannot be detected and must be specified.

Common Rendition Fields Audio and video renditions share the following fields, extending the WebCodecs decoder config:

broadcast By default a rendition's track lives in the same broadcast that served the catalog. The broadcast field overrides that, naming a different broadcast that publishes the track. The value is a relative path, resolved against the path of the broadcast that served the catalog. It uses the . and .. semantics of a relative URL reference (), for example ../source. A publisher MUST NOT use an absolute path, and a consumer MUST ignore a rendition whose broadcast escapes above the root. This lets a publisher author a catalog that points at tracks it does not republish. For example, a transcoder produces a catalog listing its own downstream renditions alongside the untouched source rendition, referencing the latter in the source broadcast rather than copying the bytes through. A consumer subscribes to such a rendition in the referenced broadcast, using the rendition's track name unchanged.

container The container used to frame this rendition's media, as described in . If absent, it defaults to { "kind": "legacy" }.

jitter The maximum delay, in milliseconds, between a frame being ready and the publisher flushing it. A consumer's jitter buffer SHOULD be at least this large to avoid stalling. If absent, a consumer SHOULD assume each frame is flushed immediately. For example: If each frame is flushed immediately, a video track's jitter is 1000/framerate. If up to 3 B-frames may be emitted in a row, it is 3 * 1000/framerate. If frames are buffered into 2 second segments, it is 2000. An audio frame's duration is codec dependent. AAC often uses 1024 samples per frame, so at 44100Hz an immediately-flushed track's jitter is 23.

Container Audio and video tracks use a container to encapsulate the media payload. A rendition declares its container via the container field of its catalog entry ():

The kind field selects the framing; a consumer MUST ignore a rendition whose kind it does not recognize. Every container shares the same group rules: Each moq-lite group MUST start with a keyframe. If the codec does not support delta frames (e.g. audio), a group MAY consist of multiple keyframes. Otherwise, a group MUST consist of a single keyframe followed by zero or more delta frames.

legacy The default, used when the container field is absent. Each frame starts with a timestamp, a QUIC variable-length integer (62-bit max) encoded in microseconds. The remainder of the payload is codec specific; see the WebCodecs specification for specifics. For example, h.264 with no description field would be annex.b encoded, while h.264 with a description field would be AVCC encoded.

cmaf Each frame is a complete fragmented MP4 fragment (moof+mdat), carrying its own timestamps. The init field is the initialization segment (ftyp+moov) for the track, base64-encoded (). A consumer MUST feed init to the decoder before the first frame.

loc Each frame is a Low Overhead Container frame : a property block, carrying the timestamp among other properties, followed by the codec payload.

Security Considerations TODO Security

IANA Considerations This document has no IANA actions.

moq-lite is designed to fanout live content 1->N across the internet. It leverages QUIC to prioritize important content, avoiding head-of- line blocking while respecting encoding dependencies. While primarily designed for media, the transport is payload agnostic and can be proxied by relays/CDNs without knowledge of codecs, containers, or encryption keys. W3C In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes the commonly used base 64, base 32, and base 16 encoding schemes. It also discusses the use of line-feeds in encoded data, use of padding in encoded data, use of non-alphabet characters in encoded data, use of different encoding alphabets, and canonical encodings. [STANDARDS-TRACK] A Uniform Resource Identifier (URI) is a compact sequence of characters that identifies an abstract or physical resource. This specification defines the generic URI syntax and a process for resolving URI references that might be in relative form, along with guidelines and security considerations for the use of URIs on the Internet. The URI syntax defines a grammar that is a superset of all valid URIs, allowing an implementation to parse the common components of a URI reference without knowing the scheme-specific requirements of every possible identifier. This specification does not define a generative grammar for URIs; that task is performed by the individual specifications of each URI scheme. [STANDARDS-TRACK] Cisco Cisco Microsoft This specification describes a Low Overhead Media Container (LOC) format for encoded and encrypted audio and video media data to be used primarily for interactive Media over QUIC Transport (MOQT). It may be used in the MOQT Streaming Format (MSF) specification, which defines a catalog format for publishers to declare and describe their LOC tracks and for subscribers to consume them. Examples are also provided for building media applications using LOC and MOQT.

Acknowledgments TODO acknowledge.