]> Unheaded

US stevie@bellis.tech

Internet Independent Submission Wotan is the memory and I/O bus for the Unheaded Protocol, providing addressable per-flow storage for BPF programs executing within the Limited Domain. The Wotan protocol specifies the BPF helper interface for memory access, the address space layout for per-flow data structures, a five-level cache hierarchy (L0 through L4), and the topic-based I/O model for interaction with userspace services. This memo defines the memory model, helper functions, address space, cache miss protocol, gRPC streaming contracts, triple-role architecture, reliability guarantees, and I/O topic naming conventions for systems implementing the Unheaded Protocol's computational layer. Draft-03 introduces a structured error code taxonomy with severity levels, helper return codes for common operations, and error recovery procedures. Draft-02 security patches W1-W8 are retained.

Introduction The Unheaded Protocol specifies a 20-byte register file (the Monad) that travels with every packet through a Limited Domain. BPF programs at each hop read and write the Monad, performing stateless per-packet computation. Many use cases require state beyond the 20-byte Monad: buffering input, accumulating results, maintaining per-flow state machines, or storing scratch memory for complex algorithms. Wotan provides this state via a hierarchical memory model: L0: Monad (20 bytes, in packet, per-hop latency ~320 ns) L1: Per-hop BPF map cache (64-byte cache lines, ~100-200 ns latency) L2: Per-flow ring buffer RAM (configurable size, ~1-10 us latency) L3: Write-Ahead Log (persistent storage, ~100 us-1 ms latency) L4: Sophia dictionaries (instruction decode, ~100-200 ns latency) This memo defines the Wotan memory protocol: the BPF helper interface, address space layout, cache coherency model, and userspace I/O interaction.

Cross-References This document is part of the Unheaded Protocol specification family: Protocol Foundation : Defines the Monad wire format (20 bytes, FROZEN at v0x01), per-hop processing, IANA registries, IANA registration procedures, and the wire format immutability threat model. Sophia Dictionary Format : Defines the semantic layer including sub-dictionary type systems for hierarchical knowledge and QPACK compression headers for dictionary entries.

Terminology and Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. The following terms are used:

Flow Label:

The IPv6 Flow Label field (20 bits) or derived hash used to key per-flow state in Wotan. Maps packets to unique per-flow ring buffers.

Ring Buffer:

A BPF ring buffer (BPF_MAP_TYPE_RINGBUF) allocated per flow, used as L2 memory (general-purpose RAM) with configurable size via --ring-size.

Cache Line:

A 64-byte unit of L1 cache (per-hop BPF map), with tag, valid, dirty, and LRU tracking.

Write-Back:

Transfer of dirty cache lines from L1 (per-hop map) to L2 (ring buffer) for persistence or hand-off.

Memory-Mapped I/O:

Designated address ranges that publish to or read from Wotan topics (e.g., write to address 0x0000C000 publishes to compute.screen).

Error Code Taxonomy (NEW in draft-03)

Overview Draft-02 defined error handling for BPF helpers using standard errno codes (-ENOENT, -EFAULT, -ENOMEM, -EACCES, -EINVAL, -EAGAIN). Draft-03 introduces a structured error code taxonomy that classifies errors by severity, origin, and recommended recovery action. This taxonomy applies to all Wotan operations: BPF helper functions, gRPC streaming, WAL operations, and control frame exchanges.

Error Severity Levels Each error code is assigned a severity level:

Structured Error Code Format Error codes in draft-03 use a 32-bit structured format:

Severity:

3-bit severity level (0-4, see above). Bits 31-29.

Origin:

5-bit origin identifier. Bits 28-24.

Category:

8-bit error category. Bits 23-16.

Detail:

8-bit error detail code. Bits 7-0. Interpretation depends on the category.

Helper Return Code Mapping BPF helper functions continue to return standard errno codes for backward compatibility. The structured error code is available via an auxiliary error detail mechanism.

BPF Helper Return Codes

Auxiliary Error Detail When a BPF helper returns a negative errno, implementations SHOULD write the full 32-bit structured error code to a per-CPU BPF array map:

Shim programs MAY read this map after a helper returns an error to obtain detailed error information:

> 29) & 0x7; u8 origin = (*err >> 24) & 0x1F; u8 category = (*err >> 16) & 0xFF; u8 detail = *err & 0xFF; // Handle based on severity/category } } ]]>

Common Error Codes

L1 Cache Errors

63 (read) [ERROR, L1, BOUNDS, 0x02] -EFAULT Offset > 56 (write, 8-byte) [INFO, L1, CONCURRENCY, 0x01] -EAGAIN CAS comparison failed [ERROR, L1, BOUNDS, 0x03] -EFAULT CAS alignment error (not 8-byte) [WARN, L1, RATE_LIMIT, 0x01] -EBUSY Cache-miss rate exceeded (W6) ]]>

L2 Ring Buffer Errors

L3 WAL Errors

gRPC Streaming Errors

Sophia Lookup Errors

Error Recovery Procedures (NEW in draft-03)

Recovery by Severity Level

Automatic Recovery State Machine Each Wotan subsystem (L1, L2, L3, gRPC, Sophia) maintains an independent recovery state machine:

All operations succeeding DEGRADED -> Some operations failing, recovery in progress RECOVERING -> Active recovery procedure executing FAILED -> Recovery exhausted, subsystem disabled Transitions: HEALTHY -> DEGRADED: Trigger: error_count > threshold (configurable, default 10) Action: Enable recovery procedures DEGRADED -> RECOVERING: Trigger: Recovery procedure initiated Action: Execute recovery steps per severity level RECOVERING -> HEALTHY: Trigger: 3 consecutive successful operations Action: Reset error counters, resume normal operation RECOVERING -> FAILED: Trigger: Recovery attempts exhausted (max 5) Action: Disable subsystem, alert operator FAILED -> RECOVERING: Trigger: Operator intervention (manual restart command) Action: Re-execute recovery from clean state DEGRADED -> HEALTHY: Trigger: error_count drops below threshold / 2 Action: Resume normal operation ]]>

Recovery Metrics Implementations MUST export the following recovery metrics:

Cross-Subsystem Recovery When one subsystem enters FAILED state, dependent subsystems MUST be notified:

L2 ring buffer -> L3 WAL gRPC streaming -> L2 ring buffer Sophia lookup -> L1 cache Example: L3 WAL enters FAILED state: 1. L2 ring buffer: Switch to overflow-drop mode (no L3 drain) 2. L1 cache: Continue operating (L1 hits unaffected) 3. gRPC: Continue streaming (events may lack persistence guarantee) 4. Alert: Emit CRITICAL alert on Wotan alerts.* topic 5. Dashboard: Display L3 subsystem as red (FAILED) ]]>

Architecture Overview

Role in the Unheaded Protocol Wotan bridges Monad computation to memory and I/O: Shim programs (BPF) running at each hop read/write Wotan memory via BPF helpers (bpf_wotan_read, bpf_wotan_write, bpf_wotan_cas). Wotan maintains per-flow state keyed by IPv6 Flow Label. Wotan interfaces with userspace via ring buffer events and pub/sub topics. Wotan implements cache miss handling, prefetching, and Write-Ahead Log management.

Memory Hierarchy

Wotan implements transparent L1->L2 promotion on cache miss, L2->L3 flush on overflow, and L3->L2 recovery on process restart.

Separation of Compute and Memory The Monad is transient compute state (stateless by design). Wotan is persistent state machine storage. This separation allows: Shim programs to remain stateless with respect to the packet format. External state to be accessed in a controlled, measurable manner. Cache miss latency to be handled without blocking per-hop logic. Memory updates to be tracked in Anamnesis for observability.

BPF Helper Interface BPF Shim programs access Wotan memory via three helper functions. All helpers operate on a 32-bit address space keyed by IPv6 Flow Label. Error codes follow the taxonomy defined in Section 3.

bpf_wotan_read Read from Wotan memory.

Arguments: - flow_label: 20-bit IPv6 Flow Label (zero-extended to u32) - addr: 32-bit address within the flow's address space - buf: pointer to destination buffer - len: number of bytes to read (MUST be 1, 2, or 4) Returns: - On success: number of bytes read (len) - -ENOENT (-2): flow_label not found [ERROR, L2, RESOURCE, 0x01] - -EFAULT (-14): addr out of bounds [ERROR, L1, BOUNDS, 0x01] - -ENOMEM (-12): L1 cache miss [WARN, L1, RESOURCE, 0x01] - -EACCES (-13): not authorized [ERROR, CORE, ACCESS, 0x01] - -EINVAL (-22): len not in {1, 2, 4} [ERROR, CORE, PROTOCOL, 0x01]

bpf_wotan_write Write to Wotan memory.

Arguments: - flow_label: 20-bit IPv6 Flow Label - addr: 32-bit address within the flow's address space - buf: pointer to source buffer - len: number of bytes to write (MUST be 1, 2, or 4) Returns: - On success: number of bytes written (len) - -ENOENT (-2): flow_label not found [ERROR, L2, RESOURCE, 0x01] - -EFAULT (-14): addr out of bounds [ERROR, L1, BOUNDS, 0x02] - -ENOMEM (-12): L1 cache miss [WARN, L1, RESOURCE, 0x01] - -EACCES (-13): not authorized [ERROR, CORE, ACCESS, 0x01] - -EINVAL (-22): len not in {1, 2, 4} [ERROR, CORE, PROTOCOL, 0x01]

bpf_wotan_cas Atomic compare-and-swap on Wotan memory.

Arguments: - flow_label: 20-bit IPv6 Flow Label - addr: 32-bit address (MUST be 8-byte aligned per PATCH W3) - expected: expected current value (u32) - desired: value to write if current == expected Returns: - 0: swap successful [INFO, L1, NONE, 0x00] - -EAGAIN (-11): current != expected [INFO, L1, CONCURRENCY, 0x01] - -ENOENT (-2): flow_label not found [ERROR, L2, RESOURCE, 0x01] - -EFAULT (-14): addr out of bounds or not 8-byte aligned [ERROR, L1, BOUNDS, 0x03] - -EACCES (-13): not authorized [ERROR, CORE, ACCESS, 0x01]

Error Handling (Enhanced in draft-03) Implementations MUST handle all specified error codes. The structured error taxonomy (Section 3) provides additional detail beyond errno:

RECOMMENDED error handling by severity: INFO (-EAGAIN): Retry immediately (CAS retry loop, max 10 iterations) WARNING (-ENOMEM): Retry via BPF_TAIL_CALL (max 3 attempts) ERROR (-ENOENT, -EFAULT, -EACCES, -EINVAL): Skip operation, use default value, emit EVENT_ANOMALY Programs MUST NOT crash on negative returns; they MUST check return values and branch accordingly.

UPC Memory Model Extensions (NEW in draft-03 update)

Overview The Unheaded Protocol Computer (UPC) extends the Wotan memory model with dedicated BPF map regions for program code, screen I/O, keyboard input, and block device emulation. These extensions enable general- purpose computation within the BPF datapath.

ROM_MAP (Program Code Storage) The ROM_MAP is a BPF hash map that stores MBC program instructions. The program counter (PC) indexes into this map.

ROM_MAP is loaded from a UPCFlat binary (see UPCFlat Binary Format) before the first packet is processed. ROM_MAP MUST NOT be modified during program execution.

RAM_MAP (Read-Write Memory) The RAM_MAP is a BPF hash map that provides the UPC's general-purpose read-write memory. All LD/ST family instructions access this map.

Address Space Layout The RAM_MAP address space follows the layout defined in UPC Memory Region Types:

SCREEN_MAP (Video Output) The SCREEN_MAP stores the pixel framebuffer and is separate from RAM_MAP for performance isolation.

Writes to RAM_MAP addresses in the SCREEN range (0x0007_0000 to 0x0007_FA00) are intercepted and redirected to SCREEN_MAP. A SYS_DRAW_FRAME syscall (0x01) copies the current pixel buffer from RAM_MAP to SCREEN_MAP and emits an EVENT_SCREEN_WRITE (0x14) to the Anamnesis ring buffer.

KBD_MAP (Keyboard Input)

The SYS_GET_KEY syscall (0x02) reads from KBD_MAP. The BPF program checks the sequence counter to determine if a new key event has occurred since the last read. Wotan userspace writes key events to KBD_MAP via bpf_map_update_elem.

CPU_MAP (Processor State)

CPU state persists across packet hops. Each IPv6 flow label maps to an independent CPU instance. The MBC CPU processes one instruction per hop (per-packet computation model).

WAL Specification (NEW in draft-03 update)

Overview The Write-Ahead Log (WAL) provides L3 persistence for Wotan memory. It is an append-only log of dirty cache line writes, enabling crash recovery by replaying WAL records to reconstruct L2 state.

WAL Record Format Each WAL record is exactly 76 bytes:

Total: 8 + 4 + 64 = 76 bytes per record.

WAL Operation

Append When a dirty cache line is evicted from L1 or during periodic writeback (default: every 1 ms), the WAL appends a record:

Implementations MUST call fsync() after each write to guarantee that the record is durable before acknowledging the writeback.

Recovery On process restart, WAL records are replayed in order:

Compaction WAL compaction reduces the file size by removing superseded records (records where a later record writes to the same address):

latest_record map 3. Write retained records to new WAL file 4. Atomically rename new file over old 5. Release compaction lock ]]>

Compaction MUST NOT run concurrently with append operations.

WAL Configuration

TTY Subsystem (NEW in draft-03 update)

Overview The TTY subsystem provides console I/O for UPC programs that use standard Unix-style write(2) and read(2) syscalls on file descriptors 0 (stdin), 1 (stdout), and 2 (stderr). TTY output is captured by the BPF compute engine and forwarded to Wotan as EVENT_TTY_WRITE (0x18) events.

Circular Buffer Each UPC flow maintains a TTY circular buffer in L2 memory for console output accumulation:

Write Operation (SYS_WRITE to fd 1 or 2) When a UPC program issues SYS_WRITE with fd=1 or fd=2:

Read Operation (SYS_READ from fd 0) When a UPC program issues SYS_READ with fd=0:

Event Emission TTY events are published on the Wotan topic:

Dashboard subscribers MAY render TTY output in a terminal emulator widget for real-time program output monitoring.

TTY Configuration

Security Considerations

Topic Injection Attacks A malicious application could publish events to unauthorized topics. Mitigation (MANDATORY): - Topic access control via Sophia (per-program topic whitelist) - Verify publisher identity (program ID from BPF context) - Enforce least-privilege (default: deny all topics) - Log unauthorized publish attempts - Emit ANOMALY event to Anamnesis

Ring Buffer Memory Exhaustion (PATCH W6) Per-program cache-miss rate limiting (10K misses/sec budget). See draft-02 Section 8.2 for complete specification.

Cross-Flow Memory Access (PATCH W2) 64-bit composite L1 cache keys prevent birthday attack collisions. See draft-02 Section 4.1 for complete specification.

CAS Alignment Violations (PATCH W3) 8-byte alignment enforced by BPF verifier at load time. See draft-02 Section 3.2.1 for complete specification.

WAL Tampering Detection (PATCH W4) HMAC-SHA256 authentication on WAL entries. See draft-02 Section 3.1 for specification.

WAL Compaction Race Conditions (PATCH W5) Exclusive mutex during compaction. See draft-02 Section 3.3.

GOAWAY Frame DoS (PATCH W8) Frame validation and rate limiting. See draft-02 Section 10.2.

Normative Error Code Cross-Reference (13 codes) The following 13 normative error codes are defined across the Unheaded Protocol specification family. This cross-reference table provides a single point of lookup:

Error codes 0x0000-0x003F are in the Standards Range and can only be registered during initialization (Specification Required policy per RFC 8126). Extension codes 0x0040-0x00FF are available for protocol extensions. Codes 0x1F00+ are reserved for testing (greasing per the 0x1F*N+0x21 pattern).

Error Level Definitions

Error Code Information Leakage Structured error codes (Section 3) may reveal internal architecture details to an attacker observing BPF program behavior or Wotan metrics. Mitigation: - Error detail codes MUST NOT contain sensitive data (keys, addresses) - Metrics SHOULD aggregate error counts, not individual error details - The wotan_last_error map is per-CPU and ephemeral (not persisted) - External-facing error messages SHOULD use generic descriptions

Cross-Reference with Foundation and Sophia Security considerations in this memo are aligned with: Section 10 - Security Considerations: Wire format immutability threat model, parser divergence attacks, BPF containment, and integrity protection mechanisms. Section 9 - Security Considerations: Dictionary poisoning, nested dictionary security, QPACK decompression security, and BPF map access control.

IANA Considerations This memo does not request IANA registration of option types or protocol numbers; those are handled by . Wotan topic naming uses informal convention (compute., sophia., anamnesis.*). If standardization is needed, IANA may create a registry:

Wotan Error Origin Registry (NEW in draft-03)

Wotan Error Category Registry (NEW in draft-03)

Author's Address Stevie Bellis Unheaded Email: stevie@bellis.tech

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.

Changes from draft-bellis-unheaded-wotan-memory-02 The following changes are made in draft-03: Error Code Taxonomy (NEW): Added Section 3 defining a structured 32-bit error code format with severity levels (INFO, WARNING, ERROR, CRITICAL, FATAL), origin codes (16 subsystems), and category codes (12 categories). Provides fine-grained error classification beyond standard errno codes. Helper Return Codes (ENHANCED): Added structured error code annotations to all BPF helper return values (Section 5). Each errno return now maps to a full 32-bit structured code. Added auxiliary error detail mechanism via per-CPU BPF array map (wotan_last_error). Error Recovery Procedures (NEW): Added Section 3.5 defining recovery procedures by severity level, automatic recovery state machine (HEALTHY -> DEGRADED -> RECOVERING -> FAILED), recovery metrics, and cross-subsystem recovery dependency graph. Cross-References to Foundation draft-06 and Sophia draft-03 (UPDATED): Updated UNHEADED-FOUNDATION reference from draft-04 to draft-06. Updated UNHEADED-SOPHIA reference from draft-01 to draft-03. Added Section 1.1 documenting the specification family structure. Wotan Error Origin Registry (NEW IANA): Added IANA registry for error origin codes (0x00-0x1F). Wotan Error Category Registry (NEW IANA): Added IANA registry for error category codes (0x00-0xFF). Error Code Information Leakage (NEW Security): Added security consideration for structured error codes potentially revealing internal architecture details. UPC Memory Model Extensions (NEW): Added ROM_MAP, RAM_MAP, SCREEN_MAP, KBD_MAP, and CPU_MAP BPF map definitions with complete structure specifications. Defines the UPC address space layout, screen I/O redirection, keyboard input protocol, and per-flow CPU state persistence model. WAL Specification (NEW): Added complete Write-Ahead Log specification with 76-byte record format (8-byte timestamp + 4-byte address + 64-byte cache line data), append/fsync semantics, crash recovery replay procedure, and compaction protocol with exclusive locking. TTY Subsystem (NEW): Added console I/O subsystem with 4 KiB circular buffer, write/read operations on fd 0/1/2, overflow handling, EVENT_TTY_WRITE emission, and Wotan topic publication on compute.tty.{flow_label}. Updated Date: Changed date from 2026-03-05 to 2026-03-15. All draft-02 content is retained, including security patches W1-W8. No existing wire format, processing rule, or normative requirement from draft-02 is modified or removed.

Acknowledgments The Linux kernel BPF community (Alexei Starovoitov, Daniel Borkmann, Song Liu) for the infrastructure enabling per-packet computation in the kernel datapath. The authors of RFC 9669 (BPF ISA), RFC 8799 (Limited Domains), and RFC 9673 (Hop-by-Hop Processing Rehabilitation) for the foundational protocols that make this design possible. This document was co-authored with assistance from Claude (Anthropic).