Jason Transfer Protocol (JTP)

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

1.2. Terminology

Client

An endpoint that initiates connections to a server and sends JTP requests.¶

Server

An endpoint that accepts connections from clients and serves JTP responses.¶

ImageID

A 64-bit content identifier derived from the raw bytes of an image file, computed as xxHash64(image_bytes, seed=0).¶

Varint

An unsigned LEB128 (Little-Endian Base 128) encoding of a 32-bit unsigned integer, as defined in Section 4.2.¶

Catalog

The set of (ImageID, flags, filename, size) tuples returned by a LIST response.¶

Image Packet

The wire encoding of a single image, comprising flags, length, ImageID, and raw data bytes.¶

Delta Sync

The BATCH operation in which a client sends its set of known ImageIDs and the server returns only the images the client does not have.¶

2.1. Connection Reuse (Keep-Alive)

JTP supports connection reuse through a keep-alive mechanism. When enabled, multiple request/response exchanges may be performed over a single connection, amortizing connection establishment and TLS handshake costs.¶

• If the keep-alive flag (bit 0 of RequestFlags) is set in a request, the server SHOULD keep the connection open after completing the response and await the next request.¶
• If the keep-alive flag is not set, the server SHOULD close the connection after the response has been fully sent.¶
• Servers MAY implement idle timeouts and close stale keep-alive connections unilaterally.¶
• Clients SHOULD NOT assume keep-alive is honored. Clients MUST handle server-initiated connection closure gracefully at any point after a complete response has been received.¶

Legacy deployments MAY use a one-request-per-connection mode: the client opens a connection, sends exactly one request, reads the complete response, and then the connection is closed by either party.¶

3.1. TLS and Application-Layer Protocol Negotiation

When TLS is used, servers MAY advertise the following ALPN [RFC7301] protocol identifier:¶

• jtp/1¶

Clients that support ALPN SHOULD offer jtp/1 during the TLS handshake. A server that does not recognize the offered ALPN identifier MAY proceed without ALPN selection or abort the handshake.¶

JTP does not define certificate distribution. Deployments MAY use self-signed certificates, a locally trusted certificate authority, or a certificate issued by a public PKI. Clients SHOULD verify the server certificate according to the applicable PKI policy.¶

4.1. Unsigned Integers: u8, u16, u32, u64

JTP uses unsigned integers of 8, 16, 32, and 64 bits. Unless otherwise specified, all multi-byte fixed-width integers are encoded in network byte order (big-endian).¶

4.2. Variable-Length Integer: varint(u32)

The varint(u32) type uses unsigned LEB128 (Little-Endian Base 128) encoding. LEB128 is described in the DWARF debugging standard and is in common use in binary protocols (e.g., WebAssembly, Protocol Buffers). The following properties apply to the JTP varint(u32) type:¶

• Encodable range: 0 through 4,294,967,295 (0x00000000 through 0xFFFFFFFF), inclusive.¶
• Encoded length: 1 to 5 bytes.¶
• Each byte stores 7 data bits in bits 0 through 6. Bit 7 (0x80) is the continuation bit; a value of 1 indicates that additional bytes follow.¶

Canonical encoding: Implementations SHOULD produce the minimal (canonical) encoding, i.e., no unnecessary high-order zero groups. Receivers MAY reject non-canonical encodings as malformed.¶

Example: the value 4660 (0x00001234) encodes as the two-byte sequence 0xB4 0x24. Derivation: 4660 in binary is 0001 0010 0011 0100. Split into 7-bit groups from least significant: 0110100 (0x34) and 0100100 (0x24 without the continuation bit, giving 0x24 with bit 7 clear for the final byte). The first byte has bit 7 set: 0xB4. The second byte is 0x24.¶

4.3. UTF-8 Strings

Filenames in the catalog are encoded as UTF-8 [RFC3629] byte sequences. The protocol transmits an explicit byte-length prefix; no null terminator is used. Receivers SHOULD validate that filename bytes constitute well-formed UTF-8.¶

5.1. ImageID

An ImageID is a 64-bit unsigned integer computed from the raw bytes of an image file using the xxHash64 non-cryptographic hash function with a seed value of zero:¶

  ImageID = xxHash64(image_bytes, seed = 0)

The xxHash64 algorithm is defined in the xxHash specification [xxHash]. Implementors MUST use seed value 0.¶

On the wire, ImageID is transmitted as a u64 in big-endian byte order. When rendered in human-readable form (e.g., log output), the RECOMMENDED representation is the 16-character lowercase hexadecimal encoding of the 8 big-endian bytes.¶

ImageID provides content integrity verification but is NOT a cryptographic message authentication code (MAC). An adversary with the ability to modify transmitted data can also forge an ImageID. Cryptographic integrity of image content requires TLS or an equivalent channel security mechanism (see Section 12).¶

6.1. File Type Codes

The three-bit FileType sub-field (bits 0-2) encodes the image format:¶

If the file type cannot be determined, senders SHOULD use code 7 (Unknown / Other).¶

6.2. Compression (Bit 3)

When bit 3 of the Flags field is set, the image data in the enclosing image packet (see Section 8.2) is compressed using Zstandard (Zstd) [RFC8878] at an implementation-defined compression level. Receivers MUST decompress the data before use or integrity verification.¶

Integrity verification via xxHash64 (see Section 5.1) MUST be performed against the decompressed data.¶

If a receiver does not support Zstd decompression and the Compressed bit is set, the receiver SHOULD treat the packet as an error and SHOULD close the connection or send an UnsupportedFeature ERROR response (see Section 9.2).¶

6.3. Encryption (Bit 4)

Bit 4 is reserved for future encryption support. Senders MUST set this bit to 0 in the current version of the protocol. Receivers that encounter this bit set SHOULD treat the packet as an error.¶

6.4. Reserved Bits (Bits 5-7)

Bits 5 through 7 are reserved. Senders MUST set reserved bits to 0. Receivers MAY reject messages containing non-zero reserved bits as malformed.¶

7.1. RequestFlags

Servers MUST reject requests that have any reserved RequestFlags bit set to 1 by either closing the connection or transmitting an InvalidRequest ERROR response (see Section 9.2).¶

7.2. LIST Request (ReqType = 1)

The LIST request asks the server to return a complete catalog of the images it currently serves. It carries no additional payload beyond the fixed two-octet header.¶

The server responds with a LIST response as defined in Section 8.1.¶

7.3. GET_BY_ID Request (ReqType = 0)

The GET_BY_ID request asks the server to return the image data for a specified set of ImageIDs.¶

Semantics:¶

• N MAY be zero, in which case no ImageID fields are present and the server returns zero image packets.¶
• N MUST NOT exceed 255 (the maximum value of a u8).¶
• Servers MAY silently ignore ImageIDs that are not present in the catalog.¶

Response framing: The GET_BY_ID response does not include an explicit top-level count. Clients SHOULD read exactly N image packets (as defined in Section 8.2). Servers that silently skip unknown IDs will therefore produce fewer than N image packets; clients MUST be prepared for this. If the keep-alive flag is set, the connection remains open after the last image packet.¶

7.4. BATCH Request (ReqType = 2)

The BATCH request implements delta synchronization. The client provides the complete set of ImageIDs it already holds; the server responds with only those images the client does not have.¶

Semantics:¶

• The server computes the set difference (server catalog) minus (client's HaveSet) and returns exactly those images.¶
• Servers SHOULD reject BATCH requests in which HaveCount exceeds 1,000,000, responding with an InvalidRequest ERROR.¶

The server responds with a BATCH response as defined in Section 8.3.¶

7.5. LIST_AND_GET Request (ReqType = 5)

The LIST_AND_GET request retrieves all available images in a single round trip, without a prior LIST exchange. The server returns all images together with their ImageIDs; no separate catalog is needed.¶

No additional payload. The server responds with a LIST_AND_GET response as defined in Section 8.4.¶

8.1. LIST Response

The LIST response carries the server's image catalog. It begins with the fixed frame:¶

Each catalog entry has the following structure:¶

Notes:¶

• Size is the byte count of the data field that will appear in an image packet for this ImageID. If the Compressed flag is set, Size reflects the compressed data length.¶
• Filenames are informational only. Clients SHOULD NOT trust or use path components from filenames. Clients SHOULD sanitize filenames before using them as local filesystem paths.¶

8.2. Image Packet

Image packets are the common unit of image delivery used by the GET_BY_ID, BATCH, and LIST_AND_GET responses.¶

Integrity verification: After receiving an image packet (and decompressing if the Compressed flag is set), receivers SHOULD verify:¶

  ImageID == xxHash64(Data, seed = 0)

If verification fails, receivers SHOULD discard the data and SHOULD treat the condition as a transmission error. Continuing to process corrupt data is NOT RECOMMENDED.¶

8.3. BATCH Response

The client reads exactly M image packets following the header.¶

8.4. LIST_AND_GET Response

The client reads exactly N image packets. Because each image packet contains the ImageID, no separate catalog is required.¶

9.1. Structured ERROR Response

Servers that wish to provide machine-readable error information MAY transmit a structured ERROR response in place of the expected response frame:¶

9.2. Error Codes

9.3. Legacy Error Signaling

Servers MAY signal errors by closing the TCP connection or terminating the TLS session without sending a structured ERROR response. Clients MUST handle the following conditions as request failure:¶

• Unexpected TCP or TLS connection closure (EOF) before the response has been fully received.¶
• A response magic header that does not match any known value ("JTPL", "JTPB", "JTPG", "JTPE").¶
• A reserved Flags or RequestFlags bit set to 1.¶
• A varint that is non-canonical or encodes a value exceeding 0xFFFFFFFF.¶
• Any other decoding error that prevents the message from being parsed as specified in this document.¶

12.1. Transport Security

Deployments SHOULD use TLS [RFC8446] to protect JTP connections against passive eavesdropping and active tampering. Without TLS, both image content and ImageIDs are transmitted in the clear, and an on-path attacker may modify image data or inject fabricated responses.¶

12.2. Content Integrity

The xxHash64-derived ImageID provides a non-cryptographic integrity check against accidental corruption (e.g., transmission bit errors). It does NOT provide security against a malicious actor, who can compute a valid xxHash64 over any chosen payload. Applications that require cryptographic integrity assurance MUST rely on TLS record-layer integrity or a separate authenticated mechanism.¶

12.3. Denial of Service

Servers SHOULD validate and bound all count and size fields before allocating memory or performing I/O proportional to those values. Specific recommendations:¶

• Reject BATCH requests with HaveCount exceeding 1,000,000.¶
• Enforce maximum image sizes appropriate to the deployment.¶
• Enforce keep-alive idle timeouts to reclaim connection resources from inactive clients.¶
• Apply request rate limiting (e.g., using the RateLimited error code) to mitigate abusive clients.¶

12.4. Filename Safety

Filenames in the catalog are informational. Clients MUST NOT use catalog filenames as filesystem paths without first sanitizing them. In particular, clients MUST strip or reject path separators, relative path components (e.g., ".." sequences), and any characters not valid in the target filesystem. Failure to do so may allow a malicious server to write files to unintended locations (path traversal).¶

12.5. Certificate Validation

When TLS is used, clients SHOULD validate the server's certificate against a trusted trust anchor appropriate to the deployment. The use of self-signed certificates or a local CA is acceptable in controlled environments but introduces risks if the trust anchor is compromised or mis-deployed.¶

B.1. Request Formats

LIST (ReqType = 1):
  +----------+----------+
  | ReqType  | ReqFlags |
  |  (0x01)  |  (u8)    |
  +----------+----------+

GET_BY_ID (ReqType = 0):
  +----------+----------+-------+-------- - - --------+
  | ReqType  | ReqFlags | Count |  ImageID[0..N-1]    |
  |  (0x00)  |  (u8)    | (u8)  |  N x u64 big-endian |
  +----------+----------+-------+-------- - - --------+

BATCH (ReqType = 2):
  +----------+----------+-------------+-------- - - --------+
  | ReqType  | ReqFlags |  HaveCount  |  ImageID[0..N-1]    |
  |  (0x02)  |  (u8)    | varint(u32) |  N x u64 big-endian |
  +----------+----------+-------------+-------- - - --------+

LIST_AND_GET (ReqType = 5):
  +----------+----------+
  | ReqType  | ReqFlags |
  |  (0x05)  |  (u8)    |
  +----------+----------+

B.2. Response Formats

LIST response ("JTPL"):
  +--------+--------+-------+------ - - ------+
  | "JTPL" | Count  |       Entries            |
  | 4 bytes| (u16)  |  N x catalog entry (var) |
  +--------+--------+-------+------ - - ------+

  Catalog entry:
  +---------+-------+---------+----------+------+
  | ImageID | Flags | NameLen | Filename | Size |
  | (u64)   | (u8)  | (u16)   | NameLen  | var  |
  +---------+-------+---------+----------+------+

Image packet (used in GET_BY_ID, BATCH, LIST_AND_GET responses):
  +-------+--------+---------+------ - ------+
  | Flags | Length | ImageID |     Data       |
  | (u8)  | (var)  | (u64)   | Length bytes   |
  +-------+--------+---------+------ - ------+

BATCH response ("JTPB"):
  +--------+--------------+------ - - ------+
  | "JTPB" | MissingCount |    Images       |
  | 4 bytes| varint(u32)  | M x image pkt   |
  +--------+--------------+------ - - ------+

LIST_AND_GET response ("JTPG"):
  +--------+-------+------ - - ------+
  | "JTPG" | Count |    Images       |
  | 4 bytes| (u16) | N x image pkt   |
  +--------+-------+------ - - ------+

ERROR response ("JTPE"):
  +--------+-----------+------------+-- - - --+
  | "JTPE" | ErrorCode | MessageLen | Message |
  | 4 bytes|   (u8)    |   (u16)    |  var    |
  +--------+-----------+------------+-- - - --+