Structured Email

Introduction Information on websites and in email messages mostly targets human readers. However, various attempts have been made to make such information – fully or in part – machine-readable, so that tools can assist users in dealing with that information more efficiently. One widespread approach is the usage of vocabulary, which can be embedded in the HTML markup of websites. Tools such as web search engines may crawl this information and use it for improving the quality of search result snippets (e.g., by showing ratings, opening hours, or contact information). Similarly, some web shops, hotels, and airlines include Schema.org vocabulary in their order receipt email messages sent to customers. This information is extracted and used by some ISPs and open source tools (). However, these implementations differ in many details. The goal of this specification is to provide a clear and comprehensive specification for this practice and to provide ground for potential future extensions.

Conventions Used in This Document The terms "message" and "email message" refer to "electronic mail messages" or "emails" as specified in . The term "Message User Agent" (MUA) denotes an email client application as per . The terms "machine-readable data" and "structured data" are used in contrast to "human-readable" messages and denote information expressed "in a structured format (..) which can be consumed by another program using consistent processing logic" . The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Representing structured data In order to exchange structured data, one needs to choose a formal language and a serialization format. Based on this, vocabularies can be used to establish a shared understanding of structured data among different parties, such as email senders and receivers.

Knowledge representation language The Resource Description Framework () is a formal language for knowledge representation standardized by the W3C. It is underlying and thus already used for annotating websites and emails. Among the various serializations for RDF, JSON-LD () has become the most commonly used serialization used on websites (). Hence, structured data in email messages SHOULD be expressed in the JSON-LD serialization of RDF. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/1

Vocabularies Using RDF/JSON-LD, users are free to express any kind of information in structured data. For reuse and reference however, it is common to agree upon core concepts/entities and properties for a certain domain. Those are typically collected and shared in so-called vocabularies. is a widespread vocabulary, which was initially designed by search engine vendors for annotating content on websites. A small subset of its concepts is already used by email senders and processed by email providers. Users that want to add structured data into email message SHOULD use concepts from , if they fit their use case. They MAY however use any valid JSON-LD. There might also be certain vocabularies for email-specific use cases (such as [I-D.ietf-sml-structured-vacation-notices-01]), that will be specifically endorsed by the IETF or by respective RFCs. MUAs may choose freely (in consent with its users) if and how to use structured data extracted from messages. If they do not explicitly support a certain vocabulary, MUAs may also rely on extensions or passing data to outside applications, similar to the case of "email attachments" (i.e., MIME body parts with content-disposition type attachment ). For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/2

Structured data in email messages This section defines aspects of adding structured data to a MIME message and its interrelation with other body parts. As a basic distinction, we define two types of messages:

Machine-readable messages (MRM), for which their content is entirely machine-readable
Partly machine-readable messages (PMRM), for which their content is NOT entirely machine-readable (e.g., only describing some of the human-readable content)

Content-types Structured data MUST be added to a MIME message as an application/ld+json body part, which is meant to provide a machine-readable description of the human-readable content of a message part. Alternatively, senders can use a content-type from which application/ld+json can be easily derived, such as application/jose (). For backwards-compatibility, MUAs MAY choose to consider parsing machine-readable content from text/html body parts, as used by some legacy structured email senders (). Structured data can be embedded as JSON-LD or Microdata () in those emails. MUAs are free to parse either syntax. However, MUA and other senders MUST NOT use one of these legacy formats when sending structured email.

Designation This document targets structured data describing the content of an email message itself or complementary structured data to inform the MUA (see "non-representation" case below). This is opposed to structured data which is merely added to an email message as a regular MIME body part for other purposes (i.e., as a normal file attachment). To distinguish these two cases, senders MUST set a header field Content-Purpose to the value Machine-readable on a body part, which is meant to provide machine-readable content. A MUA SHOULD NOT show such body parts as a file attachment in the list of email attachments.

Placement When aiming to describe human-readable content in a machine-readable way, there may exist three general relations between both types of content. In these three cases, the machine-readable version of the content is either:

fully representative of the human-readable content ("Full representation")
describing only parts of the human-readable content ("Partial representation")
describing none of the human-readable content ("Non-representation")

Those distinctions matter for MUAs, as they can make choices for the autoprocessing or presentation of messages and their body parts. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/3

Full representation Full representation denotes the case, in which structured data describes the content of a certain body part in the sense of an "alternative version of the same information" as in the informal definition of multipart-alternative in . If a message is sent to a human recipient, a sender MUST use a multipart/alternative for each body part that is fully described by structured data. In this case, the multipart/alternative should contain a text/plain and a text/html version of the content for backwards compatibility, plus the application/ld+json body part containing the structured data representation. If the MIME tree of a message consists entirely of body parts which have a "full representation" machine-readable alternative of their content, this is called a "machine-readable message" (MRM). A MRM may be suitable for special forms of automated processing, since it contains no information for which interpretation of a human reader is required. While it may be tricky to derive if a message is a MRM in case of complex MIME trees, the majority of messages in practice will contain just one multipart/alternative body part, for which such conclusion is easy to derive. If a MUA is able to process the vocabulary of a MRM or is able to process the structured data otherwise, it SHOULD prefer the application/ld+json representation, unless instructed otherwise by the user. In case of more complex MIME structures, it is up to the discretion of the MUA how to process or render the message. If a sender intends to send a MRM, the subject of a message MUST NOT contain any additional information which is not contained in the machine-readable parts of the message. This is to ensure that MRM may be processed fully automatically if desired. Some countries require senders to include legal disclaimers in email messages. In the case of "full representation", a sender MAY include a "structured email signature" as shown in the Appendix either in the "full representation" structure data or in an additional "non-representation" body part. If a message is sent to a definitely non-human recipient (e.g., an API), human-readable content MAY be omitted from the message entirely and application/ld+json MAY be used as Content-Type in the message header.

Partial representation If structured data is intended to describe only a subset of a certain human-readable body part, it MUST be added as a multipart/related entity. This multipart/related entity MUST also contain the human textual content of the body part (e.g., text/plain and text/html). Also, any MIME body part referenced from the structured data in the structured data body part MUST be enclosed in this multipart/related entity. MUAs SHOULD render such messages as if no structured data body part would be included. MUAs MAY process the structured data for providing an enhanced user experience of their resp. the user's choice.

Non-representation In the case of non-representation, there is no overlap between structured data and the human readable content. This may be useful for special scenarios, such as embedding "preemptive" structured vacation notices as described in [I-D.happel-sml-structured-vacation-notices-00] into email messages. As in the case of partial representation, MUAs receiving such messages may take according action based on the structured data extracted.

Identifiers There are existing use cases for cross-referencing between different parts of a MIME message, embedded images probably being the most widespread one. For this purpose, defines the cid: and mid: URI schemes. In a similar fashion, cross-referencing might occur between structured data and other message parts.

Using identifiers in structured data Most nodes and properties in JSON-LD are identified using IRIs . Since any (cid:/mid:) reference forms a valid IRI, those references can be directly used in JSON-LD. There are two main cases for which cid:-identifiers SHOULD be used in structured data. First, if structured data references binary content such as images or other files, which already exist as MIME body parts within the same message. Second, if a cid: value is used in a JSON-LD @id property, the corresponding JSON-LD node can be considered to describe the MIME body part identified by that cid:. This MAY be used to denote that certain structured data is explicitly describing that MIME body part, such as an image. This particularly MUST NOT be used for the main text/plain or text/html body parts, though. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/4

Using structured data identifiers in text/html In the case of "partial representation", a MUA should still primarily display the human readable part of a message (e.g., text/plain or text/html). It might however be helpful if the MUA is able to determine which parts of human readable text refer to certain structured data - e.g., to offer actions based on structured data directly in the context of the corresponding human-readable content. For this purpose, the sender may add a HTML "data-id" property () to any HTML entity in the text/html body, which references the @id property of a JSON-LD node in the structured data. Besides referencing the corresponding JSON-LD node, a sender might also want to denote if the underlying data is "extensively described" or just "mentioned" in the human readable representation. For example, the New York Times cooking newsletter typically features few recipes, while mentioning a larger number of further recipes, also referencing their web URL. For providing an adequate user experience, the MUA should be able to understand which recipes are featured in an email and which are just mentioned. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/5

Structured data across email messages This sections deals with aspects that go beyond the scope of an individual MIME message.

Forwarding Forwarding messages including structured data needs to be considered from a privacy perspective, particularly in cases of "non-representation", when the user has no way to determine structured data from the human readable part of the message. A MUA MUST strip non-representative structured data when a user is forwarding messages to somebody else in her MUA. Note that this does not apply to automated forwarding of messages. Beyond that, privacy issues also apply to forwarding regular email messages. Improvements of the status quo might hence be considered beyond the specific context of structured email. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/6

Replies Users or agents might want to reply to structured emails with a structured email. Details for this will be specified in a separate future specification.

Updates In human-readable messages, human language can be used to update or recall information that was conveyed in prior messages. Accordingly, there needs to be a machine-readable mechanism that allows to express the update or recall of structured data. To update or recall structured data, senders MUST set the SUPERSEDES header field () of the "update" message with the message id of the original email message. An "update" message with empty structured data can be used to signal a full recall of previously sent structured data. Every "update" message MUST have its own unique message id. The processing of an "update" message by the receiving MUA is up to its own discretion, as meaningful action may depend an multiple factors. MUAs MAY consider:

An update might be triggered by a previous action of the user (e.g., canceling a reservation)
Adding the original message id as an identifier property to capture its origin when persisting structured data

For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/9

Message flags In some use cases, MUAs might benefit from information about message details without having to evaluate the full message body. For example, the $hasAttachment IMAP flag () was proposed to signal the existence of MIME attachments in a message which otherwise would need to be redetermined based on complex MIME parsing. In case of structured data, the receiving MTA (or any later MUA) MAY determine two facts when parsing the MIME tree of a message:

If the message contains any structured data according to this document
If so, if the message is a fully machine-readable message (MRM)

Accordingly, the following IMAP flags MAY be set for a message:

$hasStructuredData (in the first case)
$MRM (in the second case)

The first case may be helpful for filtering or client-side preloading of message content. The second case may be particularly helpful for automated processing without user interaction. For discussion, see also: https://github.com/hhappel/draft-happel-structured-email/issues/10

Examples The following section shows some example MIME hierarchies of email messages containing structured data.

Full representation multpart/alternative/ ├─ text/plain ├─ text/html └─ application/ld+json

Full representation (machine-readable only) application/ld+json

Full representation with inline image multpart/alternative/ ├─ text/plain └─ multipart/related/ ├─ multpart/alternative/ │ ├─ text/html │ ├─ application/ld+json └─ image/png

Partial representation multpart/related/ ├─ multipart/alternative/ │ ├─ text/plain │ └─ text/html └─ application/ld+json

Non-representation multpart/mixed/ ├─ multipart/alternative/ │ ├─ text/plain │ └─ text/html └─ application/ld+json

Appendix (Structured Email Signature) The following snippet of structured data uses the Schema.org publisher property of an EmailMessage. { "@context": "https://schema.org/", "@type": "EmailMessage", "publisher": { "@type": "Organization", "legalName": "MUSEO NACIONAL DEL PRADO DIFUSIÓN, S.A.U., S.M.E.", "legalAddress": { "@type": "PostalAddress", "addressLocality": "Madrid, Spain", "postalCode": "28014", "streetAddress": "Casado del Alisal, 10, bajo B" }, "legalRepresentative" : { "@context": "https://schema.org", "@type": "Person", "name": "Jane Doe", }, "identifier": { "@type": "PropertyValue", "name": "Registration data in the Company Register", "value": "Volume 23578, Entry 1, Section 8, Sheet M-423094, 74 Folio 74" }, "vatID": "A84888056" } }

Security and trust Email user agents that want to support structured email should follow guidance to ensure trust and security standards. These will be elaborated in a separate specification (see [I-D.draft-happel-structured-email-trust-04]).

Implementation status < RFC Editor: before publication please remove this section and the reference to > This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in . The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist. According to , "this will allow reviewers and working groups to assign due consideration to documents that have the benefit of running code, which may serve as evidence of valuable experimentation and feedback that have made the implemented protocols more mature. It is up to the individual working groups to use this information as they see fit".

Structured Email for Nextcloud Mail Nextcloud Mail is an open source Webmail app which includes a subset of structured email support since 2020 . In recent months, code has evolved to support structured email in a more general fashion .

Structured Email plugin for Roundcube Webmail An open source plugin for the Roundcube Webmail software is developed to serve as an example implementation for this specification (). Beyond that, some ISPs and open source tools provide implementation partly compliant with this specficiation ().

Yatagarasu Mail Mail is a fork of the open source Thunderbird Mobile client for Android. It adds structured email support for both reading and sending messages.

Security considerations See section "security and trust".

Privacy considerations See section "security and trust".

IANA Considerations

Creation of the SML registry group IANA will create a new registry group called "Structured Email (SML)". This group includes the "SML Email Vocabulary Registry" and "SML Vocabularies" registries described below.

Creation of the SML Email Vocabulary Registry IANA will create the following registry: Registry Name: "SML IETF Email Vocabulary" Registration Procedure: TBD

Creation of the SML Vocabularies Registry IANA will create the following registry: Registry Name: "SML Vocabularies" Registration Procedure: TBD (TABLE OF REGISTRATIONS) The registry entries contain the following fields: - Vocabulary name: a human readable name for reference - Namespace: the URI namespace prefix for the vocabulary - Documentation page: a web page with further documentation about the vocbulary - Scope: the domain described by that vocabulary - Reference: (IANA) - Notes: any additional notes

Initial Entries for the SML Vocabularies Registry The registry initially contains these entries:

Vocabulary name	Namespace	Documentation page	Scope	Reference	Notes
Schema.org	https://schema.org/	https://schema.org/	General purpose concepts
SML Email Vocabulary	TBD	TBD	Email-specific concepts defined in IETF drafts	(this draft)

Registration of the Content-Purpose MIME header field The following MIME header field is registered in the Message Headers registry, as established in /BCP 90. Header Field: Content-Purpose Description: If the MIME entity is intended as machine-readable information for the MUA. Applicable protocol: MIME Status: standards-track Author/change controller: IETF Specification document(s): This document Related information:

Registration of the $hasStructuredData keyword The following IMAP/JMAP keyword is registered in the IMAP and JMAP Keywords registry, as established in . IMAP/JMAP keyword name: $hasStructuredData Purpose Indicate to the client that a message contains structured data according to this spec. Private or Shared on a server: SHARED Is it an advisory keyword or may it cause an automatic action: This keyword can cause an automatic action. When/by whom the keyword is set/cleared: This keyword is set by the server on delivery. Related keywords: MRM; hasAttachments is peripherally related Related IMAP capabilities: None Security considerations: None Published specification: This document Intended usage: COMMON Scope: BOTH Owner/Change controller: IESG

Registration of the $MRM keyword The following IMAP/JMAP keyword is registered in the IMAP and JMAP Keywords registry, as established in . IMAP/JMAP keyword name: $MRM Purpose Indicate to the client that a message is fully machine-readable according to this specficiation. Private or Shared on a server: SHARED Is it an advisory keyword or may it cause an automatic action: This keyword can cause an automatic action. When/by whom the keyword is set/cleared: This keyword is set by the server on delivery. Related keywords: any message with MRM should have also set the hasStructuredData keyword Related IMAP capabilities: None Security considerations: None Published specification: This document Intended usage: COMMON Scope: BOTH Owner/Change controller: IESG