Automated Discovery Of Audit Reports (audits.json)

Internet-Draft	audits.json	December 2025
Gillmor & Gerchick	Expires 7 June 2026	[Page]

Abstract

This document describes a mechanism that an organization can use to enable automatic discovery of documents associated with regulatory compliance. It is motivated by regulations that require, for example, publicly accessible audits of automated decision-making processes in hiring.¶

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 7 June 2026.¶

1. Introduction

An increasing number of regulatory regimes require organizations involved in certain business practices to provide a degree of transparency in their business operations by posting reports of audits in an accessible location. For example, New York City's 2021 Local Law 144 ([NYC-LL144]) established auditing requirements for employers who use automated decision-making tools in the employment process, but it has proved challenging to even find these audits effectively (see [Auditing-the-Audits]).¶

For a business that has a website, the natural place to provide access to the audit is on that website. This document describes a standard mechanism that can be used to point to any audit posted associated with regulatory compliance.¶

The mechanism is an audits.json summary document, found at a well-known URL, which allows the hosting domain to refer to specific business operations, relevant regulatory regimes, and their associated audit reports.¶

Adoption of this mechanism should make it easier for businesses to comply with these requirements, while also making it easier for researchers, analysts, and regulators to assess compliance and evaluate the overall effectiveness of the regulations.¶

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

1.2. Terminology

An "audit report" is a document produced by an organization or an independent entity commissioned by the organization to describe -- according to some kind of regulatory compliance -- a part of the organization's business operations.¶

1.3. Goals

Provide a standardized way for an organization to publicize audit reports associated with given regulatory requirements.¶
Provide a standardized way for a reviewer of audit reports associated with a regulatory requirement to find those audit reports.¶

1.4. Non-Goals

Discovering an audit report is merely the first step in a process of having an effective regulatory regime based on audits. This specification is focused merely on this satisfying this necessary but insufficient stage of larger work.¶

This specification makes no attempt to describe the syntax or semantics of any particular audit report. The underlying assumption is that the audit reports for different regulations or compliance measures will have different requirements for content, structure, syntax, and so on.¶
This specification also does not associate a real-world organization with a particular domain name. If a researcher wants to find the audits associated with Example Corp, they need to associate Example Corp with the example.com domain name independently in order to use this specification. Likewise, any regulation making use of this mechanism needs to explicitly describe the link between the regulated entity and the domain name or domain names on which it is expected to publish this summary.¶
This specification does not describe a way for a casual visitor to a given website to find these audit reports. A regulation that requires easy public accessibility of a report may need to offer additional user interface or user experience guidance in addition to requiring the use of this mechanism.¶

2. Locating The audits.json Summary

This specification uses the .well-known URL space defined by [RFC8615].¶

A given domain hosts the audits.json summary in /.well-known/audits.json within the website operated by the organization.¶

If Example Corporation operats https://example.com/, then the summary report would be found at https://example.com/.well-known/audits.json.¶

3. audits.json Structure

The object served from the URL described in Section 2 will have Content-Type: application/json, and will consist of a single dictionary object with at least two top-level keys: operations and audits.¶

The overall object relationship looks like this:¶

Figure 1: Data Structure For `audits.json`

3.1. audits Content

The audits member is an object where each member is an object with:¶

title, simple textual string describing the audit, and¶
date, date object describing the calendar date (year, month, and day) of the audit's publication, and¶
urls, an array of URLs that point to the relevant materials for the audit¶

3.2. operations Content

The operations member is a list of objects, each of which describes some set of business operations, via the following members:¶

urls, an array of URLs, each of which describes a business operation covered by this object (for example, a job listing),¶
regs, an array of URLs, each of which refers to a piece of regulatory guidance, and¶
audits, an array of keys which can be used to point to specific audits¶

4. IANA Considerations

IANA should register audits.json in the "Well-Known URIs" registry, with the following values:¶

URI suffix: audits.json¶
Change controller: IETF (is this the right choice, if this is an informational draft? is there a better group to control updates to this specification?)¶
Specification document(s): This document¶
Status: provisional (unless we see wider adoption, in which case we should ask for permanent)¶
Related information: Any other examples we want to point to?¶

5. References

5.1. Normative References

[RFC2119]: Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174]: Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8615]: Nottingham, M., "Well-Known Uniform Resource Identifiers (URIs)", RFC 8615, DOI 10.17487/RFC8615, May 2019, <https://www.rfc-editor.org/rfc/rfc8615>.

5.2. Informative References

[Auditing-the-Audits]: Gerchick, M., Encarnación, R., Tanigawa-Lau, C., Armstrong, L., Gutiérrez, A., and D. Metaxa, "Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits", ACM, Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency pp. 29-44, DOI 10.1145/3715275.3732004, June 2025, <https://doi.org/10.1145/3715275.3732004>.
[CO-SB205]: "Colorado Senate Bill 24-205", May 2024, <https://leg.colorado.gov/sites/default/files/2024a_205_signed.pdf>.
[I-D.bhutton-json-schema-01]: Wright, A., Andrews, H., Hutton, B., and G. Dennis, "JSON Schema: A Media Type for Describing JSON Documents", Work in Progress, Internet-Draft, draft-bhutton-json-schema-01, 10 June 2022, <https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-01>.
[JV]: Tekuri, S. K., "JSONSchema Validation using Go", n.d., <https://github.com/santhosh-tekuri/jsonschema>.
[NYC-LL144]: "Rules of the City of New York: Automated Employment Decision Tools", April 2023, <https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391>.

Appendix A. Test Vectors

A.1. Example audits.json

{
  "operations": [
    {
      "urls": [
        "https://example.com/jobs/sysadmin",
        "https://example.com/jobs/clerk"
      ],
      "regs": [
        "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
      ],
      "audits": [ "fizz", "sparkle" ]
    },
    {
      "urls": [
        "https://example.com/jobs/cfo",
        "https://example.com/jobs/cto"
      ],
      "regs": [
        "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
      ],
      "audits": [ "fizz" ]
    }
  ],
  "audits": {
    "fizz": {
      "title": "Fizz Resume Prioritization Service",
      "urls": [
        "https://fizz.example/bias_audits/aedt_audit_2024.pdf",
        "https://fizz.example/bias_audits/aedt_2024_data.tgz"
      ],
      "date": "2024-12-13"
    },
    "sparkle": {
      "title": "Sparkle Video Analytics Assessment",
      "urls": [
        "https://sparkle.example/video_analytics_2023_audit.pdf"
      ],
      "date": "2024-01-15"
    }
  }
}

Appendix B. JSON Schema For audits.json

The following JSON Schema (see [I-D.bhutton-json-schema-01]) can be used to validate an audits.json summary file.¶

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "http://ietf.org/json_schemas/audit_schema.json",
  "title": "Audit Report Summary",
  "description": "Automated Discovery of Audit Reports",
  "type": "object",
  "properties": {
    "operations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "urls": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          },
          "regs": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          },
          "audits": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "property of top-level 'audits' object",
            "uniqueItems": true,
            "minItems": 1
          }
        },
        "required": [
          "urls",
          "regs",
          "audits"
        ]
      }
    },
    "audits": {
      "type": "object",
      "additionalProperties": {
        "type": "object",
        "properties": {
          "date": {
            "type": "string",
            "format": "date"
          },
          "title": {
            "type": "string"
          },
          "urls": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          }
        },
        "required": [
          "date",
          "title",
          "urls"
        ]
      }
    }
  },
  "required": [
    "operations",
    "audits"
  ]
}

For example, you can validate this using with [JV]:¶

jv audits-schema.json audits.json¶

Note that JSON Schema cannot represent foreign key constraints, so the fact that each /operations/*/audits must reference a named member of /audits needs to be enforced separately.¶

Appendix C. Examples of Regulatory Regimes

Please propose more pointers for this subsection!¶

[NYC-LL144] established requirements for employers in New York City to publish audits of automated decision-making tools used for hiring¶
Colorado's SB 205 [CO-SB205] requires developers of some artificial intelligence systems to publish reports about the design and deployment of such systems on their websites¶

C.1. Example Regulatory Text

A regulation or law that relies on this mechanism to point toward relevant audits can cite it using text similar to the following:¶

Any Covered Organization that has a public-facing website shall publicly point to the Required Audits associated with each position using a widely understood mechanism such as the audits.json specification. The Required Audits associated with this act shall refer to the act as https://springfield.example/citycouncil/acts/2025/114¶

(note that this text presumes that the regulation has defined the terms "Covered Organization" and "Required Audits")¶