| Internet-Draft | Privacy-Preserving FL for Agent Systems | July 2026 |
| Kale | Expires 3 January 2027 | [Page] |
This document describes an architecture for privacy-preserving federated learning in multi-tenant agent systems. The architecture is intended for deployments in which agents, tools, services, or model operators need to learn from distributed operational data without centralizing tenant data.¶
The architecture separates agent communication from learning coordination. Existing or emerging agent protocols can provide discovery, messaging, authentication, and transport. This document defines the privacy and security requirements for the learning layer: cohort formation, update submission, secure aggregation, differential privacy, privacy accounting, auditability, and model distribution.¶
The document is scoped to cross-tenant and cross-organization settings. It does not define a new agent protocol, a new transport protocol, or a new machine learning algorithm.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 2 January 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Agent systems increasingly operate across tenants, organizations, services, and trust domains. In these settings, agents often produce operational traces that can improve later behavior. Examples include tool invocation outcomes, retrieval quality signals, planner decisions, task completion signals, safety intervention results, and user feedback.¶
Centralizing these traces can conflict with tenant isolation requirements, contractual restrictions, data protection law, and enterprise security policy. Training only inside each tenant boundary avoids centralization, but it prevents learning from patterns that appear across tenants.¶
Federated learning can reduce the need to centralize raw data by allowing each tenant to compute model updates locally [McMahan17] [Kairouz21]. Differential privacy can bound the information revealed through updates or aggregate outputs [Dwork14]. Secure aggregation can prevent an aggregation service from observing individual tenant updates. Used together, these mechanisms can support cross-tenant learning while reducing privacy risk.¶
These mechanisms do not remove all risk. Model updates can leak information. Multi-round training can increase leakage. Small cohorts can expose participants. Malicious tenants can poison a model. Agent traces can contain sensitive user data, tool outputs, credentials, instructions, and policy decisions. A deployment that uses federated learning still needs explicit privacy accounting, access control, update validation, and audit procedures.¶
This document describes an architecture and a set of requirements for such deployments.¶
This document covers:¶
This document does not cover:¶
This document is intended to complement agent protocol work [AIPROTO] rather than replace it.¶
Agent communication protocols can provide discovery, addressing, session management, authentication, authorization, tool invocation, and message delivery. A privacy-preserving learning layer needs additional semantics that are not usually defined by those protocols:¶
When an agent protocol supports extension metadata, these fields can be carried as protocol extensions. When it does not, they can be carried in a separate coordination API. In either case, the learning layer needs consistent semantics across implementations.¶
This document is also related to privacy-preserving measurement work, which is discussed in Section 1.3.¶
The IETF has standardization work for privacy-preserving measurement: the Distributed Aggregation Protocol (DAP) [DAP] and Verifiable Distributed Aggregation Functions (VDAF) [VDAF]. A reasonable question is whether federated learning for agent systems can simply use DAP. This section states why DAP is not sufficient on its own for the problem in this document, and which parts of that work this architecture can reuse.¶
DAP is designed for aggregate statistics over user measurements: each measurement is submitted once, aggregated in a batch, and the output is a low-dimensional aggregate. Federated learning differs in three ways that matter for privacy:¶
At the same time, two ideas from that work map directly onto this architecture:¶
Feedback on whether existing VDAFs, or new VDAFs, can serve as the update-validity mechanism for federated learning updates is specifically invited (see Appendix C).¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
Consider an agent deployment with multiple tenants. Each tenant has local traces from agent activity. These traces may include prompts, tool calls, retrieval results, generated content, user feedback, safety decisions, and task outcomes.¶
The operator wants to improve one or more models or policies using signals from multiple tenants. At the same time, the operator must preserve tenant isolation and limit disclosure of sensitive data.¶
A centralized approach sends traces to a shared service. That approach can expose sensitive content and can violate data handling requirements. A local-only approach keeps data inside each tenant, but it prevents learning from cross-tenant patterns. A federated approach keeps raw traces local, but it still exposes updates that may leak information.¶
The architecture in this document addresses the following requirements:¶
The architecture has the following participants:¶
Figure 1 shows the relationships between these roles.¶
Tenant A Tenant B Tenant C
+---------+ +---------+ +---------+
| Local | | Local | | Local |
| Data + | | Data + | | Data + |
| Training| | Training| | Training|
+----+----+ +----+----+ +----+----+
| updates or | |
| shares | |
v v v
+-----------------------------------------+
| Aggregation Service |
+---------------------+-------------------+
| aggregate only
v
+-------------+ +---------+ +----------+
| Coordinator |<->| Privacy | | Model |
| (tasks, | | Account.|-->| Registry |
| cohorts) | +---------+ +----------+
+------+------+ | |
| v v
| +---------------------+
+--------->| Audit Log |
+---------------------+
Policy Authority and Verifier constrain
task creation, eligibility, and release.
These roles can be implemented by separate services or combined in a deployment. Combining roles can simplify operations, but it changes the trust model.¶
Agent systems can produce data categories that are different from ordinary application telemetry. A deployment SHOULD classify data before using it for learning.¶
Relevant categories include:¶
Some of these categories can contain secrets, regulated data, business records, or third-party content. Local preprocessing SHOULD remove or transform data that is not needed for the learning task. Local preprocessing SHOULD NOT rely on the model being trained to decide what is safe to expose.¶
Each learning task MUST define:¶
Participants MUST be able to inspect the task configuration before contributing an update. A participant MUST reject a task if the configuration is missing required fields or conflicts with local policy.¶
A typical round proceeds as follows:¶
Figure 2 summarizes the flow for one participant.¶
Coordinator Participant Aggregation Svc
| round metadata, | |
| model version | |
|--------------------->| |
| | verify task config |
| | local training |
| | clip update |
| | add noise share |
| | update / shares |
| |-------------------->|
| | | aggregate
| aggregate (only, for valid cohort) |
|<-------------------------------------------|
| accountant update | |
| release decision | |
| registry record | |
The order of steps can vary. For example, distributed differential privacy can require participants to add calibrated noise before secure aggregation. Central differential privacy can require the aggregation service or a trusted component to add noise after aggregation. The chosen model MUST be recorded in the task configuration.¶
For cross-tenant or cross-organization deployments, secure aggregation [Bonawitz17] is RECOMMENDED. Without secure aggregation, the aggregation service can observe each participant update. That creates a direct leakage path. Group communication among cohort members can be secured using existing mechanisms such as MLS [RFC9420].¶
Secure aggregation SHOULD provide the following properties:¶
Secure aggregation does not by itself provide differential privacy. The aggregate can still leak information, especially with small cohorts, repeated rounds, or correlated updates.¶
Secure aggregation also protects individual updates only under its stated trust and collusion assumptions. Reconstruction of an individual contribution from an aggregate has been demonstrated even when updates are quantized, in settings where the cohort is small and a participant in the cohort is curious [SECAGG-INV]. Cohort-size controls (Section 4.8) and differential privacy are therefore needed in addition to secure aggregation, not as alternatives to it.¶
A deployment MUST state the privacy unit. It is not enough to state epsilon and delta without saying what entity is protected.¶
The privacy unit MAY be:¶
For agent systems, user-level or tenant-level privacy is often more relevant than record-level privacy. Record-level privacy can understate risk when many records belong to the same user or tenant. User-level accounting methods exist for this purpose and SHOULD be used when a single user or tenant can contribute many records [USERDP].¶
A deployment MUST also state the differential privacy model:¶
Local differential privacy reduces trust in the coordinator or aggregation service, but it can reduce utility. Central differential privacy can improve utility, but it requires trust in the component that sees the unnoised aggregate. Distributed differential privacy can reduce trust while preserving more utility, but it adds protocol complexity.¶
Federated learning is usually iterative. A single training round does not describe the full privacy risk.¶
The privacy accountant MUST track:¶
Implementations SHOULD use an accountant suitable for iterative training, such as Renyi differential privacy or another accepted composition method [Abadi16] [DPFL-REVIEW]. The chosen method MUST be documented.¶
Training MUST stop, or the task MUST be reauthorized, when the configured privacy budget is exhausted.¶
Small cohorts increase disclosure risk. A deployment MUST configure a minimum cohort size for each task.¶
The coordinator MUST NOT complete a round if the number of valid participants is below the configured minimum. The coordinator SHOULD delay, merge, or cancel such a round rather than lowering the threshold at runtime.¶
The minimum cohort size SHOULD reflect:¶
The architecture does not assume a single update type. A model update can be a full gradient, a full set of model parameters, a set of sufficient statistics, or a set of parameter-efficient fine-tuning (PEFT) adapters.¶
For agent systems, the model being trained is often a large language model or a component built on one. Full fine-tuning of such models across tenants is rarely practical because of update size and cost. In these deployments the update is usually a parameter-efficient adapter, most commonly a Low-Rank Adaptation (LoRA) adapter, rather than a full gradient [FEDLLM].¶
The update type changes how clipping, noise, secure aggregation, and accounting apply. A deployment MUST record the update type in the task configuration. Clipping and noise MUST be applied to the values that are actually transmitted, not to a notional full gradient.¶
When LoRA or similar low-rank adapters are used, implementations need to account for the following:¶
Parameter-efficient updates reduce communication cost and can reduce the surface for some reconstruction attacks, but they do not remove the need for the privacy controls in this document. Adapter updates can still leak information about local data, and a malicious participant can still use an adapter update to poison a model or to attempt to alter safety behavior.¶
Secure aggregation as described in Section 4.5 limits what the aggregation service can observe. It does not by itself let participants confirm that the aggregation service behaved correctly. A participant generally cannot tell whether its update was included, whether other eligible updates were dropped, whether the reported aggregate matches the updates that were submitted, or whether the released model is the one the round produced.¶
For high-assurance deployments, the architecture supports aggregation integrity through commitments and a signed record of each round. The following objects MAY be produced for a round:¶
When these objects are produced, participants SHOULD be able to verify that the round manifest, participant-set commitment, and aggregate commitment are consistent with the model version they are asked to use in a later round. A deployment that claims aggregation integrity MUST state which objects are produced, who signs them, and how a participant or auditor verifies them.¶
Verifiable aggregation does not replace differential privacy or cohort controls. It addresses the correctness of the aggregation process, not the information content of the aggregate or the released model.¶
This document does not define a new wire protocol. It defines metadata that any carrying protocol or coordination API needs to represent.¶
Task metadata SHOULD include:¶
Round metadata SHOULD include:¶
Update metadata SHOULD include:¶
The update_type field identifies the form of the update, for example full_gradient, full_parameters, statistics, or lora_adapter. When the update is a low-rank adapter, the update schema SHOULD also carry the adapter rank so that the aggregation service can apply a rank-aware aggregation method.¶
Participant identifiers in update metadata SHOULD be pseudonymous where possible. The coordinator can still require an authenticated channel or credential for eligibility enforcement.¶
A model release SHOULD record:¶
Release metadata SHOULD be available to participants and auditors subject to access-control policy.¶
A deployment SHOULD consider at least the following adversaries:¶
The task configuration MUST state which of these adversaries are in scope.¶
Model updates can reveal information about local data. This remains true even when raw data never leaves a tenant. Multi-round training can increase leakage because an attacker can observe how updates change over time.¶
Deployments SHOULD use secure aggregation, differential privacy, cohort-size limits, clipping, and privacy accounting to reduce this risk. Deployments SHOULD avoid releasing per-round metrics that allow differencing across small cohorts.¶
The protection offered by these mechanisms is not absolute, and the protection can be weaker than assumed. Differential privacy applied by clipping and adding noise has been shown to reduce reconstruction risk for shallow models while providing weaker protection for deeper models at usable privacy budgets, and clipping-aware reconstruction attacks have been demonstrated against central differential privacy [DP-GLA] [FL-INFERENCE]. Deployments SHOULD treat differential privacy as a measured reduction in risk under a stated privacy unit and budget, not as a guarantee that updates cannot be inverted. The privacy unit and budget MUST be chosen with the model architecture and update dimension in mind.¶
An attacker may try to infer whether a user, record, tenant, or property was present in training. The risk is higher with small cohorts, rare features, high-dimensional updates, and repeated rounds.¶
Deployments SHOULD define the privacy unit before training. User-level or tenant-level accounting SHOULD be used when the protected entity can contribute many records.¶
The final model can leak information even if individual updates are hidden. Secure aggregation does not prevent leakage through the trained model.¶
Deployments SHOULD evaluate released models for memorization and sensitive-output behavior. A model release SHOULD be blocked when evaluation indicates unacceptable disclosure risk.¶
Malicious participants can submit updates that degrade model quality, insert a backdoor, or affect behavior for selected prompts, tools, users, or tenants. This risk applies to full updates and to parameter-efficient adapter updates.¶
Deployments SHOULD apply update validation, anomaly detection, contribution limits, and release evaluation. Byzantine-resilient aggregation can help, but it is not a substitute for model evaluation. Update validation and anomaly detection require some visibility into updates, which can conflict with secure aggregation and differential privacy. This tension is discussed in Section 6.12.¶
An attacker who can create many participants can influence cohort selection and aggregation. This can reduce privacy and damage model quality.¶
The coordinator MUST authenticate participants and enforce eligibility. Cohort selection SHOULD prevent one administrative entity from appearing as many independent participants unless that behavior is intended and accounted for.¶
A model update from one task, round, or model version MUST NOT be accepted for another. Each update MUST be bound to a task identifier, round identifier, model version, and participant credential or pseudonym.¶
Implementations SHOULD use nonces, timestamps, signatures, and round deadlines to prevent replay.¶
Agent traces can include sensitive content that ordinary telemetry does not contain. Examples include tool credentials, retrieved documents, customer records, approval decisions, policy prompts, and generated content.¶
Local preprocessing SHOULD remove fields not required for the learning task. Tool outputs and retrieved content SHOULD NOT be included in training data unless the task has an explicit policy allowing that use.¶
Agent traces can be influenced by prompt injection, malicious documents, tool metadata, compromised tools, and adversarial retrieval content. If those traces are used for learning, the attack can move from one tenant into a shared model. Security analyses of agent protocols discuss related risks [AGENTRFC].¶
Deployments SHOULD separate trusted labels from untrusted content. They SHOULD record provenance for training examples and SHOULD reject training data from sources that do not meet task policy.¶
Secure aggregation usually assumes limits on collusion. If the aggregation service colludes with enough participants, or if too many participants drop out, individual updates can become exposed.¶
The task configuration MUST state the collusion and dropout assumptions of the secure aggregation method. A round MUST fail if those assumptions are not met.¶
Audit logs are needed for accountability, but logs can also expose sensitive data.¶
Logs SHOULD record events, identifiers, policy decisions, and accounting outputs. Logs SHOULD NOT include raw training data or plaintext model updates. Access to logs MUST be controlled.¶
Two goals in this architecture pull in opposite directions. Privacy mechanisms, in particular secure aggregation and local differential privacy, are designed to prevent any party from seeing an individual participant update. Defenses against poisoning and backdoors usually rely on inspecting individual updates to detect anomalies. A deployment cannot fully satisfy both with the same mechanism.¶
A deployment SHOULD make this trade-off explicit and choose an approach that matches its threat model:¶
The chosen approach MUST be recorded in the task configuration, and the residual poisoning risk under that approach SHOULD be stated in the privacy and security analysis.¶
The coordinator selects cohorts. A compromised or malicious coordinator can use that position against a specific participant even when secure aggregation is in use. A concrete attack: the coordinator selects a cohort of the minimum size in which every participant except the target is controlled by, or colluding with, the attacker. The attacker's own contributions are known, so the target's contribution can be recovered from the aggregate by subtraction. Repeating this across rounds defeats the purpose of secure aggregation without breaking its cryptography.¶
Mitigations include:¶
Combining the coordinator and aggregation service roles in one operator increases the impact of this adversary and SHOULD be reflected in the trust analysis for the deployment mode (Section 7.1).¶
This architecture can be deployed in at least three modes:¶
Each mode has different trust assumptions. The deployment mode MUST be documented.¶
The model registry SHOULD store model versions, release metadata, and rollback state. Participants SHOULD be able to verify the model version used for a training round.¶
The registry SHOULD prevent a model trained under one policy from being released under another policy without review.¶
Some deployments may require evidence that a participant or service is running approved code in an approved environment. Remote attestation, following the RATS architecture [RFC9334], can support that requirement. In RATS terms, the participant or service acts as the Attester, and the Verifier role in this architecture (Section 4.1) evaluates the attestation evidence.¶
Attestation is OPTIONAL in this architecture. If used, the task configuration SHOULD state what evidence is required, who verifies it, and how verification results affect eligibility.¶
Retention periods SHOULD be defined for:¶
Retaining intermediate data can improve debugging, but it increases exposure. Deployments SHOULD retain the minimum data needed for security, accountability, and policy obligations.¶
Tenant data in agent systems is rarely identically distributed. Tenants differ in industry, workload mix, tool sets, languages, and user populations. Training over non-IID data is known to slow convergence and degrade the quality of a single global model [NonIID] [Kairouz21].¶
Heterogeneity interacts with privacy. Slower convergence means more rounds, and more rounds consume more privacy budget for the same utility. Distinctive tenants are also easier targets for membership and property inference because their contributions are more separable. Deployments SHOULD consider heterogeneity when setting the maximum number of rounds and the privacy budget, and SHOULD treat highly distinctive participant populations as a factor in minimum cohort size (Section 4.8).¶
Techniques such as personalization layers or per-tenant adapters can address heterogeneity, but they change what is shared and therefore what leaks. If per-tenant components are kept local and only shared components are aggregated, the privacy analysis applies to the shared components. The task configuration MUST identify which parameters are shared.¶
Federated learning reduces the need to move raw data, but it is not a privacy guarantee by itself. Privacy depends on the full system: participant selection, update design, secure aggregation, privacy accounting, model release policy, and operational controls. General guidance on privacy in protocols is given in [RFC6973].¶
Deployments SHOULD perform a privacy analysis before enabling a learning task. The analysis SHOULD identify:¶
Claims about compliance SHOULD be limited. This architecture can support a compliance program, but it does not by itself establish compliance with any law or regulation.¶
The privacy threats in this document map onto the categories of [RFC6973] as follows. Disclosure and identification arise through update leakage, membership inference, and model inversion (Section 6.2, Section 6.3, Section 6.4). Correlation arises when participant identifiers, round metadata, or per-round metrics allow activity linking across rounds and domains; pseudonymous update metadata (Section 5.3) and restraint in releasing per-round metrics reduce it. Surveillance corresponds to a curious coordinator or aggregation service observing contribution patterns over time, addressed by secure aggregation and cohort controls. Secondary use arises when traces collected for agent operation are used for training beyond the stated task purpose; the task_purpose field and local policy checks (Section 4.3) are the controls. Exclusion arises when participants cannot see how their data was used; release metadata availability (Section 5.4) and aggregation integrity evidence (Section 4.10) reduce it.¶
Retention policies (Section 7.4) govern stored artifacts. They do not remove a participant's influence from a trained model. When a tenant departs the platform, withdraws consent, or invokes a legal deletion right after contributing to training rounds, deleting the tenant's updates and local data does not undo the effect those updates had on released model versions.¶
A deployment MUST decide, before training begins, how contribution removal is handled. Available approaches, each with limits:¶
The chosen approach SHOULD be recorded in the task configuration and disclosed to participants before enrollment. Model registry records (Section 7.2) SHOULD retain enough lineage to identify which released versions include a given participant's rounds, since every removal approach depends on that information.¶
This document has no IANA actions.¶
The following example is illustrative. Values are not recommendations.¶
{
"learning_task": {
"task_id": "agent-tool-ranking-2026-07",
"task_purpose": "Improve tool ranking from local outcome labels",
"model_id": "tool-ranker",
"initial_model_version": "2026.07.01",
"participant_population": "enterprise-tenants",
"privacy_unit": "tenant",
"dp_model": "distributed",
"update_type": "lora_adapter",
"privacy_budget": {
"epsilon": 3.0,
"delta": 0.000001,
"accounting_method": "renyi-dp"
},
"training": {
"maximum_rounds": 100,
"local_epochs": 3,
"clipping_rule": {
"type": "l2",
"bound": 1.0
},
"noise_multiplier": 2.0,
"lora": {
"rank": 8,
"aggregation": "rank-aware"
}
},
"aggregation": {
"method": "secure-aggregation",
"integrity_method": "signed-round-manifest",
"minimum_cohort_size": 25,
"dropout_policy": "fail-below-threshold"
},
"release_policy": {
"requires_privacy_budget_available": true,
"requires_security_evaluation": true,
"requires_model_registry_record": true
},
"retention": {
"plaintext_updates": "not-retained",
"aggregate_updates": "30-days",
"privacy_accounting_state": "life-of-model",
"audit_logs": "365-days"
}
}
}
¶
This appendix shows one composed privacy-loss calculation for the configuration in Appendix A, to make the accounting requirement in Section 4.7 concrete. The numbers below were computed with a Renyi differential privacy accountant for the subsampled Gaussian mechanism. They are illustrative, not normative, and any implementation MUST compute its own values with a maintained accountant library.¶
Assume: privacy unit is the tenant; each round selects a cohort of 25 from a population of 250 eligible tenants (sampling rate 0.1); training runs the maximum 100 rounds; each tenant update is clipped to L2 norm 1.0; Gaussian noise with noise multiplier sigma (noise standard deviation as a multiple of the clipping bound) is applied to each contribution or as combining shares; delta is 1e-6.¶
Composed epsilon at delta = 1e-6 over 100 rounds:¶
Two observations. First, a noise multiplier that looks reasonable for a single round (1.1 is common in the literature for one-shot training) exceeds a budget of epsilon = 3.0 more than twofold when composed over 100 rounds at this sampling rate. A per-round intuition about noise is not a substitute for composed accounting, and a task configuration that states a budget without a consistent noise multiplier, round count, and sampling rate is incoherent. This is why Section 4.7 requires the accountant to track all of these inputs together. Second, the configuration in Appendix A uses sigma = 2.0, which stays within its stated budget of epsilon = 3.0 with a small margin; training MUST stop before any round that would exceed the budget.¶
Utility at tenant-level privacy with sigma = 2.0 depends on the update dimension and the model; for high-dimensional full-gradient updates it may be poor, which is one motivation for parameter-efficient updates (Section 4.9) and for distributed rather than local noise placement (Section 4.6).¶
The following questions are open, and input is invited on each. Responses on the mailing list where this document is discussed, referencing the issue number, are welcome.¶
This version:¶
This version:¶