]> 5}="yes"?> Futurewei Technologies USA

USA tte@cs.fau.de

Sympotech

CA USA ludwig@clemm.org

ANIMA This document builds on the architectural foundation of the IETF ANIMA "Autonomous Network Infrastructure" to propose an architecture for in-network intelligence in support of network automation. The key aspect of this architecture is the use of AI programmed and validated software running decentralized on the network.

Introduction

From LLM to agentic NetOps Since the release of the first mayor "AI agent", chatgpt at the end of 2022, an almost exponential evolution of these agents has sparked a wide area of scenarios how they can be used - and change the world. Given how those AI agents are usually trained around (natural) languages, they are also called Large Language Models (LLM). In networking, agents are already used systematically for analytics of collected network data for various purposes: configuration analysis, anomaly detection, security violation recognition and optimizations. Network equipment vendors are developing and providing LLM for such network operations, are integrating these into network operations product offering and declaring the next wave of network operations to be agentic network operations. However beneficial this approach is, it is layering LLM based "intelligence" on top of an otherwise mostly unchanged network infrastructure and operational technologies framework/toolset. Which makes perfect sense in the short term especially when wanting to continue selling existing solutions plus additional agentic components, but ultimately, this is like adding a robot as a driver to a normal car - instead of building an autonomous car.

Intent Based Networking (IBN) describes the concepts and components of the widely evolving approach to network automation through the introduction of abstractions that allow to represent the networks behavior in (abstract) terms relevant to operators as well as to continuously adjust the behavior of the network under change to comply to equally abstract terms called "Intent". While the use of Agentic components has not been discussed in RFCs yet, such components are starting to be deployed in operators for a subset of the components shown in RFC9315.

Agentic Netops in IBN In agentic network operations, LLM are primarily "out of band" in network operations software running in sites like network operation centers, and the primary change to the network is the desire to continously collect more data so that the LLM can provide more detailled analysis or make better decisions. Primary uses of such agentic components are complex behavior analysis of the network, such as "does the network operate under non-normal conditions", and current work primarily focusses on the even more difficult to answer question then "what is the root cause of the currently experienced anomaly".

Limitations of Agentic Netops There are two key limitation of agentic netwops today: The components of intent based network very rarely if at all built with agentic components are those that actually influence the network configuration directly. This is due to the heuristic nature for the output of LLMs and hence the risk of injecting not only solutions to problems but also potentially larger problems in case of such heuristic mistakes of an LLM. The IBN components have no good operator accessible options to be developed and run on the network equipment itself. In result, it only has the option to be developed and run centrally - even if/when that is not the best choice. This document attempts to address exactly these two points: establish a model allowing more safer use of agentic functionaly and describe the components recommended to deploy them distributed on network equipment itself.

Programmability to the rescue To create confidence into any agentic behavior to perform under any possible conditions, one can perform exhaustive testing, but if the underlying behavior is intrinsically heuristic, then this may not suffice alone to get the necessary confidence. More importantly though, whenever the automation task itself is simple enough that it can be resolved through classical programmed automation software - then exactly that classical programmed automation software could today most likely be also developed by LLM agents. Most complex tasks performed by LLM agents is already performed by the agent visibly or invisibly (dependingon the agent) programming "one-time-use" programs to perform the agent action. And specific type of LLM are also optimized for programming tasks. In additional to the natural affinity of LLM agents to rely on programming, the generation of actual programmed code also has other benefits Validation is not only possible through black box testing as most likely for the LLM - feed large number of test cases and validate results, but also through potentially formal methods or additional white-box testing by creating those internal validation points. Non-LLM programs do not require potentially more expensive LLM inference components which may not be available on all network equipment locally. The traditional (and only) known The proposal of this document is thus to introduce the full scope of IBN compomnents also in a distributed fashion through the use of agentic LLM programmed automation - with associated also LLM driven exhaustive testing and whereever possible model driven formal method validation. This option is of course nonwithstanding the ability to also directly use agentic LLM based automation whereever feasible in a centralized or distributed fashion, but it explicitly introduces the layered approach of agentic LLM Development (or automation software) and Operations - Agentic DevOps. The following chapters describe the proposed architcture and its components.

Background

The Autonomic Network Infrastructure (ANI) To resolve the fragility of the network infrastructure contrl plane as described in , the ANIMA working group of the IETF has defined as set of protocols and architecture components to provide the most basic, fully automated network infrastructur functionality that is not subject to the self-referential reliability problems. It is called "Autonomic Network Infrastructure". The ANI effectively consists of a minimum, non-configurable router for management plane traffic that runs a fixed, scalable routing protocol (RPL) requiring no management policies This is called the "Autonomic Control Plane" (ACP). Inside the ACP, the "Bootstrap Secure Key Infrastructure" (BRSKI) protocol autonomously enrolls devices with PKI keying material relying only on centralized, pre-existing security backend infrastructure, such as a "Certificate Authority" when bringing up new devices. The keying material enrolled, typically X.509 certificates allows for the ANI to operate in a authenticated and confidential manner. The "GeneRic Autonomic Signal Protocol" (GRASP) running also inside the ACP provides network wide coordination such as service announcement and discovery - to support self-orchestration of any networkautomation The BRSKI protocol is used to allow autonomous enrolment of devices with mutually trusted security Traffic of the ANI is multiplexed onto any interfaces of the devices that also carry data-plane traffic. In summary, the functionality of the ANI is designed to be the minimum that allows the layer on top of it to operate as if there was an authenticated and confidential out-of-band network with the option for network wide "broadcast" traffic for mutual coordination.

Network Automation Layer ANIMA defines the network automation layer as consisting of "Autonomous Functions" (AF) that are composed of element on all or some devices called "Autonomic Service Agents" (ASA). These effectively can be seen as an abstraction of control plane protocol or management plane processes interacting with other processes to provide specific functions of network automation. describes basic aspects of these ASA. This layer ultimately is the one that still needs to be explored further, which i the core of this document

Solution Architecture provides a conceptual overview of the proposed architecture to build network automation from which may be as simple as improved control plane or management protocols all the way to autonomous networks. At the core of the architecture is the solution to the above described problem of not trusting automatin solely happening through the heuristics of LLM, which introduce into the actual automation the possible complete and unpredictable conmplexity of the whole LLM. This solution is to simply set up the architecture such that the LLM is not making decisions in the network, but instead it is primarily programming the actual network automation software.

enviroment | +-----------------------------------------------------+ | ^ | AI programmed software | | download (ASA, inference) | (LLM) data collection v | and analysis +-----------------------------------------------------+ | Agent Programmed Network Device | |+-----------------------------------+ +-------------+| || Intelligence Execution Plane | | Legacy || || AF ASA (inference DNN) | | Control / || || control/management processes | | Mgmt || || | | Plane || || Hypervisor / AF/ASA SW-management | | Processes || |+-----------------------------------+ +-------------+| | ^ ^ ^ | | | Mnagement/Control traffic +-------- | | | | Credentials/PKI | +--+ | | | Coordination services | | | | v v v | |+---------------------------------+ +---------------+| || Autnomic Network Infrastructure | | User traffic || || ACP BRSKI GRASP | | Data Plane || || IPsec encrypted data plane | | HW accelerate || |+---------------------------------+ +---------------+| | ^ ^ | | | | | | +--+ +----+ | | | | | +-----------------------------------------------------+ v v Network interfaces ]]>

Agentic Network DevOps Center As described above, agent Netops is already being developed today, but that approach is solely expecting unchanged Legacy Control and Management Plane as well as User Traffic Data Plane on the Network Devices. These functions equally exist in this architecture. The following text does primarily focus though on the function, which is the Dev(evelopment) part of DevOps for the actual automation software running on the network devices: control plane, network management plane - automation/autonomic.

Developing automation

Programming / Automation Intent The goal of the Agentic DevOps Center is to allow intent based development of automation software which will then run as ASA (processes) on the network devices. The degree to which this automation intent requires protocol experts will evolve over time the more the LLM can be trained with programming and behavioural patterns such as can today already be seen in how LLM do evolve to become better and better at other programming tasks, programming languages, and algorithms. An initial version of such agentic Network DevOps programming may utilize very protocol centric prompt descriptions, such as for example describing the desired functionality for an extension to an existing protocol, such as BGP. To support such an agentic extension of existing protocols, the pre-agent implemention may be brought over frm the legacy control/management plane of the network device into the new, agent-managed intelligent execution plane, with possible fallback to the legacy implementation. Later versions of the programming should hopefully be able to describe the task in prompts based on desired outcome, instead of protocol details. When the LLM is accordingly capable of performing the mapping to the available (or desired) protocol mechanisms.

Network Simulation / Emulation The most important part of the DevOps design is the ability to simulate or emulate as much as possible of the target network devices behavior in conjunction with small validation network topolocies or ultimately even the complete target deployment network topology. When letting an LLM develop software, it will make more mistakes than an experienced programmer. But it can much easier also be automated to run its (broken) software - in the simulation environment - and then troubleshoot and fix the broken software. Device and Network simulation will typically consist of virtual machine based variants of the actual target network device software, such that a complete network of those devices can be run on a single (large scale) compute unit, which is then driven by the LLM that performs the programming.

Agent Programmable Network Device The following sections summarize core aspects considered to be required or beneficial for the system design of network devices intended to be enhanced with automation applications - which ultimately should become also agent programmed. Effectively it discusses design aspects and requirements of the components shown to be part of the network device in .

Agent execution environment This section discusses what type of environment is applicable/useful and should potentially be standardized to support tunning (Agent programmed) network automation programs and/or network automation programs that by themselves include agentic aspects such as leveraging on-platform inference to be autonomously agentic.

Hypervisor Execution environments for virtual machines (VM) are called Hypervisors. While they are widely used in network to create/run routers as virtual machines themselves, the likely create too much overhead solely for adding network automation programs - unless such network automation itself is intended as a separate additional execution environment independent of pre-existing virtual-machine/container based routers that should be automated. User and hence agent programmable software running on a network device itself can take many form. Evolving from rather (by todays standards) constrained hardware and software environments, initial programming options where built utilizing lightweight (small code size) scripting languages like Tcl, and later type of products python. Only when the control plane of network devices was built on top of more general-purpose operating systems like linux was it easily possible to add more flexible environments for third-party programmable software, for example through linux infrastructures like KVM for virtual machines or container environments.

Containers The likely best execution environment for automation software are containers. Ongoing industry efforts such as the Open Container Initiative (OCI) are also driving standardization for this approach. It allows any type of application as long as it can run on the OS (and its API) used on the router platform. This option may not be possible to make available on router platforms with monolythic, non-linux like operating systems though.

Interpreted Interpreted automation software is a good fallback option for such older, monolithic operating system based router platforms. Such an approach will limit though what type of automation can easily be done, because it for example will need to be implemented in one of few (if noy only one) supported interpreted language. Python is a widely accepted programming language which seems likely a good candidate. On router platforms which can not support containerized third-part provided automation software, support for the below detailled aspects of the ANI may likely also be difficult.

Agent to device APIs To operate for device and network automation, the agent exeuction environment need to provide - ideally standardized - interfaces for the following functionality: Use the CLI of the router up to and including highest privilege level. Access the file system of the router read/modify/write/delete. Ideally without having to go through the CLI or in general none or as little as possible actual router software - to ensure that this access is as little, or not at all impacted by misbehavior of the router software. Connect to any theoretically network accessible responder socket of the router to utilize its functionality. Wether this is a dedicated management interface such as SSH or netconf sockets or simply network protocol responder sockets that the automation software would want to test. Direct access to any diagnostics (hardware/software) interfaces of the router, for example any RS232/USB or ethrernet based "console" diagnostic port.

ANI/containers and router software manageabiliy The "Autonomic Management Infrastructure" (ANI), as specified and exemplified by - (and extended by later RFCs) provides the core infrastructure to allow zero-touch trusted remote access to network routers/switches that support it, even passing management traffic automatically and securely across multiple ANI capable routers that are otherwise completely unconfigured. Likewise, automation agent software on routers themselves are capable to securely talk with each other and any management plane processes on the router that make their API available via network sockets reachable from the ANI (ACP). Software management of the router such as router software upgrade/downgrade and reboot should be possible across the ANI from remote locations as well as from automation agents locally.

Basic Linux example To put the above abstracted requirements into a practical example with details of commonly required problem solutions: The router uses Linux. Before the introduction of the ANI, software management of the router is like that of a simple linux system: rebooting the router is performed by rebooting the bare-metal linux used for the router software, or "reboot the PC". With the introduction of the ANI and management agent execution environments, the low-level infrastructure is changes such that all the router software runs inside one manageable execution environment, such as a container. TO minimize the changes needed, this container still should perform as much as possible of additional hardware (re-)initialization as possible. The ANI itself runs "natively" on the bare-metal linux of the device. It consists of a very small number of processes (e.g.: BRSKI, RPL routing process, GRASP, and on few "headend" routers functions such as Registrar or CA). Likewise running "natively" on the bare-metal linux must be the necessary container platform software. It must be accessible from the ANI - and can thus be remotely controlled across the ANI. Automation agents run in their own or automation-shared execution environments (e.g.: containers). As part of physical boostrap, a device would have one main automation agent which is first started from the container management. This automation agent could then perform basic system tasks such as validating the on-disk software and if considered acceptable, it would instruct the container managemeent to start the actual router software. Then additional automation tasks would be started depending on the needs/configuration. Instead of physcially rebooting the full physcial device software, upgrade or bug induced reboot of the router software should simply be a restart of the router software - including to restart it with a different set of firmware/config-files. And because the ANI allows ongoing network connectivity via the reboot of the router software itself, even remote management can fully observe all shutdown/reboot messages otherwise only observable (most often) via local interfaces such as consoles. Even Ethernet consoles do not provide an automated way to remotely access them but rely on a co-located "server" system for network management.

Router control and management plane For the pre-existing router software which constitutes the control plane and (legacy) management plane to be "controlled" by new automation agent applications, it is beneficial to have some "internal" virtual network connectivity. That should typically be easy to add through appropriate linux kernel constructs when using linux as the OS kernel mechanism. See also .

Data Plane In this (version of) document, there is no ask to support user programmable forwarding plane to extend/improve/automate the network functionality. This is because the high speed at relatively low power consumption of high-speed networking equipment is today achieved mostly by very specialized hardware called Network Processing Units (NPU). Making this user extensible would introduce significant additional complexity and also introduce a whole other scope of functionality.

eBPF Extensible Berkely Packet Filters (eBPF) is a method to download compiled scripting language developed programs into the linux network kernel which can then process network packets. BPF started out as a mechanism for linux process level tools like tcpdump and later wireshark to promiscuously receive network packets and filter the undesired ones efficiently inside the linux kernel and only pass the ones of interest to user land. BPF was later extended (eBPF) to allow forwarding, modifying and creating packets, making it now the preferred high performance experimental tool for not only specific packet processing in process level applications that want to receive and send uncommon pcket header, but also for forwarder/router functions. eBPF also brings in its current implementation a good degree of protecting the system against malicious or mal-behaving eBPF scripts and can thus safely be operationalized for third-party eBFP programs. This document asks for support of eBPF in support of automation; primarily in support of diagnostics, data gathering and active testing, such as generation, reception and measurements of active testing (for example TWAMP). Likewise, new automation protocols not using well established transport stacks (UDP/TCP/QUIC) may use eBPF as an appropriate way to support such new/different transport protol processing inside the kernel for new user-level automation protocols. In no case should eBPF be used to process user traffic for solely the goals for AI assisted network automation. If any existing or planned protocol implementations for user traffic already is or plans to use eBPF, then this is outside the scope of this document. For example non UDP/TCP protocols like RSVP may today be more easily implemented on linux based router operating systems with eBPF.

Inband management traffic challenges Adding the aforementioned management infrastructure and specifically the ANI to router hardware and software design involved more than the potentially necessary refactoring of how the router (control plane) software can be run so that it can be fully managed, including restarting/upgrading/downgrading. The main challenge for the hardware is to allow multiplexing of the management plane traffic so that it can be sent/received by ANI and automation plane applications without relying on the router software stack - or relying on it as little as possible. In the standardized method does not provide a good low-level isolation but instead an multiplexing method that is (theoretically) easy to implement everywhere: The ANI simply relies on a single IPv6 host stack also used by the user plane, but only use link-local addresses - so that the ANI can operate before any routing is configured or working. Relying on the IPv6 host stack of the router itself does require for the host stack to be operational. A method of multiplexing ANI packets at a lower level would hence be preferrable. On typical server PC hardware for example the Baseboard Management Controller hardware typically relieas on an internal ethrernet switch for every physical interface and switches received packets to either the BMC or the CPU based on destination MAC address. In other words, on ethernet the system software and the BMC software are two separate host-stacks multiplexed by layer 2 hardware switch. This type of multiplexing may equally be possible to perform in advanced router hardware through appropriate programming, but this functionality is the most complex issue to resolve for most reliable separation of router software (and its potential issues) and the agentic management plane. On the linux level, it is relatively straightforward to separate traffic from/for different MAC addresses to different container/container-groups through the use of kernel-level modules such as MACVLAN.

Acknowledgements

Security Considerations TBD.

Changelog

draft-eckert-anima-ai4an-00 Initial version

This document specifies the GeneRic Autonomic Signaling Protocol (GRASP), which enables autonomic nodes and Autonomic Service Agents to dynamically discover peers, to synchronize state with each other, and to negotiate parameter settings with each other. GRASP depends on an external security environment that is described elsewhere. The technical objectives and parameters for specific application scenarios are to be described in separate documents. Appendices briefly discuss requirements for the protocol and existing protocols with comparable features. Autonomic functions need a control plane to communicate, which depends on some addressing and routing. This Autonomic Control Plane should ideally be self-managing and be as independent as possible of configuration. This document defines such a plane and calls it the "Autonomic Control Plane", with the primary use as a control plane for autonomic functions. It also serves as a "virtual out-of-band channel" for Operations, Administration, and Management (OAM) communications over a network that provides automatically configured, hop-by-hop authenticated and encrypted communications via automatically configured IPv6 even when the network is not configured or is misconfigured. This document specifies automated bootstrapping of an Autonomic Control Plane. To do this, a Secure Key Infrastructure is bootstrapped. This is done using manufacturer-installed X.509 certificates, in combination with a manufacturer's authorizing service, both online and offline. We call this process the Bootstrapping Remote Secure Key Infrastructure (BRSKI) protocol. Bootstrapping a new device can occur when using a routable address and a cloud service, only link-local connectivity, or limited/disconnected networks. Support for deployment models with less stringent security requirements is included. Bootstrapping is complete when the cryptographic identity of the new key infrastructure is successfully deployed to the device. The established secure connection can be used to deploy a locally issued certificate to the device as well. Intent and Intent-Based Networking are taking the industry by storm. At the same time, terms related to Intent-Based Networking are often used loosely and inconsistently, in many cases overlapping and confused with other concepts such as "policy." This document clarifies the concept of "intent" and provides an overview of the functionality that is associated with it. The goal is to contribute towards a common and shared understanding of terms, concepts, and functionality that can be used as the foundation to guide further definition of associated research and engineering problems and their solutions. This document is a product of the IRTF Network Management Research Group (NMRG). It reflects the consensus of the research group, having received many detailed and positive reviews by research group participants. It is published for informational purposes. This document proposes guidelines for the design of Autonomic Service Agents for autonomic networks. Autonomic Service Agents, together with the Autonomic Network Infrastructure, the Autonomic Control Plane, and the GeneRic Autonomic Signaling Protocol, constitute base elements of an autonomic networking ecosystem.

Other LLM cases for networks The following sections discuss aspects of the possible use or non-use of LLMs that are deemed to be non-core for the document right now but can help illustrate further aspects of interest.

LLM in Data Centers (DC) Well designed Data Centers (DC) typically do not suffer from the complexity described in . Instead of carrying all user-traffic (so-called data plane), control-plane and management traffic across he same links and nodes (so called in-band control-plane and manament plane), the local nature of a DC allows to easily use so-called out-of-band management. Each network device is connected to management control stateions via specific management ports and using network switches solely for such management traffic. Any configuration performed on the actual user-traffic network equipment does not impact any management traffic. If any automation system incurs mistakes in its operation, it can always undo them because its own traffic is never affected. Control Plane protocols that run through the actual user-traffic network paths is kept at a minimum to further increase the resilience of such Data Center designs. Management links can easily support high speed, allowing for high-data rates of observed data collection to support more intelligent decision making from network management systems. In such DC environments, the use of LLM directly on user-traffic network equipment is not really needed, because the equipment can always and better be conrolled via this network management out-of-band network from managemenet systems in the DC - with LLM whenever beneficial. Note though that this type of design is not ubiquitous whereever it is physically feasible because beside the local, in-building nature of the network in question, the cost factor of such an additional out-of-band network is another key factor. For example, industrial networks inside a factory/plant do not have such out-of-band networks because of various factors: Traditionally, there was no need for agile behavior of networks because all aspects of network operators where fully planned upfront, and secondly, the longer-range wiring as well as often environmentally challenging setup of network equipment made it financially inappropriate to install a separate out-of-band nework.

LLM in control plane The decentralized "intelligence" that make network automatically support changes in connected users/device as well as changes including failure and recovery of components of the network infrastructure itself - is typically called the control plane of the network. It goes back to the days of the ARPANET with distributed routing protocols at its core. Since those days though, the actual degree of in-network automation has mostly stagnated, and often the only evolution is an ever richer set of policies that need to be network management controlled. Common routing protocols such as OSPF and ISIS can not automatically determine how to administer address aggregation methods such areas to better scale routing tables. Or auto-configure virtual links (OSPF) where neeeded. Multicast protocols like PIM require network operations decisions for very basic functionality such as determining the best location for specific protocol functions such as "Rendezvous Points". BGP itself is purely a complex set of prioritized policy rules to provide managed interdomain connectivity. Most other protocols in networks can not auto-configure their own security and hence rely on network operations to do this. All these protocols operate in-band on top of network connectivity that only works when they themselves operate perfectly. This all results in a highly fragile nature of todays core network control plane infrastructure as well as the forwarding plane functionality it has to use for its own traffic forwarding. In result of the awareness into this fragile foundation of networks control plane, network operations experts have a high degree of reservations against directly introducting heuristic behavior such as that from LLMs into this layer of the the network infrastructure.

Security Inference Security inspection of packets starts to use such LLM, relying on trained instead of programmed "Deep Packet Inspection" (DPI), but struggles with false positives due to the heuristic nature of the LLM. However, the larger the models become that these inference accelerators can support, the more easily they simply transform into a very accuratecy pattern matching engine with a high degree of determinism. Complex and costly programming as traditionally used for DPI is replaced with training and validation of correct behavior is part of that training.