Repository Structure — GNUnet documentation

# Repository Structure

## `src` Overview

This section gives a brief overview of the GNUnet source code. Specifically, we sketch the function of each of the subdirectories in the `gnunet/src/` directory. The order given is roughly bottom-up (in terms of the layers of the system).

Todo

Reorganise this list

`util/` --- libgnunetutil

  Library with general utility functions, all GNUnet binaries link against this library. Anything from memory allocation and data structures to cryptography and inter-process communication. The goal is to provide an OS-independent interface and more 'secure' or convenient implementations of commonly used primitives. The API is spread over more than a dozen headers, developers should study those closely to avoid duplicating existing functions. see libgnunetutil.

`hello/` --- libgnunethello

  HELLO messages are used to describe under which addresses a peer can be reached (for example, protocol, IP, port). This library manages parsing and generating of HELLO messages.

`block/` --- libgnunetblock

  The DHT and other components of GNUnet store information in units called 'blocks'. Each block has a type and the type defines a particular format and how that binary format is to be linked to a hash code (the key for the DHT and for databases). The block library is a wrapper around block plugins which provide the necessary functions for each block type.

`statistics/` --- statistics service

  The statistics service enables associating values (of type uint64_t) with a component name and a string. The main uses is debugging (counting events), performance tracking and user entertainment (what did my peer do today?).

`arm/` --- Automatic Restart Manager (ARM)

  The automatic-restart-manager (ARM) service is the GNUnet master service. Its role is to start gnunet-services, to re-start them when they crashed and finally to shut down the system when requested.

`peerinfo/` --- peerinfo service

  The peerinfo service keeps track of which peers are known to the local peer and also tracks the validated addresses for each peer (in the form of a HELLO message) for each of those peers. The peer is not necessarily connected to all peers known to the peerinfo service. Peerinfo provides persistent storage for peer identities --- peers are not forgotten just because of a system restart.

`datacache/` --- libgnunetdatacache

  The datacache library provides (temporary) block storage for the DHT. Existing plugins can store blocks in Sqlite, Postgres or MySQL databases. All data stored in the cache is lost when the peer is stopped or restarted (datacache uses temporary tables).

`datastore/` --- datastore service

  The datastore service stores file-sharing blocks in databases for extended periods of time. In contrast to the datacache, data is not lost when peers restart. However, quota restrictions may still cause old, expired or low-priority data to be eventually discarded. Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.

`template/` --- service template

  Template for writing a new service. Does nothing.

`ats/` --- Automatic Transport Selection

  The automatic transport selection (ATS) service is responsible for deciding which address (i.e. which transport plugin) should be used for communication with other peers, and at what bandwidth.

`nat/` --- libgnunetnat

  Library that provides basic functions for NAT traversal. The library supports NAT traversal with manual hole-punching by the user, UPnP and ICMP-based autonomous NAT traversal. The library also includes an API for testing if the current configuration works and the `gnunet-nat-server` which provides an external service to test the local configuration.

`fragmentation/` --- libgnunetfragmentation

  Some transports (UDP and WLAN, mostly) have restrictions on the maximum transfer unit (MTU) for packets. The fragmentation library can be used to break larger packets into chunks of at most 1k and transmit the resulting fragments reliably (with acknowledgment, retransmission, timeouts, etc.).

`transport/` --- transport service

  The transport service is responsible for managing the basic P2P communication. It uses plugins to support P2P communication over TCP, UDP, HTTP, HTTPS and other protocols. The transport service validates peer addresses, enforces bandwidth restrictions, limits the total number of connections and enforces connectivity restrictions (e.g. friends-only).

`peerinfo-tool/` --- gnunet-peerinfo

  This directory contains the gnunet-peerinfo binary which can be used to inspect the peers and HELLOs known to the peerinfo service.

`core/`

  The core service is responsible for establishing encrypted, authenticated connections with other peers, encrypting and decrypting messages and forwarding messages to higher-level services that are interested in them.

`testing/` --- libgnunettesting

  The testing library allows starting (and stopping) peers for writing testcases. It also supports automatic generation of configurations for peers ensuring that the ports and paths are disjoint. libgnunettesting is also the foundation for the testbed service

`testbed/` --- testbed service

  The testbed service is used for creating small or large scale deployments of GNUnet peers for evaluation of protocols. It facilitates peer deployments on multiple hosts (for example, in a cluster) and establishing various network topologies (both underlay and overlay).

`nse/` --- Network Size Estimation

  The network size estimation (NSE) service implements a protocol for (securely) estimating the current size of the P2P network.

`dht/` --- distributed hash table

  The distributed hash table (DHT) service provides a distributed implementation of a hash table to store blocks under hash keys in the P2P network.

`hostlist/` --- hostlist service

  The hostlist service allows learning about other peers in the network by downloading HELLO messages from an HTTP server, can be configured to run such an HTTP server and also implements a P2P protocol to advertise and automatically learn about other peers that offer a public hostlist server.

`topology/` --- topology service

  The topology service is responsible for maintaining the mesh topology. It tries to maintain connections to friends (depending on the configuration) and also tries to ensure that the peer has a decent number of active connections at all times. If necessary, new connections are added. All peers should run the topology service, otherwise they may end up not being connected to any other peer (unless some other service ensures that core establishes the required connections). The topology service also tells the transport service which connections are permitted (for friend-to-friend networking)

`fs/` --- file-sharing

  The file-sharing (FS) service implements GNUnet's file-sharing application. Both anonymous file-sharing (using gap) and non-anonymous file-sharing (using dht) are supported.

`cadet/` --- cadet service

  The CADET service provides a general-purpose routing abstraction to create end-to-end encrypted tunnels in mesh networks. We wrote a paper documenting key aspects of the design.

`tun/` --- libgnunettun

  Library for building IPv4, IPv6 packets and creating checksums for UDP, TCP and ICMP packets. The header defines C structs for common Internet packet formats and in particular structs for interacting with TUN (virtual network) interfaces.

`mysql/` --- libgnunetmysql

  Library for creating and executing prepared MySQL statements and to manage the connection to the MySQL database. Essentially a lightweight wrapper for the interaction between GNUnet components and libmysqlclient.

`dns/`

  Service that allows intercepting and modifying DNS requests of the local machine. Currently used for IPv4-IPv6 protocol translation (DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The service can also be configured to offer an exit service for DNS traffic.

`vpn/` --- VPN service

  The virtual public network (VPN) service provides a virtual tunnel interface (VTUN) for IP routing over GNUnet. Needs some other peers to run an "exit" service to work. Can be activated using the "gnunet-vpn" tool or integrated with DNS using the "pt" daemon.

`exit/`

  Daemon to allow traffic from the VPN to exit this peer to the Internet or to specific IP-based services of the local peer. Currently, an exit service can only be restricted to IPv4 or IPv6, not to specific ports and or IP address ranges. If this is not acceptable, additional firewall rules must be added manually. exit currently only works for normal UDP, TCP and ICMP traffic; DNS queries need to leave the system via a DNS service.

`pt/`

  protocol translation daemon. This daemon enables 4-to-6, 6-to-4, 4-over-6 or 6-over-4 transitions for the local system. It essentially uses "DNS" to intercept DNS replies and then maps results to those offered by the VPN, which then sends them using mesh to some daemon offering an appropriate exit service.

`identity/`

  Management of egos (alter egos) of a user; identities are essentially named ECC private keys and used for zones in the GNU name system and for namespaces in file-sharing, but might find other uses later

`revocation/`

  Key revocation service, can be used to revoke the private key of an identity if it has been compromised

`namecache/`

  Cache for resolution results for the GNU name system; data is encrypted and can be shared among users, loss of the data should ideally only result in a performance degradation (persistence not required)

`namestore/`

  Database for the GNU name system with per-user private information, persistence required

`gns/`

  GNU name system, a GNU approach to DNS and PKI.

`dv/`

  A plugin for distance-vector (DV)-based routing. DV consists of a service and a transport plugin to provide peers with the illusion of a direct P2P connection for connections that use multiple (typically up to 3) hops in the actual underlay network.

`regex/`

  Service for the (distributed) evaluation of regular expressions.

`scalarproduct/`

  The scalar product service offers an API to perform a secure multiparty computation which calculates a scalar product between two peers without exposing the private input vectors of the peers to each other.

`consensus/`

  The consensus service will allow a set of peers to agree on a set of values via a distributed set union computation.

`reclaim/`

  A decentralized personal data sharing service used to realize a decentralized identity provider. Supports OpenID Connect. See also https://reclaim.gnunet.org.

`rest/`

  The rest API allows access to GNUnet services using RESTful interaction. The services provide plugins that can exposed by the rest server.