Omninet

Omninet is an experimental universal virtual networking fabric. It connects nodes, clusters, and external clients into isolated, VPC-like networks with the core networking features you need.

It uses a WireGuard-based backnet for encrypted connectivity between nodes, and a VXLAN + Open vSwitch (OVS) overlay for flexible multi-tenant networking.

Warning

This project is experimental and not ready for production use cases.

Note

Some parts of the code were initially generated using AI to move faster from a PoC to a working version. Most of the code has since been reviewed and refactored.

Why this project?

When you run workloads across multiple machines, you need to connect them. Cloud providers give you VPCs with a few clicks. On your own hardware, you wire together WireGuard, bridges, iptables, and DHCP yourself.

Most open source tools solve one problem. Tailscale connects devices. Calico and Cilium handle Kubernetes. But if you run VMs, containers, and services outside Kubernetes, you still have to glue things together. You also miss basic things like floating IP management, public IP assignment, and traffic shaping.

Omninet tries to be that glue. One control plane for isolated networks, firewalls, bandwidth limits, and floating IP management. Nodes behind NAT with no static public IP can join the network with a few commands and work like any other node. Connect VMs and containers from Docker, Podman, containerd, or anything else to the same VPC. Route traffic between clusters. Give external clients secure VPN access to your network.

The heavy lifting is handled by Omninet. Encryption, routing, IP allocation, NAT. A plugin system handles the last mile. Built-in plugins cover Docker, Podman, Containerd CNI and libvirt as of now. You can write your own plugins later to connect anything else without touching the core.

The goal is simple. Omninet provides the building blocks. You build what you need on top of it.

Core Components

• Manager: Central gRPC authority for node registration, VPC configuration, firewall, traffic shaping, and public IP assignment
• Gateway: Cluster hub for WireGuard routing. Runs a Rust userspace WireGuard (boringtun) with bandwidth-aware P2P direct peering
• Node: Worker machine running the VPC overlay stack, DHCP, metadata, and plugin integrations
• Internet Gateway: Dedicated node for 1:1 public IP NAT (DNAT/SNAT via iptables)

Key Features

• Backnet: WireGuard mesh (10.0.0.0/8) with hub-and-spoke topology and cluster segmentation
  • Any node with internet connectivity can join, even behind NAT
  • Gateway tracks bandwidth and automatically establishes direct P2P WireGuard tunnels between high-traffic node pairs
  • Gateway-to-gateway mesh for cross-cluster routing
  • Offline resilience: nodes restore last-known-good config if manager is unreachable
• VPC Overlay: VXLAN-based isolation with user-defined CIDRs and 5-table OVS OpenFlow pipeline (anti-spoofing, firewall, L2 forwarding)
• Firewall: Per-interface stateful ACLs with conntrack, deny-by-default, up to 200 rules (TCP/UDP/ICMP, CIDR, port)
• Traffic Shaping: TC/HTB-based bandwidth limits per interface with separate private (intra-VPC) and public (internet) rate classes
• Public IPs: Internet gateways register IP pools; 1:1 NAT maps public IPs to private VPC addresses
• Metadata Service: EC2-compatible endpoint at 169.254.169.254 (instance-id, local-ipv4, vpc-id, etc.)
• Integrations: Ideally it should work with vm, container, linux machine. Curent Integrations -
  • Docker Plugin: Native Docker network + IPAM driver on /run/docker/plugins/omninet.sock
  • libvirt Integration: VPC-backed networks for VMs with QEMU hook auto-wiring, per-VM DHCP, and multi-IP support

LocalAPI (node-local Unix socket)

The node agent exposes a gRPC server on /var/run/omninet-node.sock (mode 0660) — the LocalAPI. It is the only entry point that workload runtime plugins (CNI, Docker, libvirt) and omninetctl node ... use to talk to the agent. The contract lives in pkg/proto/omninet.proto under the NodeLocal service and the implementation is internal/agent/node_local_api.go.

Auth model: file-system permissions. The socket is created with mode 0660, so only root and members of the group that owns /var/run/omninet-node.sock (typically root) can connect. There is no JWT on this path — possession of the socket is the credential. Plugins that run unprivileged should be added to that group.

Methods (see the proto for full signatures):

Verb	Purpose
JoinVPC / LeaveVPC	Bring up / tear down the OVS bridge + VXLAN port for a VPC on this node
RequestSubnet	Lease a sub-CIDR from the manager and bind it to this node
ReleaseSubnet	Return a previously requested subnet
ListOwnedSubnets	Subnets currently leased to this node
GetVPCInfo / ListVPCInfo	VPC metadata (VNI, CIDR, gateway IP)
CreateInterface	Allocate an IP (or use the supplied one) and create a workload-attached link
DeleteInterface	Tear down an interface and release its IP
ListInterfaces	Enumerate interfaces on a VPC

Tasks

• Core Features
• Strengthen JWT authentication
• Add a rescue and recovery path in case machine become unaccesible due to misconfiguration
• Track interface bandwidth usage
• Add a Prometheus exporter
• Add WireGuard gateway failover support
• Add external VPN access support
• Build Python and Go clients
• Build a basic topology dashboard
• More integrations
  • Podman
  • Containerd (CNI)
• Add PostgreSQL / MySQL support for the manager
• Integrate golang-migrate + atlas (CI also) for schema migrations
• Auto-TLS Support
• Cleanup zombie interfaces (docker might have force removed container)
• For natted private nodes, collect prometheus metrics
• Move all node management and token generation commands over API though omninetctl
• On node agent, if we delete shared gateway or bridge that can cause trouble

License

MIT