Networking Guide

Edit this page Create issue

Applications running in gVisor often communicate with the outside world. gVisor networking is architected to enforce a strong isolation boundary without restricting application behavior. To that end, gVisor implements its own network stack called netstack.

This document describes how packets move to and from gVisor, the architecture of netstack, and how netstack can be used independently as a userspace network stack.

How packets get to and from gVisor

Whether running directly via runsc or indirectly through Docker, packets flow between gVisor and the host in largely the same manner.

Networking

The gVisor sandbox process (called the sentry) is started in a network namespace with one or more virtual network devices. As with Docker, there is typically one loopback device and one VETH device present. gVisor scrapes addresses, routes, and the like from those devices and configures the sentry to use those same addresses and routes. Thus applications in gVisor accept and generate packets as though they were running on the host while still maintaining the strong sandbox boundary.

The sentry, which for security cannot open host sockets of its own, is initialized with a single AF_PACKET socket. AF_PACKET sockets send and receive raw packets, i.e. those that include link, network, and transport headers. gVisor ingresses and egresses all non-loopback traffic across that socket.

Netstack architecture

Threading in netstack is fairly simple. Link endpoints, the most common of which is the fdbased endpoint, spawn their own goroutine(s) that receive incoming packets and pass them up netstack. TCP packets are enqueued and asynchronously handled by the TCP implementation’s own goroutines. Other protocols are handled inline, and the dispatcher goroutine handles all processing up to enqueueing packets at the socket where it can be read into userspace.

Outgoing packets can be processed on different goroutines – syscall, TCP, or the link endpoint’s – until typically reaching a queueing discipline. There another goroutine writes batches of queued packets out the link endpoint.

Netstack supports a variety of underlying link layers. Currently supported link layers include AF_PACKET sockets, AF_XDP sockets, shared memory, and Go channels.

Netstack aims to be usable independent of gVisor. As a fully-featured userspace network stack, it can be (and is) easily reused in other projects. Note that, while netstack’s API is fairly stable, it doesn’t guarantee stability and is not published with Go module-style versions.

Host networking

gVisor can also be run with host networking via the --network=host flag. This uses the hostinet package, which trades the security and isolation of netstack for the performance of native Linux networking.