Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for mounting Unix sockets #235

Closed
jmillikin-stripe opened this issue May 24, 2019 · 7 comments
Closed

Add support for mounting Unix sockets #235

jmillikin-stripe opened this issue May 24, 2019 · 7 comments
Labels
area: filesystem Issue related to filesystem area: networking Issue related to networking priority: p2 Normal priority type: enhancement New feature or request

Comments

@jmillikin-stripe
Copy link
Contributor

We want to run a gVisor sandbox without network access (--network=none), and implement restricted access to a few hardened upstreams by mounting in Unix sockets. This pattern works in Docker because the sockets and container exist in the same kernel, but gVisor's filesystem sidecar doesn't seem to support mounting Unix sockets.

#33 (comment) suggests this was intentional, for security. Would it be possible to relax that restriction, either by default or with a CLI flag?

Alternatively, is there any recommended way to add a "network gopher" that could run outside the sandbox and proxy TCP traffic?

@iangudger
Copy link
Contributor

I think @prattmic was looking into this recently.

@prattmic
Copy link
Member

Relaxing this restriction is something that we've considered recently, at least for connecting to existing UDS (as opposed to binding new sockets). Would only being able to connect work for your use-case?

Also note that it wouldn't be possible to send in-sandbox file descriptors over such a UDS to a process outside the sandbox, though I doubt you are trying to do that.

@adunham-stripe
Copy link
Contributor

I think that @jmillikin-stripe may not be near a laptop this week, so I'll jump in here and expand a bit: @jmillikin-stripe, please keep me honest!

I'm pretty sure that we'd only be using the "connect to existing UDS" features, and we're not interested in passing file descriptors over it. The main use-case here is being able to proxy and filter traffic leaving a container in a useful manner; one thing that we've done with regular Docker is to run the container with --network none, and send all traffic via a CONNECT proxy on a Unix socket mounted into the container. This lets us allow/denylist domains to connect to via a single interface (listening on the Unix socket on the host), and make much stronger isolation guarantees (i.e. "no Docker networking") compared to e.g. trying to wire something up with iptables.

@prattmic
Copy link
Member

Thanks for the details, I think that is a very reasonable use case. It will need a bit more investigation, but I think it is something we should be able to allow, especially if it is something that one of you want to look into hooking up.

We do also have all the pieces to allow a custom network gofer (netgofer), but no API for it at the moment. And for this kind of fairly simple use case that would likely be overkill vs a simple UDS.

@prattmic prattmic added area: filesystem Issue related to filesystem area: networking Issue related to networking type: enhancement New feature or request priority: p2 Normal priority labels May 29, 2019
@adunham-stripe
Copy link
Contributor

especially if it is something that one of you want to look into hooking up.

I'm interested in looking into this, but I'll admit to being unfamiliar with gVisor's codebase. I found this chunk of code, which does not check for .IsSocket(), but I'm not sure if I'm barking up the wrong tree here. I'm happy to dig further, but before I do that: any suggestions/pointers?

@prattmic
Copy link
Member

I'm happy to help figure out our codebase. :) BTW, we've just recently set up a Gitter channel if you want to join for more real-time questions.

Connecting to a socket generally goes through the connect(2) syscall, not open(2), which will ultimately become a Tlconnect 9p RPC [1].

The good news is that the sentry (sandbox kernel) side already supports connecting to UDS through a gofer. It's just not supported by the gofer used in runsc.

In the sentry both the fs and networking paths are fairly complicated and UDS combines both, so for posterity, here is the flow of connecting to a gofer-backed UDS:

linux.Connect (connect(2) syscall handler) -> unix.SocketOperations.Connect -> transport.connectionedEndpoint.Connect -> gofer.endpoint.BidirectionalConnect -> e.file.Connect which is just a Tlconnect 9p call.

I don't think you should need to change any of the sentry code. On the gofer side, we'll call p9.File.Connect, which is where we need an implementation.

We have pretty extensive UDS tests in our syscall tests. I'll have to think about the best way to hook up these gofer-backed sockets to those tests.

Hopefully this should help you get a prototype working, and we can see where to go from there (should this be behind a flag, etc).

[1] N.B. Tlconnect is a custom extension we've added to 9p. It's not standard: https://github.com/google/gvisor/blob/master/pkg/p9/version.go#L126

@prattmic
Copy link
Member

Note that this feature still has some issues:

  • Connecting to the wrong kind of socket may crash the gofer.
  • Unsafe use of unet.Socket that may cause the garbage collector to close the socket FD.

I'll be sending out fixes for these (and more complete tests) soon.

@prattmic prattmic changed the title Please add support for mounting Unix sockets Add support for mounting Unix sockets Oct 10, 2019
gvisor-bot pushed a commit that referenced this issue Oct 11, 2019
fsgofer.attachPoint.Attach has a bunch of funky special logic to create a RW
file or connect a socket rather than creating a standard control file like
localFile.Walk.

This is unecessary and error-prone, as the attach point still has to go through
Open or Connect which will properly convert the control file to something
usable. As such, switch the logic to be equivalent to a simple Walk.

Updates #235

PiperOrigin-RevId: 274235091
gvisor-bot pushed a commit that referenced this issue Oct 11, 2019
This change fixes several issues with the fsgofer host UDS support. Notably, it
adds support for SOCK_SEQPACKET and SOCK_DGRAM sockets [1]. It also fixes
unsafe use of unet.Socket, which could cause a panic if Socket.FD is called
when err != nil, and calls to Socket.FD with nothing to prevent the garbage
collector from destroying and closing the socket.

A set of tests is added to exercise host UDS access. This required extracting
most of the syscall test runner into a library that can be used by custom
tests.

Updates #235
Updates #1003

[1] N.B. SOCK_DGRAM sockets are likely not particularly useful, as a server can
only reply to a client that binds first. We don't allow bind, so these are
unlikely to be used.

PiperOrigin-RevId: 272500970
gvisor-bot pushed a commit that referenced this issue Oct 15, 2019
fsgofer.attachPoint.Attach has a bunch of funky special logic to create a RW
file or connect a socket rather than creating a standard control file like
localFile.Walk.

This is unecessary and error-prone, as the attach point still has to go through
Open or Connect which will properly convert the control file to something
usable. As such, switch the logic to be equivalent to a simple Walk.

Updates #235

PiperOrigin-RevId: 274827872
gvisor-bot pushed a commit that referenced this issue Oct 18, 2019
This change fixes several issues with the fsgofer host UDS support. Notably, it
adds support for SOCK_SEQPACKET and SOCK_DGRAM sockets [1]. It also fixes
unsafe use of unet.Socket, which could cause a panic if Socket.FD is called
when err != nil, and calls to Socket.FD with nothing to prevent the garbage
collector from destroying and closing the socket.

A set of tests is added to exercise host UDS access. This required extracting
most of the syscall test runner into a library that can be used by custom
tests.

Updates #235
Updates #1003

[1] N.B. SOCK_DGRAM sockets are likely not particularly useful, as a server can
only reply to a client that binds first. We don't allow bind, so these are
unlikely to be used.

PiperOrigin-RevId: 275558502
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: filesystem Issue related to filesystem area: networking Issue related to networking priority: p2 Normal priority type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants