Running one or more Urbit ships in containers on a host behind a NAT

by Rudd-O published 2021/04/22 22:58:00 GMT+0, last modified 2021-04-22T23:26:58+00:00
These instructions will ensure you get direct network connectivity instead of indirect connections, leading to faster Urbit.

Caveat: this guide speaks only to Urbit's native networking port — this does not address the HTTP port you would normally load on your Web browser to see the UI.  That requires a bit more configuration, not covered in this guide.

If your host running your Urbit ships has a public IP address, you don't need the iptables rules in the next section of this guide  However, you still need to allocate a specific UDP port for each Urbit.  We'll assume you'll allocate them starting from port 30000 up to port 40000.  You don't need it, if your host running the docker containers has a public IP address.

The Docker manual says something to this effect.

-p 8080:80/udp Map UDP port 80 in the container to port 8080 on the Docker host.

In fact, that's all you need, if your machine has no firewall rejecting INPUT traffic.  Set a specific port in your Urbit container's run command line.  urbit or urbit-king will take a parameter for the port, which is -p 30000.  Then when you start the container, the command line parameters for Urbit must include that.  In addition to setting that command line parameter, you must also run the container with the Docker parameter -p 30000:30000/udpnot as parameters to Urbit, but as parameters to the Docker command, near where you add the mounts and other Docker options.

Repeat, incrementing the port, for each new ship you start in a Docker container.

Making this work behind a NAT

Caveat: this will not work if you are behind a carrier-grade NAT / double-NAT.  This will only work if your router actually has a public IP address.

If your machine running Docker containers is NOT exposed to the Internet — rather, it uses the Internet via NAT on a router, the typical IP connectivity situation for people running machines at home — then you most likely need to add port forwarding on your router machine.

The mechanics are simple: outside people coming to your Urbits need to have their traffic directly shunted to the machine running the Urbits, and Urbits connecting to outside people need their connections tracked using regular NAT.  On Linux routers, this would encompass adding rules that do DNAT in the PREROUTING chain of the nat table, encompassing all of the ports you know that your machine is exposing via Docker, and also a MASQUERADE rule in the POSTROUTING chain of the nat table, to ensure responses to traffic originating from Urbits have a return path.

To continue with our example, if all your Urbits on your Docker machine 192.168.1.1 are using the port range 30000:40000 (from 30000 to 40000 inclusive), you would add the following iptables rules (or their equivalent for the equipment you are using as router):

# First rule: ensure that traffic destined to your Docker machine is in fact routed to it.
iptables -t filter -A FORWARD -d 192.168.1.1 -p udp -m udp --dport 30000:40000 \
-m comment --comment "urbit: from internet yes" \
-j ACCEPT

# Second rule: ensure that traffic coming from your Docker machine is not subject to NAT.
# The purpose of this rule is to make this traffic skip the next rule.
iptables -t nat -A PREROUTING -s 192.166.1.1 -p udp -m udp --dport 30000:40000 \
-m comment --comment "urbit: do not DNAT connections to Urbit ports coming from Urbit" \
-j ACCEPT

# Third rule: ensure that traffic coming to your router is DNATted to your Docker machine.
# The purpose of this rule is to shunt traffic from the outside world to your Docker machine.
iptables -t nat -A PREROUTING -p udp -m udp --dport 30000:40000 \
-m comment --comment "urbit: DNAT connections to Urbit port coming from the internet to Urbit" \
-j DNAT --to-destination 192.168.1.1

# Fourth rule (usually the default in NAT setups): ensure traffic coming from your Docker machine is routed to the outside world.
iptables -t nat -A POSTROUTING -s 192.168.1.1 \
-j MASQUERADE

That's all.  Obviously these rules will need to be adjusted to your scenario — in particular because rule order matters — but these rules are the gist of what you need.

Testing correct DNAT UDP rule operation

You'll need three things:

  1. The machine where you will run your Urbits.  This will be 192.168.1.1 in this example.
  2. A machine outside of your local area network, and by that, I mean outside of your NAT zone.  An Amazon EC2 instance is a good example of that.
  3. A computer program, the code of which I will provide now.  You must have Python 3 to run it.

Here is the server program.  Save this to a file udpserver.py on your 192.168.1.1 machine.

# import module
import socket, sys

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# IP and Port for receving connection
client_address = ('0.0.0.0', int(sys.argv[1]))

# print address and port
print ("[+] Client IP {} | Port {}".format(client_address[0], client_address[1]))

# bind socket with server
sock.bind(client_address)

# Create Loop
while True:
     # Wait for a connection
     print('[+] Waiting for a client connection')
     # connection established
     data, client_address = sock.recvfrom(4096)
     print(data)

OK, now you can run it locally on your machine with the command line python3 udpserver 30000 (the sample shows what you type italicized).  This should be its output:

[user@controller ~]$ python3 udpserver.py 30000
[+] Client IP 0.0.0.0 | Port 30000
[+] Waiting for a client connection

In another terminal window, use the nc client to test it (this is usually installed on every Linux machine):

[user@controller ~]$ echo hello | nc -u localhost 30000
[user@controller ~]$

In the first terminal window, you should see the new output:

b'hello\n'
[+] Waiting for a client connection

This indicates that your machine is actively listening on port 30000 (UDP).

Now it's time to take this roadshow to your external machine.  Keep the udpserver.py program running, and log onto your external machine:

[user@controller ~]$ ssh user@5.6.7.8
[user@aws-34-us-east ~]$ whoami
user
[user@aws-34-us-east ~]$

Great!  Now, you will run the same nc command, but this time you won't connect to localhost, and you also won't connect to 192.168.1.1 — you will connect to the IP of the router where you added your iptables rules.  www.whatsmyip.org is helpful to find that one out.  We will assume the address your router has is 9.10.11.12:

[user@aws-34-us-east ~]$ echo world | nc -u 9.10.11.12 30000
[user@aws-34-us-east ~]$

At this point, you should see this appear on the first terminal window:

b'world\n'
[+] Waiting for a client connection

If this appears, that means you have UDP DNAT port forwarding on your router set up correctly.