DNS server in docker behind LVS direct routing ?

Latest response

As we're struggeling with the small packet performance of virtual DNS-servers in RHEV, I'm wondering if running them in docker containers would be a better option.

So, today we have 2x LVS routers routing DNS to 2 "real servers". The LVS router is configured as:

-A -u x.y.z.4:53 -s wrr
-a -u x.y.z.4:53 -r x.y.z.5:53 -g -w 1
-a -u x.y.z.4:53 -r x.y.z.6:53 -g -w 1

and the real-servers (x.y.z.5 and x.y.z.6) has the service address x.y.z.4 configured on the loopback interface. This works fine (except for performance issues under high load).

Now I would like to do the same with docker for the real servers. Does anybody have any suggestion for how to configure the network in the docker containers to achieve this? Will NAT from host scale to 10s of thousands connections per second?

I believe as a minimum I will need the service address bound to the loopback interface in the container, so I now naively tried doing "RUN ip address add 1.2.3.4/32 dev lo" from a Dockerfile, but got "RTNETLINK answers: Operation not permitted". Get the same from within a running container, unless I run it with "--privileged=true".

ja
Log in to join the conversation

Responses

ja Guru 1340 points
25 June 2014 10:43 AM

Just to clearify.. We still want to run LVS on separate servers, loadbalancing incoming requests over docker containers on separate hosts:

LVS-server1 -> docker-host1 -> docker-container1
LVS-server1 -> docker-host2 -> docker-container2
LVS-server1 -> docker-host3 -> docker-container3
etc..

pixdrift ‏‏‎ ‎'s picture Guru 6603 points
30 June 2014 7:33 AM

Jan-Frode,

To clarify.

You want to run a DNS service in docker containers.

Each physical host (3 in the above example) will be receiving DNS traffic on their external interface (x.y.z.5, x.y.z.6)

You want the DNS traffic that hits the (x.y.z.5/6) hosts forwarded to the DNS services running in the docker container hosted on that host.

So you want to expose 53 from the docker container, and forward any traffic that hits the host on 53 to that exposed port in the docker container?

Or have I missed something major?

ja Guru 1340 points
30 June 2014 10:49 PM

You're missing what I didn't say (sorry :-) The physical hosts should be running several DNS containers (because of different DNS filters, and logging rules for different countries we have different sets of dns-servers), so we can't just forward port 53 from host to container directly. But forwarding all traffic destined for x.y.z.4:53 to a given container should work I guess.

But I'm a bit uncertain if it will be OK to NAT 10-20.000 queries/s to the container, or if it will be better to give it an externally routable address over a bridge.. (and IPv6 is probably also easier over a bridge). I haven't quite understood docker networking properly yet.., but I believe exposing containers with external addresses should be an option.

ja Guru 1340 points
4 July 2014 12:24 PM

I'm slowing seeing a solution I think... Run the various flavors for DNS containers NAT'ed from different ports on the host, and tell LVS about this. I.e.

For DNS-server-set-1, public address x.y.z.4:53:

LVS config:

-A -u x.y.z.4:53 -s wrr
-a -u x.y.z.4:53 -r x.y.z.5:53 -g -w 1
-a -u x.y.z.4:53 -r x.y.z.6:53 -g -w 1

Containers running with command:

docker run -p 53:53 -p 53:53/udp janfrode/unbound

Set-2, x.y.z.7:53:

LVS config:

-A -u x.y.z.7:53 -s wrr
-a -u x.y.z.7:53 -r x.y.z.5:54 -g -w 1
-a -u x.y.z.7:53 -r x.y.z.6:54 -g -w 1

Containers running with command:

docker run -p 54:53 -p 54:53/udp janfrode/unbound

I will try this and report back if it works or not.

ja Guru 1340 points
4 July 2014 1:39 PM

Current status: https://github.com/janfrode/docker-unbound/blob/master/LVS.md

David Powles's picture Red Hat Guru 25394 points
30 June 2014 5:53 AM

Hi Jan-Frode. Could be a pretty complex question, but I've reached out to some folks here who might be able to offer some suggestions.

rh Red Hat Community Member 50 points
30 June 2014 7:50 AM

Hi Jan-Frode,

You can expose port 53 on the container running on docker-host to receive traffic.

However if you do want to assign ip address 1.2.3.4/32 to the interface "lo" without running the container in privileged mode, here is how you can do it.

NOTE : All commands below to be run on the docker host.

1) Run the container. Learn the containers PID

sudo docker inspect -f '{{.State.Pid}}'

e.g.

$sudo docker inspect -f '{{.State.Pid}}' 63f36fc01b5f

9634

2) create its namespace entry in /var/run/netns/
for the "ip netns" command

$ pid=9634
$ sudo mkdir -p /var/run/netns
$ sudo ln -s /proc/$pid/ns/net /var/run/netns/$pid

3) Add the IP address to the device (in this case "lo")

$ sudo ip netns exec $pid ip addr add 1.2.3.4/32 dev lo

4) Run "ip addr" command within the container and you should see the device "lo" having the above IP address.

-Vivek

ja Guru 1340 points
4 July 2014 12:15 PM

What about adding loopback address from Dockerfile, is that possible? If I do "RUN ip address add 1.2.3.4/32 dev lo" it fails to build:

Step 9 : RUN ip address add 1.2.3.4/32 dev lo
---> Running in 8f66e34d2a71
RTNETLINK answers: Operation not permitted
2014/07/04 14:12:57 The command [/bin/sh -c ip address add 1.2.3.4/32 dev lo] returned a non-zero code: 2

So is there an equivivalente to --privileged=true we can use for "docker build" ?