DNS server in docker behind LVS direct routing ?

As we're struggeling with the small packet performance of virtual DNS-servers in RHEV, I'm wondering if running them in docker containers would be a better option.

So, today we have 2x LVS routers routing DNS to 2 "real servers". The LVS router is configured as:

-A -u x.y.z.4:53 -s wrr
-a -u x.y.z.4:53 -r x.y.z.5:53 -g -w 1
-a -u x.y.z.4:53 -r x.y.z.6:53 -g -w 1

and the real-servers (x.y.z.5 and x.y.z.6) has the service address x.y.z.4 configured on the loopback interface. This works fine (except for performance issues under high load).

Now I would like to do the same with docker for the real servers. Does anybody have any suggestion for how to configure the network in the docker containers to achieve this? Will NAT from host scale to 10s of thousands connections per second?

I believe as a minimum I will need the service address bound to the loopback interface in the container, so I now naively tried doing "RUN ip address add 1.2.3.4/32 dev lo" from a Dockerfile, but got "RTNETLINK answers: Operation not permitted". Get the same from within a running container, unless I run it with "--privileged=true".