How to integrate docker registry with NFS

Resolution

We need to make sure if the steps for configuring NFS server and client are correctly followed to be integrated with Openshift docker registry.

Export an NFS Volume

• Ensure that nfs-utils is installed on the system :

   yum install nfs-utils

• Then as root create the directory that will be exported :

mkdir -p /var/export/regvol
chown nfsnobody:nfsnobody /var/export/regvol
chmod 777 /var/export/regvol

• Edit /etc/exports and add the following line:

/var/export/regvol *(rw,sync,all_squash)

• Enable and start NFS services:

systemctl enable rpcbind nfs-server
systemctl start rpcbind nfs-server nfs-lock 
systemctl start nfs-idmap

Note : the volume is owned by nfsnobody and access by all remote users is "squashed" to be access by this user. This essentially disables user permissions for clients mounting the volume. While another configuration might be preferable, one problem you may run into is that the container cannot modify the permissions of the actual volume directory when mounted.

NFS Firewall

We will need to open ports on the firewall on the master to enable the nodes to communicate with us over NFS. First, let's add rules for NFS to the running state of the firewall.

• On the master as root:

iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 111 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2049 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 20048 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 50825 -j ACCEPT
iptables -I OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 53248 -j ACCEPT

• Next, let's add the rules to /etc/sysconfig/iptables. Put them at the top of the OS_FIREWALL_ALLOW set:

-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 53248 -j ACCEPT
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 50825 -j ACCEPT
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 20048 -j ACCEPT
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 2049 -j ACCEPT
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 111 -j ACCEPT

• Now, we have to edit NFS' configuration to use these ports. First, let's edit /etc/sysconfig/nfs. Change the RPC option to the following:

RPCMOUNTDOPTS="-p 20048"

• Change the STATD option to the following:

STATDARG="-p 50825"

• Then, edit /etc/sysctl.conf:

fs.nfs.nlm_tcpport=53248
fs.nfs.nlm_udpport=53248

• Then, persist the sysctl changes:

sysctl -p

• Lastly, restart NFS:

systemctl restart nfs

Allow NFS Access in SELinux Policy

By default policy, containers are not allowed to write to NFS mounted directories. We want to do just that with our database, so enable that on all nodes where the pod could land (i.e. all of them) with:

setsebool -P virt_use_nfs=true

Attaching Registry Storage

• One can refer the Documentation for how to use and define persistent storage for registry