High memory usage by etcd in OpenShift 3.x

Issue

• High memory usage by etcd
• After increasing the memory on etcd nodes from 16GB -> 32GB, memory still being consumed quickly
• Large number of Event resources are being generated in the cluster
• Etcd cluster is unstable

[root@example ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:          32013       30610         238         345        1164         796
Swap:          4095        1474        2621
[root@example ~]# systemctl restart etcd
[root@example ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:          32013       11235       18500         334        2278       20185
Swap:          4095        1455        2640

• Restart of etcd freed up 18GB of Mem, while about half an hour later it is back to:

[root@example ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:          32013       25502        1607         369        4903        5833
Swap:          4095        1452        2643