Containers can't access published ports of other containers on the same host

Issue

Containers have published ports and they are available both from the host and outside (assuming that firewalld is set properly). However when one of containers tries to access the other using host:port, then it hangs.