Issue

I am able to:
  Connect to the hypervisor with virt-manager without problem. 
  Open a console with virt-viewer directly to guest with no problem.

What I cannot do is:
  Open a guest console inside virt-manager.  It errors with

Error: viewer connection to hypervisor host got refused or disconnected!

The command execution was simply:

virt-manager -c qemu+tls://virthost/system

   and selecting the guest from the remote connection in the interface.

I've also done this below with the same result:

virt-manager -c qemu+tls://virthost/system --show-domain-console=$(virsh domuuid virtguest)