pam_listfile does not restrict password less ssh authentication

Issue

pam_listfile does not restrict users with Key-based authentication (password-less)

Pam Entry(/etc/pam.d/sshd) : auth required pam_listfile.so item=user sense=allow file=/etc/sshd_userlist onerr=succeed