Smart Card configure for optional use

I have a Red Hat 7 system and have successfully set it up to us Smart Card, so when I use PuttyCac I get prompted for my PIN and I get access. The problem is that I have users that don't have a SmartCard and can't log in. No one the user with a SmartCard or not cannot use SCP to connect to the system.

I need to figure out what I set and somehow make the use of SmartCards as optional.