Puppet fails with Error 500 on SERVER: Server Error: Failed when searching for node client.example.com

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6.x
  • Red Hat Enterprise Linux

Issue

  • # puppet agent -t fails with:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Failed when searching for node client.example.com: Failed to find client.example.com via exec: Execution of '/etc/puppetlabs/puppet/node.rb client.example.com' returned 1

Resolution

  • Check the entries for ssl_ca, ssl_cert, ssl_key in /etc/puppetlabs/puppet/foreman.yaml on the satellite server. The file should have entries as:
---
:url: "https://satellite.example.com"
:ssl_ca: "/etc/pki/katello/puppet/puppet_client_ca.crt"
:ssl_cert: "/etc/pki/katello/puppet/puppet_client.crt"
:ssl_key: "/etc/pki/katello/puppet/puppet_client.key"
:user: ""
:password: ""
:puppetdir: "/opt/puppetlabs/server/data/puppetserver"
:puppetuser: "puppet"
:facts: true
:timeout: 60
:report_timeout: 60
:threads: null

  • Restart the katello service after making correct entries:
    # katello-service restart

  • Regenerate puppet certs and sign them:
    On the client run # puppet agent -tv.

  • The issue can also be due to incorrect Host Group Parameter or Host Parameters set on the client.
    Login to satellite web ui > Hosts > All Hosts > search and click on the client host > Edit > Parameters tab:
    Delete the Host Group Parameter or Host Parameters set.

For more KB articles/solutions related to Red Hat Satellite 6.x Puppet Issues, please refer to the Red Hat Satellite Consolidated Troubleshooting Article for Red Hat Satellite 6.x Puppet Issues

Root Cause

  • /etc/puppetlabs/puppet/foreman.yaml had incorrect entries for ssl_ca, ssl_cert, ssl_key.
  • Incorrect Host Group Parameter or Host Parameters set on the client.

Diagnostic Steps

  • # puppet agent -t on client fails with:
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: Error 500 on SERVER: Server Error: Failed to find client.example.com via exec: Execution of '/etc/puppetlabs/puppet/node.rb client.example.com' returned 1: 
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Failed when searching for node client.example.com: Failed to find client.example.com via exec: Execution of '/etc/puppetlabs/puppet/node.rb client.example.com' returned 1: 
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

2 Comments

Log in to comment
Fulvio Carrus's picture Red Hat Active Contributor 361 points
30 January 2020 2:26 PM

The same error can happen if a host parameter has invalid yaml value. In that case, the ENC for that host cannot be compiled, and the Puppet agent cannot retrieve it. A quick check can be done by visiting the host's ENC page at https://satellite.example.com/api/hosts/host.example.com/enc

To find out which parameter is not valid, check out production.log on the master Satellite system, it should point out the incorrect value.

SE Newbie 5 points
25 January 2023 3:20 PM

The error can also be seen when there are a large number of facts on the client. Increasing the timeout values within /etc/puppetlabs/puppet/foreman.yaml on the respective capsule resolved our issue