[Satellite6] Running ansible role on hosts with large amount of facts fails with "500 Server Error: Internal Server Error"

Solution Verified - Updated -

Issue

Running ansible role fails with "500 Server Error: Internal Server Error"
Ansible runs takes a lot of time to process


2020-05-26T14:34:15 [D|app|71f77069] Body: {"notifications":[{"id":2969,"seen":false,"level":"success","text":"A job 'Run Ansible roles' has finished successfully","created_at":"2020-05-26T12:16:35.891Z","group":"Jobs","actions":{"links":[{"href":"/job_invocations/1779","title":"Job Details"}]}},{"id":2968,"seen":false,"level":"success","text":"A job 'Run Ansible roles' has finished successfully","created_at":"2020-05-26T11:55:32.914Z","group":"Jobs","actions":{"links": [{"href":"/job_invocations/1732","title":"Job Details"}]}}]} 2020-05-26T14:34:15 [I|app|71f77069] Completed 200 OK in 14ms (Views: 0.2ms | ActiveRecord: 1.1ms) 2020-05-26T14:34:15 [W|app|0a57ae8d] Some subtrees exceeded 100 limit of facts, dropped 487 keys 2020-05-26T14:34:25 [I|app|0a57ae8d] Import facts for 'roletest1' completed. Added: 0, Updated: 21, Deleted 0 facts 2020-05-26T14:34:25 [D|tax|0a57ae8d] Current location set to Default Location 2020-05-26T14:34:25 [D|tax|0a57ae8d] Current organization set to Default Organization 2020-05-26T14:34:25 [W|app|0a57ae8d] Action failed 2020-05-26T14:34:25 [D|app|0a57ae8d] Backtrace for 'Action failed' error (NoMethodError): undefined method `-' for nil:NilClass /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-4.0.3.5/app/services/foreman_ansible/fact_parser.rb:48:in `get_interfaces' /usr/share/foreman/app/services/fact_parser.rb:69:in `interfaces' /usr/share/foreman/app/services/fact_parser.rb:84:in `suggested_primary_interface' /usr/share/foreman/app/models/host/base.rb:186:in `primary_interface_type' /usr/share/foreman/app/models/host/base.rb:192:in `populate_fields_from_facts' /usr/share/foreman/app/models/host/managed.rb:416:in `populate_fields_from_facts' /usr/share/foreman/app/models/concerns/puppet_host_extensions.rb:3:in `populate_fields_from_facts' /usr/share/foreman/app/models/host/base.rb:168:in `block in parse_facts' /usr/share/foreman/app/services/foreman/telemetry_helper.rb:27:in `telemetry_duration_histogram' /usr/share/foreman/app/models/host/base.rb:167:in `parse_facts' /usr/share/foreman/app/models/host/base.rb:153:in `import_facts' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_default_hostgroup-5.0.0/lib/default_hostgroup_base_host_patch.rb:13:in `import_facts' /usr/share/foreman/app/models/host/managed.rb:330:in `import_facts' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_default_hostgroup-5.0.0/lib/default_hostgroup_base_host_patch.rb:6:in `import_facts' /usr/share/foreman/app/controllers/api/v2/hosts_controller.rb:308:in `facts' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks' /usr/share/foreman/app/controllers/api/v2/base_controller.rb:163:in `disable_json_root' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/mapper.rb:48:in `serve' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.20/lib/katello/middleware/event_daemon.rb:10:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/static_dispatcher.rb:65:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/extractor/recorder.rb:137:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/middleware/checksum_in_headers.rb:27:in `call' /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/tempfile_reaper.rb:15:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/etag.rb:25:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/conditional_get.rb:38:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/head.rb:12:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call' /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:232:in `context' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:226:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call' /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.0.20/lib/katello/prevent_json_parsing.rb:12:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/method_override.rb:22:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/runtime.rb:22:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/sendfile.rb:111:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.0.0/lib/secure_headers/middleware.rb:13:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:68:in `block in call' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `each' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `call' /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/rack/thread_handler_extension.rb:77:in `process_request' /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler/thread_handler.rb:140:in `accept_and_process_next_request' /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler/thread_handler.rb:108:in `main_loop' /usr/share/gems/gems/passenger-4.0.18/lib/phusion_passenger/request_handler.rb:441:in `block (3 levels) in start_threads' /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context' 2020-05-26T14:34:25 [I|app|0a57ae8d] Rendering api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout 2020-05-26T14:34:25 [I|app|0a57ae8d] Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (1.9ms) 2020-05-26T14:34:25 [I|app|0a57ae8d] Completed 500 Internal Server Error in 10621ms (Views: 4.9ms | ActiveRecord: 2284.4ms) 2020-05-26T14:34:25 [I|app|5a61008e] Started GET "/notification_recipients" for 10.89.140.76 at 2020-05-26 14:34:25 +0200 2020-05-26T14:34:25 [I|app|5a61008e] Processing by NotificationRecipientsController#index as JSON 2020-05-26T14:34:25 [D|tax|5a61008e] Current location set to none 2020-05-26T14:34:25 [D|tax|5a61008e] Current organization set to Default Organization 2020-05-26T14:34:25 [D|not|5a61008e] Cache Hit: notification, reading cache for notification-4

Environment

Satellite6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content