Vector shows "error=connection closed before message completed" warnings

Solution Unverified - Updated -

Issue

  • Vector Pods are showing the following HTTP error. error=connection closed before message completed warnings in their logs:

    $ oc logs collector-2nzx4
    [..]
    2024-01-17T15:09:15.991612Z WARN sink{component_kind="sink" component_id=example_splunk_receiver component_type=splunk_hec_logs component_name=example_splunk_receiver}: vector::internal_events::http_client: HTTP error. error=connection closed before message completed error_type="request_failed" stage="processing" internal_log_rate_limit=true
    2024-01-17T15:09:15.991676Z WARN sink{component_kind="sink" component_id=example_splunk_receiver component_type=splunk_hec_logs component_name=example_splunk_receiver}: vector::sinks::util::retries: Retrying after error. error=Failed to make HTTP(S) request: connection closed before message completed internal_log_rate_limit=true
    2024-01-17T15:09:53.249900Z WARN sink{component_kind="sink" component_id=example_splunk_receiver component_type=splunk_hec_logs component_name=example_splunk_receiver}: vector::internal_events::http_client: HTTP error. error=connection closed before message completed error_type="request_failed" stage="processing" internal_log_rate_limit=true
    [..]
    
  • The CollectorHighErrorRate alert is firing as a result of these messages.

Environment

  • Red Hat OpenShift Container Platform (OCP) 4.x
  • Red Hat OpenShift Logging 5.8

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content