Alertmanager does not use the proxy environment variables

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP) 4.

Issue

  • While using a global HTTP/HTTPS proxy it is not possible to send alerts to Alertmananger from a Webhook Receiver accessed via Internet.
  • The sidecar proxy container from Alertmanager can access the proxy, but the Alertmanager container cannot access it.
  • Ideally, the proxy variables should be added to the container with values configured in the cluster-wide Proxy config resource.
  • How to configure a proxy in alertmanager.yaml as the alerts do not seem to be honoring the clusterwide proxy settings?

Resolution

DISCLAIMER
Links contained herein to external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.

global:
  resolve_timeout: 5m
  http_config:
    proxy_url: 'http://x.x.x.x:8080'

Diagnostic Steps

After having configured a cluster-wide proxy, please try to access your endpoint from the Alertmanager container. It works only from the sidecar proxy container from Alertmanager.

Only the Alertmanager proxy sidecar container has the proxy environment variables. You can reach your endpoint from that container but not from the Alertmanager container itself.

To get the current alertmanager.yaml configuration please use the following command:

oc get -n openshift-monitoring secret alertmanager-main --template='{{ index .data "alertmanager.yaml" }}' | base64 --decode > alertmanager.yaml

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments