Why there are out-bound ipv4 traffic on ports 443, 9997 to S3 buckets in ROSA?

Solution In Progress - Updated -

Environment

  • Red Hat OpenShift On AWS(ROSA)

Issue

  • Why there are out-bound ipv4 traffic on ports 443, 9997 to s3 buckets (ROSA).
  • I can see some traffic to s3 bucket from ROSA cluster through port 443 and 9997. What are they used for?
  • Our security team has found an issue where an outbound traffic is being initiated by/through a public subnets. The public subnet should only be answering to new sessions and not initiating them. We found some 443, 9997 traffic to s3 buckets and others. What are these traffic ?

Resolution

Tcp 443 could be anything on below document list for http protocol, tcp port 9997 is used for cluster audit logs (allowlist 5, item 4 in doc).

Please refer to document for a list of expected outbound traffic.
* osd-aws-privatelink-firewall-prerequisites

*.osdsecuritylogs.splunkcloud.com

Used by the splunk-forwarder-operator as a logging forwarding endpoint to be used by the SRE Platform Team for log-based alerting.
  • The only s3 bucket outside of the account that should be used is for install logs when the cluster is first provisioned and used bastion host in AWS.

For 9997 - it's ec2 traffic. During install, s3 traffic outside of the account is expected. After installation, there will be regular s3 traffic for backups but it should remain inside the account (It can be confirm with bucket name with below command) :

 $oc -n openshift-velero get veleroinstalls/cluster -o custom-columns=BUCKET:status.storageBucket.name

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments