Menu Close
2.10.4.3. Other infrastructure components
- A VPC
- DNS entries
- Load balancers (classic or network) and listeners
- A public and a private Route 53 zone
- Security groups
- IAM roles
- S3 buckets
If you are working in a disconnected environment, you are unable to reach the public IP addresses for EC2 and ELB endpoints. To resolve this, you must create a VPC endpoint and attach it to the subnet that the clusters are using. The endpoints should be named as follows:
-
ec2.<region>.amazonaws.com -
elasticloadbalancing.<region>.amazonaws.com -
s3.<region>.amazonaws.com
Required VPC components
You must provide a suitable VPC and subnets that allow communication to your machines.
| Component | AWS type | Description | |
|---|---|---|---|
| VPC |
| You must provide a public VPC for the cluster to use. The VPC uses an endpoint that references the route tables for each subnet to improve communication with the registry that is hosted in S3. | |
| Public subnets |
| Your VPC must have public subnets for between 1 and 3 availability zones and associate them with appropriate Ingress rules. | |
| Internet gateway |
| You must have a public Internet gateway, with public routes, attached to the VPC. In the provided templates, each public subnet has a NAT gateway with an EIP address. These NAT gateways allow cluster resources, like private subnet instances, to reach the Internet and are not required for some restricted network or proxy scenarios. | |
| Network access control |
| You must allow the VPC to access the following ports: | |
| Port | Reason | ||
|
| Inbound HTTP traffic | ||
|
| Inbound HTTPS traffic | ||
|
| Inbound SSH traffic | ||
|
| Inbound ephemeral traffic | ||
|
| Outbound ephemeral traffic | ||
| Private subnets |
| Your VPC can have private subnets. The provided CloudFormation templates can create private subnets for between 1 and 3 availability zones. If you use private subnets, you must provide appropriate routes and tables for them. | |
Required DNS and load balancing components
Your DNS and load balancer configuration needs to use a public hosted zone and can use a private hosted zone similar to the one that the installation program uses if it provisions the cluster’s infrastructure. You must create a DNS entry that resolves to your load balancer. An entry for api.<cluster_name>.<domain> must point to the external load balancer, and an entry for api-int.<cluster_name>.<domain> must point to the internal load balancer.
The cluster also requires load balancers and listeners for port 6443, which are required for the Kubernetes API and its extensions, and port 22623, which are required for the Ignition config files for new machines. The targets will be the master nodes. Port 6443 must be accessible to both clients external to the cluster and nodes within the cluster. Port 22623 must be accessible to nodes within the cluster.
| Component | AWS type | Description |
|---|---|---|
| DNS |
| The hosted zone for your internal DNS. |
| etcd record sets |
| The registration records for etcd for your control plane machines. |
| Public load balancer |
| The load balancer for your public subnets. |
| External API server record |
| Alias records for the external API server. |
| External listener |
| A listener on port 6443 for the external load balancer. |
| External target group |
| The target group for the external load balancer. |
| Private load balancer |
| The load balancer for your private subnets. |
| Internal API server record |
| Alias records for the internal API server. |
| Internal listener |
| A listener on port 22623 for the internal load balancer. |
| Internal target group |
| The target group for the internal load balancer. |
| Internal listener |
| A listener on port 6443 for the internal load balancer. |
| Internal target group |
| The target group for the internal load balancer. |
Security groups
The control plane and worker machines require access to the following ports:
| Group | Type | IP Protocol | Port range |
|---|---|---|---|
|
|
|
|
|
|
|
| ||
|
|
| ||
|
|
| ||
|
|
|
|
|
|
|
| ||
|
|
|
|
|
|
|
|
Control plane Ingress
The control plane machines require the following Ingress groups. Each Ingress group is a AWS::EC2::SecurityGroupIngress resource.
| Ingress group | Description | IP protocol | Port range |
|---|---|---|---|
|
| etcd |
|
|
|
| Vxlan packets |
|
|
|
| Vxlan packets |
|
|
|
| Internal cluster communication and Kubernetes proxy metrics |
|
|
|
| Internal cluster communication |
|
|
|
| Kubernetes kubelet, scheduler and controller manager |
|
|
|
| Kubernetes kubelet, scheduler and controller manager |
|
|
|
| Kubernetes Ingress services |
|
|
|
| Kubernetes Ingress services |
|
|
Worker Ingress
The worker machines require the following Ingress groups. Each Ingress group is a AWS::EC2::SecurityGroupIngress resource.
| Ingress group | Description | IP protocol | Port range |
|---|---|---|---|
|
| Vxlan packets |
|
|
|
| Vxlan packets |
|
|
|
| Internal cluster communication |
|
|
|
| Internal cluster communication |
|
|
|
| Kubernetes kubelet, scheduler, and controller manager |
|
|
|
| Kubernetes kubelet, scheduler, and controller manager |
|
|
|
| Kubernetes Ingress services |
|
|
|
| Kubernetes Ingress services |
|
|
Roles and instance profiles
You must grant the machines permissions in AWS. The provided CloudFormation templates grant the machines permission the following AWS::IAM::Role objects and provide a AWS::IAM::InstanceProfile for each set of roles. If you do not use the templates, you can grant the machines the following broad permissions or the following individual permissions.
| Role | Effect | Action | Resource |
|---|---|---|---|
| Master |
|
|
|
|
|
|
| |
|
|
|
| |
|
|
|
| |
| Worker |
|
|
|
| Bootstrap |
|
|
|
|
|
|
| |
|
|
|
|