Floating IP not pinging Externally
I have successfully deployed everything in Redhat Openstack 11 with following settings. I was not able to ping the floating IP externally rather i can perform ping, ssh and other things using namespace.
I have three controllers and two hypercoverged Compute.
VLAN for RHOSP 11 Setup
172.26.11.0/24 - Provision Network ( VLAN2611 )
172.26.12.0/24 - Internal Network ( VLAN2612 )
172.26.13.0/24 - Tentant Network ( VLAN2613 )
172.26.14.0/24 - Storage Network ( VLAN2614 )
172.26.16.0/24 - Storage Managment ( VLAN2616 )
172.26.17.0/24 - Management Network ( VLAN2617 )
172.30.10.0/23 - External Network ( VLAN3010 )
Server Setup:
[stack@director ~]$ nova list
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| 3e37a6ed-1b0a-49de-9aa8-5515949ad11a | overcloud-compute-0 | ACTIVE | - | Running | ctlplane=172.26.11.13 |
| 3bab2815-1df8-4b1a-ab70-fa1d00dd5889 | overcloud-compute-1 | ACTIVE | - | Running | ctlplane=172.26.11.25 |
| 531cc5ad-ceb2-40c4-9662-1a984eea1907 | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=172.26.11.12 |
| 598cb725-ed9d-4e7f-b8d1-3d5ac0df86d8 | overcloud-controller-1 | ACTIVE | - | Running | ctlplane=172.26.11.23 |
| a92cbacd-301e-4201-aa74-b100eb245345 | overcloud-controller-2 | ACTIVE | - | Running | ctlplane=172.26.11.28 |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
Controller-0 IP's Assigned:
All other two controllers will have the same IP address configuration.
[stack@director ~]$ ssh heat-admin@172.26.11.12
Last login: Wed Feb 14 09:23:13 2018 from 172.26.11.254
[heat-admin@overcloud-controller-0 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether c8:1f:66:e1:1a:c3 brd ff:ff:ff:ff:ff:ff
inet 172.26.11.12/24 brd 172.26.11.255 scope global em1
valid_lft forever preferred_lft forever
inet 172.26.11.22/32 brd 172.26.11.255 scope global em1
valid_lft forever preferred_lft forever
inet6 fe80::ca1f:66ff:fee1:1ac3/64 scope link
valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000
link/ether c8:1f:66:e1:1a:c4 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ca1f:66ff:fee1:1ac4/64 scope link
valid_lft forever preferred_lft forever
4: em3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000
link/ether c8:1f:66:e1:1a:c5 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ca1f:66ff:fee1:1ac5/64 scope link
valid_lft forever preferred_lft forever
5: em4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether c8:1f:66:e1:1a:c6 brd ff:ff:ff:ff:ff:ff
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether c6:05:34:74:27:e0 brd ff:ff:ff:ff:ff:ff
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether c8:1f:66:e1:1a:c4 brd ff:ff:ff:ff:ff:ff
inet6 fe80::800e:f6ff:fe6d:245/64 scope link
valid_lft forever preferred_lft forever
8: vlan2612: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 9a:12:3a:34:7a:7c brd ff:ff:ff:ff:ff:ff
inet 172.26.12.12/24 brd 172.26.12.255 scope global vlan2612
valid_lft forever preferred_lft forever
inet 172.26.12.18/32 brd 172.26.12.255 scope global vlan2612
valid_lft forever preferred_lft forever
inet6 fe80::9812:3aff:fe34:7a7c/64 scope link
valid_lft forever preferred_lft forever
9: vlan2613: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:2d:8b:7b:f1:21 brd ff:ff:ff:ff:ff:ff
inet 172.26.13.20/24 brd 172.26.13.255 scope global vlan2613
valid_lft forever preferred_lft forever
inet6 fe80::f82d:8bff:fe7b:f121/64 scope link
valid_lft forever preferred_lft forever
10: vlan2614: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether c2:ea:76:13:4e:16 brd ff:ff:ff:ff:ff:ff
inet 172.26.14.18/24 brd 172.26.14.255 scope global vlan2614
valid_lft forever preferred_lft forever
inet6 fe80::c0ea:76ff:fe13:4e16/64 scope link
valid_lft forever preferred_lft forever
11: vlan2616: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 82:e6:64:04:d7:23 brd ff:ff:ff:ff:ff:ff
inet 172.26.16.12/24 brd 172.26.16.255 scope global vlan2616
valid_lft forever preferred_lft forever
inet6 fe80::80e6:64ff:fe04:d723/64 scope link
valid_lft forever preferred_lft forever
12: vlan2617: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether d2:74:4f:18:b5:3c brd ff:ff:ff:ff:ff:ff
inet 172.26.17.14/24 brd 172.26.17.255 scope global vlan2617
valid_lft forever preferred_lft forever
inet6 fe80::d074:4fff:fe18:b53c/64 scope link
valid_lft forever preferred_lft forever
13: vlan3010: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 32:e2:86:b9:d2:3e brd ff:ff:ff:ff:ff:ff
inet 172.30.10.21/23 brd 172.30.11.255 scope global vlan3010
valid_lft forever preferred_lft forever
inet6 fe80::30e2:86ff:feb9:d23e/64 scope link
valid_lft forever preferred_lft forever
14: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether f2:7e:78:3c:ee:49 brd ff:ff:ff:ff:ff:ff
15: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether a2:4d:a0:64:3a:4e brd ff:ff:ff:ff:ff:ff
16: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
link/gre 0.0.0.0 brd 0.0.0.0
17: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
18: gre_sys@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65490 qdisc pfifo_fast master ovs-system state UNKNOWN qlen 1000
link/ether f6:71:95:be:da:53 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f471:95ff:febe:da53/64 scope link
valid_lft forever preferred_lft forever
Controller-0 OVS Bridge :
qg is external interface of SDN router
qr is internal interface of SDN router
These interfaces are directly created inside the br-int. In older versions of RHOSP. There is no patch between the br-int and br-ex. So the qg will be created directly in br-ex. In this version, we find that both interfaces are created inside the br-int, if i change the external bridge as br-int in all L3 agents, then the router interfaces shows down. Even-though all the communication of ping and ssh happens inside the qrouter namespaces itself.
[heat-admin@overcloud-controller-0 ~]$ sudo ovs-vsctl show
f6411a64-6dbd-4a7d-931a-6a99b63d7911
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qg-0f094325-6c"
tag: 10
Interface "qg-0f094325-6c"
type: internal
Port "qr-fff1e03e-44"
tag: 8
Interface "qr-fff1e03e-44"
type: internal
Port "tapef7874a7-a3"
tag: 8
Interface "tapef7874a7-a3"
type: internal
Port "ha-a3430c62-90"
tag: 4095
Interface "ha-a3430c62-90"
type: internal
Port "ha-37bad2be-92"
tag: 9
Interface "ha-37bad2be-92"
type: internal
Port "tap102385e5-b7"
tag: 4
Interface "tap102385e5-b7"
type: internal
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "gre-ac1a0d0f"
Interface "gre-ac1a0d0f"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.15"}
Port "gre-ac1a0d10"
Interface "gre-ac1a0d10"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.16"}
Port "gre-ac1a0d16"
Interface "gre-ac1a0d16"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.22"}
Port br-tun
Interface br-tun
type: internal
Port "gre-ac1a0d0c"
Interface "gre-ac1a0d0c"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.12"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "vlan2617"
tag: 2617
Interface "vlan2617"
type: internal
Port "vlan2612"
tag: 2612
Interface "vlan2612"
type: internal
Port "vlan2613"
tag: 2613
Interface "vlan2613"
type: internal
Port br-ex
Interface br-ex
type: internal
Port "vlan3010"
tag: 3010
Interface "vlan3010"
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port "vlan2614"
tag: 2614
Interface "vlan2614"
type: internal
Port "vlan2616"
tag: 2616
Interface "vlan2616"
type: internal
Port "bond1"
Interface "em2"
Interface "em3"
ovs_version: "2.6.1"
Neutron Agent List
[heat-admin@overcloud-controller-0 ~]$ neutron agent-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
| 08afba9b-1952-4c43-a3ec- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| 1b6a1cf49370 | | controller-1.localdomain | | | | |
| 1c7794b0-726c-4d70-81bc- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| df761ad105bd | | controller-1.localdomain | | | | |
| 23aba452-ecb2-4d61-96b5-f8224c | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| 6de482 | | controller-0.localdomain | | | | |
| 2acabaa4-cad1-4e25-b102-fe5f72 | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 0de5b8 | | controller-2.localdomain | | | | |
| 38074c45-565c-45bb- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| ae21-c636c9df73b1 | | controller-2.localdomain | | | | |
| 58b8a5bd-e438-4cb5-9267-ad87c6 | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 10dbb3 | | controller-1.localdomain | | | | |
| 5fbe010b-34af- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| 4a14-9965-393f37587682 | | controller-0.localdomain | | | | |
| 6e1d3d2a- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| 6ec4-47ab-8639-2ae945b19adc | | controller-2.localdomain | | | | |
| 901c0300-5081-412d- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| a7e8-2e77acc098bf | | controller-2.localdomain | | | | |
| b0b47dfb- | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 7d78-46e3-9c22-b1172989cfef | | controller-0.localdomain | | | | |
| cb0b6b69-320d-48dd- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| b3e3-f504889edae9 | | controller-0.localdomain | | | | |
| cdf555d7-0537-4bdc- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| bf77-5abe77709fe3 | | compute-0.localdomain | | | | |
| ddd0bb3e-0429-4e10-8adb- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| b81233e75ac0 | | controller-1.localdomain | | | | |
| e7524f86-81e4-46e5-ab2c- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| d6311427369d | | compute-1.localdomain | | | | |
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
One of the L3 Agent:
[heat-admin@overcloud-controller-0 ~]$ neutron agent-show 901c0300-5081-412d-a7e8-2e77acc098bf
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------------+-------------------------------------------------------------------------------+
| Field | Value |
+---------------------+-------------------------------------------------------------------------------+
| admin_state_up | True |
| agent_type | L3 agent |
| alive | True |
| availability_zone | nova |
| binary | neutron-l3-agent |
| configurations | { |
| | "agent_mode": "legacy", |
| | "gateway_external_network_id": "", |
| | "handle_internal_only_routers": true, |
| | "routers": 1, |
| | "interfaces": 1, |
| | "floating_ips": 1, |
| | "interface_driver": "neutron.agent.linux.interface.OVSInterfaceDriver", |
| | "log_agent_heartbeats": false, |
| | "external_network_bridge": "", |
| | "ex_gw_ports": 1 |
| | } |
| created_at | 2018-02-01 06:54:56 |
| description | |
| heartbeat_timestamp | 2018-02-02 13:25:52 |
| host | overcloud-controller-2.localdomain |
| id | 901c0300-5081-412d-a7e8-2e77acc098bf |
| started_at | 2018-02-02 11:02:27 |
| topic | l3_agent |
+---------------------+-------------------------------------------------------------------------------+
Neutron Router and DHCP Agent.
Neutron Virtual DHCP agent is available is used to ping to the SDN router gateway
[heat-admin@overcloud-controller-0 ~]$ ip netns
qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb
qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10
Router Gateway using QDHCP
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.173
PING 172.30.10.173 (172.30.10.173) 56(84) bytes of data.
64 bytes from 172.30.10.173: icmp_seq=1 ttl=64 time=1.16 ms
64 bytes from 172.30.10.173: icmp_seq=2 ttl=64 time=0.090 ms
64 bytes from 172.30.10.173: icmp_seq=3 ttl=64 time=0.092 ms
^Z
[1]+ Stopped sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.173
Floating IP of a Instance using QDHCP
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.178
PING 172.30.10.178 (172.30.10.178) 56(84) bytes of data.
From 172.30.10.178 icmp_seq=1 Destination Host Unreachable
From 172.30.10.178 icmp_seq=2 Destination Host Unreachable
From 172.30.10.178 icmp_seq=3 Destination Host Unreachable
From 172.30.10.178 icmp_seq=4 Destination Host Unreachable
^C
--- 172.30.10.178 ping statistics ---
6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5000ms
pipe 4
Router Gateway using QROUTER
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.173
PING 172.30.10.173 (172.30.10.173) 56(84) bytes of data.
64 bytes from 172.30.10.173: icmp_seq=1 ttl=64 time=0.115 ms
64 bytes from 172.30.10.173: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from 172.30.10.173: icmp_seq=3 ttl=64 time=0.063 ms
64 bytes from 172.30.10.173: icmp_seq=4 ttl=64 time=0.056 ms
^Z
[5]+ Stopped sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.173
Floating IP of a Instance using QROUTER
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.178
PING 172.30.10.178 (172.30.10.178) 56(84) bytes of data.
From 172.30.10.178 icmp_seq=1 Destination Host Unreachable
From 172.30.10.178 icmp_seq=2 Destination Host Unreachable
From 172.30.10.178 icmp_seq=3 Destination Host Unreachable
From 172.30.10.178 icmp_seq=4 Destination Host Unreachable
^Z
[6]+ Stopped sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.178
Route of QRouter
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default gateway 0.0.0.0 UG 0 0 0 qg-e8f74c7c-58
30.30.30.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-6a11beee-45
link-local 0.0.0.0 255.255.255.0 U 0 0 0 ha-4ad3b415-1b
169.254.192.0 0.0.0.0 255.255.192.0 U 0 0 0 ha-4ad3b415-1b
172.30.10.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-e8f74c7c-58
IP Route of QRouter
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ip route
default via 172.30.10.10 dev qg-e8f74c7c-58
30.30.30.0/24 dev qr-6a11beee-45 proto kernel scope link src 30.30.30.254
169.254.0.0/24 dev ha-4ad3b415-1b proto kernel scope link src 169.254.0.1
169.254.192.0/18 dev ha-4ad3b415-1b proto kernel scope link src 169.254.192.3
172.30.10.0/24 dev qg-e8f74c7c-58 proto kernel scope link src 172.30.10.173
Router Gateway IP & Floating IP
Router gateway IP and floating ip is assigned for qg
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
21: ha-4ad3b415-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:08:33:4b brd ff:ff:ff:ff:ff:ff
inet 169.254.192.3/18 brd 169.254.255.255 scope global ha-4ad3b415-1b
valid_lft forever preferred_lft forever
inet 169.254.0.1/24 scope global ha-4ad3b415-1b
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe08:334b/64 scope link
valid_lft forever preferred_lft forever
22: qg-e8f74c7c-58: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:90:73:04 brd ff:ff:ff:ff:ff:ff
inet 172.30.10.173/24 scope global qg-e8f74c7c-58
valid_lft forever preferred_lft forever
inet 172.30.10.178/32 scope global qg-e8f74c7c-58
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe90:7304/64 scope link
valid_lft forever preferred_lft forever
23: qr-6a11beee-45: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:cd:08:bf brd ff:ff:ff:ff:ff:ff
inet 30.30.30.254/24 scope global qr-6a11beee-45
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fecd:8bf/64 scope link
valid_lft forever preferred_lft forever
Expected Answer:
We should be able to take the machine floating IP externally.
We are not able to ping the floating IP assigned to the instance.
Close
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.