Floating IP not pinging Externally
I have successfully deployed everything in Redhat Openstack 11 with following settings. I was not able to ping the floating IP externally rather i can perform ping, ssh and other things using namespace.
I have three controllers and two hypercoverged Compute.
VLAN for RHOSP 11 Setup
172.26.11.0/24 - Provision Network ( VLAN2611 )
172.26.12.0/24 - Internal Network ( VLAN2612 )
172.26.13.0/24 - Tentant Network ( VLAN2613 )
172.26.14.0/24 - Storage Network ( VLAN2614 )
172.26.16.0/24 - Storage Managment ( VLAN2616 )
172.26.17.0/24 - Management Network ( VLAN2617 )
172.30.10.0/23 - External Network ( VLAN3010 )
Server Setup:
[stack@director ~]$ nova list
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| 3e37a6ed-1b0a-49de-9aa8-5515949ad11a | overcloud-compute-0 | ACTIVE | - | Running | ctlplane=172.26.11.13 |
| 3bab2815-1df8-4b1a-ab70-fa1d00dd5889 | overcloud-compute-1 | ACTIVE | - | Running | ctlplane=172.26.11.25 |
| 531cc5ad-ceb2-40c4-9662-1a984eea1907 | overcloud-controller-0 | ACTIVE | - | Running | ctlplane=172.26.11.12 |
| 598cb725-ed9d-4e7f-b8d1-3d5ac0df86d8 | overcloud-controller-1 | ACTIVE | - | Running | ctlplane=172.26.11.23 |
| a92cbacd-301e-4201-aa74-b100eb245345 | overcloud-controller-2 | ACTIVE | - | Running | ctlplane=172.26.11.28 |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
Controller-0 IP's Assigned:
All other two controllers will have the same IP address configuration.
[stack@director ~]$ ssh heat-admin@172.26.11.12
Last login: Wed Feb 14 09:23:13 2018 from 172.26.11.254
[heat-admin@overcloud-controller-0 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether c8:1f:66:e1:1a:c3 brd ff:ff:ff:ff:ff:ff
inet 172.26.11.12/24 brd 172.26.11.255 scope global em1
valid_lft forever preferred_lft forever
inet 172.26.11.22/32 brd 172.26.11.255 scope global em1
valid_lft forever preferred_lft forever
inet6 fe80::ca1f:66ff:fee1:1ac3/64 scope link
valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000
link/ether c8:1f:66:e1:1a:c4 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ca1f:66ff:fee1:1ac4/64 scope link
valid_lft forever preferred_lft forever
4: em3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000
link/ether c8:1f:66:e1:1a:c5 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ca1f:66ff:fee1:1ac5/64 scope link
valid_lft forever preferred_lft forever
5: em4: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether c8:1f:66:e1:1a:c6 brd ff:ff:ff:ff:ff:ff
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether c6:05:34:74:27:e0 brd ff:ff:ff:ff:ff:ff
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether c8:1f:66:e1:1a:c4 brd ff:ff:ff:ff:ff:ff
inet6 fe80::800e:f6ff:fe6d:245/64 scope link
valid_lft forever preferred_lft forever
8: vlan2612: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 9a:12:3a:34:7a:7c brd ff:ff:ff:ff:ff:ff
inet 172.26.12.12/24 brd 172.26.12.255 scope global vlan2612
valid_lft forever preferred_lft forever
inet 172.26.12.18/32 brd 172.26.12.255 scope global vlan2612
valid_lft forever preferred_lft forever
inet6 fe80::9812:3aff:fe34:7a7c/64 scope link
valid_lft forever preferred_lft forever
9: vlan2613: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:2d:8b:7b:f1:21 brd ff:ff:ff:ff:ff:ff
inet 172.26.13.20/24 brd 172.26.13.255 scope global vlan2613
valid_lft forever preferred_lft forever
inet6 fe80::f82d:8bff:fe7b:f121/64 scope link
valid_lft forever preferred_lft forever
10: vlan2614: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether c2:ea:76:13:4e:16 brd ff:ff:ff:ff:ff:ff
inet 172.26.14.18/24 brd 172.26.14.255 scope global vlan2614
valid_lft forever preferred_lft forever
inet6 fe80::c0ea:76ff:fe13:4e16/64 scope link
valid_lft forever preferred_lft forever
11: vlan2616: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 82:e6:64:04:d7:23 brd ff:ff:ff:ff:ff:ff
inet 172.26.16.12/24 brd 172.26.16.255 scope global vlan2616
valid_lft forever preferred_lft forever
inet6 fe80::80e6:64ff:fe04:d723/64 scope link
valid_lft forever preferred_lft forever
12: vlan2617: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether d2:74:4f:18:b5:3c brd ff:ff:ff:ff:ff:ff
inet 172.26.17.14/24 brd 172.26.17.255 scope global vlan2617
valid_lft forever preferred_lft forever
inet6 fe80::d074:4fff:fe18:b53c/64 scope link
valid_lft forever preferred_lft forever
13: vlan3010: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether 32:e2:86:b9:d2:3e brd ff:ff:ff:ff:ff:ff
inet 172.30.10.21/23 brd 172.30.11.255 scope global vlan3010
valid_lft forever preferred_lft forever
inet6 fe80::30e2:86ff:feb9:d23e/64 scope link
valid_lft forever preferred_lft forever
14: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether f2:7e:78:3c:ee:49 brd ff:ff:ff:ff:ff:ff
15: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether a2:4d:a0:64:3a:4e brd ff:ff:ff:ff:ff:ff
16: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
link/gre 0.0.0.0 brd 0.0.0.0
17: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
18: gre_sys@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65490 qdisc pfifo_fast master ovs-system state UNKNOWN qlen 1000
link/ether f6:71:95:be:da:53 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f471:95ff:febe:da53/64 scope link
valid_lft forever preferred_lft forever
Controller-0 OVS Bridge :
qg is external interface of SDN router
qr is internal interface of SDN router
These interfaces are directly created inside the br-int. In older versions of RHOSP. There is no patch between the br-int and br-ex. So the qg will be created directly in br-ex. In this version, we find that both interfaces are created inside the br-int, if i change the external bridge as br-int in all L3 agents, then the router interfaces shows down. Even-though all the communication of ping and ssh happens inside the qrouter namespaces itself.
[heat-admin@overcloud-controller-0 ~]$ sudo ovs-vsctl show
f6411a64-6dbd-4a7d-931a-6a99b63d7911
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qg-0f094325-6c"
tag: 10
Interface "qg-0f094325-6c"
type: internal
Port "qr-fff1e03e-44"
tag: 8
Interface "qr-fff1e03e-44"
type: internal
Port "tapef7874a7-a3"
tag: 8
Interface "tapef7874a7-a3"
type: internal
Port "ha-a3430c62-90"
tag: 4095
Interface "ha-a3430c62-90"
type: internal
Port "ha-37bad2be-92"
tag: 9
Interface "ha-37bad2be-92"
type: internal
Port "tap102385e5-b7"
tag: 4
Interface "tap102385e5-b7"
type: internal
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "gre-ac1a0d0f"
Interface "gre-ac1a0d0f"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.15"}
Port "gre-ac1a0d10"
Interface "gre-ac1a0d10"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.16"}
Port "gre-ac1a0d16"
Interface "gre-ac1a0d16"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.22"}
Port br-tun
Interface br-tun
type: internal
Port "gre-ac1a0d0c"
Interface "gre-ac1a0d0c"
type: gre
options: {df_default="true", in_key=flow, local_ip="172.26.13.20", out_key=flow, remote_ip="172.26.13.12"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "vlan2617"
tag: 2617
Interface "vlan2617"
type: internal
Port "vlan2612"
tag: 2612
Interface "vlan2612"
type: internal
Port "vlan2613"
tag: 2613
Interface "vlan2613"
type: internal
Port br-ex
Interface br-ex
type: internal
Port "vlan3010"
tag: 3010
Interface "vlan3010"
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port "vlan2614"
tag: 2614
Interface "vlan2614"
type: internal
Port "vlan2616"
tag: 2616
Interface "vlan2616"
type: internal
Port "bond1"
Interface "em2"
Interface "em3"
ovs_version: "2.6.1"
Neutron Agent List
[heat-admin@overcloud-controller-0 ~]$ neutron agent-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
| 08afba9b-1952-4c43-a3ec- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| 1b6a1cf49370 | | controller-1.localdomain | | | | |
| 1c7794b0-726c-4d70-81bc- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| df761ad105bd | | controller-1.localdomain | | | | |
| 23aba452-ecb2-4d61-96b5-f8224c | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| 6de482 | | controller-0.localdomain | | | | |
| 2acabaa4-cad1-4e25-b102-fe5f72 | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 0de5b8 | | controller-2.localdomain | | | | |
| 38074c45-565c-45bb- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| ae21-c636c9df73b1 | | controller-2.localdomain | | | | |
| 58b8a5bd-e438-4cb5-9267-ad87c6 | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 10dbb3 | | controller-1.localdomain | | | | |
| 5fbe010b-34af- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| 4a14-9965-393f37587682 | | controller-0.localdomain | | | | |
| 6e1d3d2a- | Metadata agent | overcloud- | | :-) | True | neutron-metadata-agent |
| 6ec4-47ab-8639-2ae945b19adc | | controller-2.localdomain | | | | |
| 901c0300-5081-412d- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| a7e8-2e77acc098bf | | controller-2.localdomain | | | | |
| b0b47dfb- | DHCP agent | overcloud- | nova | :-) | True | neutron-dhcp-agent |
| 7d78-46e3-9c22-b1172989cfef | | controller-0.localdomain | | | | |
| cb0b6b69-320d-48dd- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| b3e3-f504889edae9 | | controller-0.localdomain | | | | |
| cdf555d7-0537-4bdc- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| bf77-5abe77709fe3 | | compute-0.localdomain | | | | |
| ddd0bb3e-0429-4e10-8adb- | L3 agent | overcloud- | nova | :-) | True | neutron-l3-agent |
| b81233e75ac0 | | controller-1.localdomain | | | | |
| e7524f86-81e4-46e5-ab2c- | Open vSwitch agent | overcloud- | | :-) | True | neutron-openvswitch-agent |
| d6311427369d | | compute-1.localdomain | | | | |
+--------------------------------+--------------------+--------------------------------+-------------------+-------+----------------+---------------------------+
One of the L3 Agent:
[heat-admin@overcloud-controller-0 ~]$ neutron agent-show 901c0300-5081-412d-a7e8-2e77acc098bf
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------------+-------------------------------------------------------------------------------+
| Field | Value |
+---------------------+-------------------------------------------------------------------------------+
| admin_state_up | True |
| agent_type | L3 agent |
| alive | True |
| availability_zone | nova |
| binary | neutron-l3-agent |
| configurations | { |
| | "agent_mode": "legacy", |
| | "gateway_external_network_id": "", |
| | "handle_internal_only_routers": true, |
| | "routers": 1, |
| | "interfaces": 1, |
| | "floating_ips": 1, |
| | "interface_driver": "neutron.agent.linux.interface.OVSInterfaceDriver", |
| | "log_agent_heartbeats": false, |
| | "external_network_bridge": "", |
| | "ex_gw_ports": 1 |
| | } |
| created_at | 2018-02-01 06:54:56 |
| description | |
| heartbeat_timestamp | 2018-02-02 13:25:52 |
| host | overcloud-controller-2.localdomain |
| id | 901c0300-5081-412d-a7e8-2e77acc098bf |
| started_at | 2018-02-02 11:02:27 |
| topic | l3_agent |
+---------------------+-------------------------------------------------------------------------------+
Neutron Router and DHCP Agent.
Neutron Virtual DHCP agent is available is used to ping to the SDN router gateway
[heat-admin@overcloud-controller-0 ~]$ ip netns
qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb
qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10
Router Gateway using QDHCP
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.173
PING 172.30.10.173 (172.30.10.173) 56(84) bytes of data.
64 bytes from 172.30.10.173: icmp_seq=1 ttl=64 time=1.16 ms
64 bytes from 172.30.10.173: icmp_seq=2 ttl=64 time=0.090 ms
64 bytes from 172.30.10.173: icmp_seq=3 ttl=64 time=0.092 ms
^Z
[1]+ Stopped sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.173
Floating IP of a Instance using QDHCP
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qdhcp-2cee840e-f683-48ed-a05f-ac993f6cac10 ping 172.30.10.178
PING 172.30.10.178 (172.30.10.178) 56(84) bytes of data.
From 172.30.10.178 icmp_seq=1 Destination Host Unreachable
From 172.30.10.178 icmp_seq=2 Destination Host Unreachable
From 172.30.10.178 icmp_seq=3 Destination Host Unreachable
From 172.30.10.178 icmp_seq=4 Destination Host Unreachable
^C
--- 172.30.10.178 ping statistics ---
6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5000ms
pipe 4
Router Gateway using QROUTER
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.173
PING 172.30.10.173 (172.30.10.173) 56(84) bytes of data.
64 bytes from 172.30.10.173: icmp_seq=1 ttl=64 time=0.115 ms
64 bytes from 172.30.10.173: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from 172.30.10.173: icmp_seq=3 ttl=64 time=0.063 ms
64 bytes from 172.30.10.173: icmp_seq=4 ttl=64 time=0.056 ms
^Z
[5]+ Stopped sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.173
Floating IP of a Instance using QROUTER
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.178
PING 172.30.10.178 (172.30.10.178) 56(84) bytes of data.
From 172.30.10.178 icmp_seq=1 Destination Host Unreachable
From 172.30.10.178 icmp_seq=2 Destination Host Unreachable
From 172.30.10.178 icmp_seq=3 Destination Host Unreachable
From 172.30.10.178 icmp_seq=4 Destination Host Unreachable
^Z
[6]+ Stopped sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ping 172.30.10.178
Route of QRouter
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default gateway 0.0.0.0 UG 0 0 0 qg-e8f74c7c-58
30.30.30.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-6a11beee-45
link-local 0.0.0.0 255.255.255.0 U 0 0 0 ha-4ad3b415-1b
169.254.192.0 0.0.0.0 255.255.192.0 U 0 0 0 ha-4ad3b415-1b
172.30.10.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-e8f74c7c-58
IP Route of QRouter
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ip route
default via 172.30.10.10 dev qg-e8f74c7c-58
30.30.30.0/24 dev qr-6a11beee-45 proto kernel scope link src 30.30.30.254
169.254.0.0/24 dev ha-4ad3b415-1b proto kernel scope link src 169.254.0.1
169.254.192.0/18 dev ha-4ad3b415-1b proto kernel scope link src 169.254.192.3
172.30.10.0/24 dev qg-e8f74c7c-58 proto kernel scope link src 172.30.10.173
Router Gateway IP & Floating IP
Router gateway IP and floating ip is assigned for qg
[heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-bb4d96e5-07e1-4ad6-b120-f11c6a2298eb ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
21: ha-4ad3b415-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:08:33:4b brd ff:ff:ff:ff:ff:ff
inet 169.254.192.3/18 brd 169.254.255.255 scope global ha-4ad3b415-1b
valid_lft forever preferred_lft forever
inet 169.254.0.1/24 scope global ha-4ad3b415-1b
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe08:334b/64 scope link
valid_lft forever preferred_lft forever
22: qg-e8f74c7c-58: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:90:73:04 brd ff:ff:ff:ff:ff:ff
inet 172.30.10.173/24 scope global qg-e8f74c7c-58
valid_lft forever preferred_lft forever
inet 172.30.10.178/32 scope global qg-e8f74c7c-58
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe90:7304/64 scope link
valid_lft forever preferred_lft forever
23: qr-6a11beee-45: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:cd:08:bf brd ff:ff:ff:ff:ff:ff
inet 30.30.30.254/24 scope global qr-6a11beee-45
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fecd:8bf/64 scope link
valid_lft forever preferred_lft forever
Expected Answer:
We should be able to take the machine floating IP externally.
We are not able to ping the floating IP assigned to the instance.
