Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

2.6. Network Topology Overview

Up to three sets of network pathways are needed for an Oracle HA cluster to operate properly:
  • The application tier, on behalf of users, must have access to the database nodes.
  • The Red Hat Cluster Suite node monitoring services need access to all nodes to determine the state of the cluster.
  • In the RAC case, Oracle Clusterware needs a high-speed pathway to implement Oracle Cache Fusion (or Global Cache Services – GCS).
Oracle RAC was certified with GCS running over bonded pairs of Ethernet interfaces, as was the Red Hat Cluster Suite heartbeat service to ensure coordinated cluster activity. It is optional to use bonded links for the Red Hat Cluster Suite heartbeat (or Oracle Clusterware GCS), but is highly recommended.

Note

The Oracle Cache Fusion links can run over standard UDP Ethernet (GbE links, at least), but can also run over a proprietary RDMA Infiniband network, and is fully supported by Oracle on RHEL, starting with RDBMS version 10.2.0.3. The Oracle GCS protocol functions like many other conventional cache protocols and must broadcast to all nodes when requesting a cache block. On a hardware bus, this is called Snooping. With GCS, this tends to generate geometric UDP broadcast traffic when node count exceeds eight. Most applications become the bottleneck long before, but GCS snoop traffic eventually limits the scalability of all RAC clusters. The IB/rdma feature (AKA Reliable Datagram Sockets – RDS/skgxp), mitigates this geometric growth, and makes larger node counts more practical. This effect is rarely seen in the majority of clusters, which tends to less than eight nodes.
The heartbeat networks for a cluster should be isolated into a dedicated, private VLAN that also filters for UDP broadcasting, in the RAC cluster case. All of these service networks should be a private and physically isolated as is possible. These fabrics should not be considered public, production networks. Make this clear when requesting network provisioning from NetOPS, or give them a copy of this document.