FlexPod MetroCluster IP solutions with ONTAP 9.7 and compliant switches
Jyh-shing Chen
FlexPod® MetroCluster™ IP (Internet Protocol) solutions combine the proven FlexPod converged infrastructures with the NetApp®ONTAP® MetroCluster IP solution capabilities to help companies maintain continued availability of business-critical data services. This blog post highlights the latest FlexPod Datacenter solution Cisco Validated Design (CVD). It also describes the compliant switches deployment architecture for MetroCluster IP, which is supported by ONTAP 9.7 to reduce solution complexities and costs. And it discusses the use of ONTAP Mediator to monitor the storage clusters and automate an unplanned switchover to resume data services quickly when a site disaster occurs.
This document provides examples of the FlexPod MetroCluster IP solution at two different scales to demonstrate how various supported components can be used to build solutions that meet compute, storage, and performances requirements. Deploying these new solution configurations with compliant switches and ONTAP Mediator can help reduce solution complexity, save costs, and speed up disaster recovery. The FlexPod MetroCluster IP solutions offer zero RPO and low RTO. They protect companies against site-wide disasters and many other single-point-of failure scenarios and help them to achieve their business continuity objectives.
FlexPod solution updates
For more than a decade, FlexPod converged infrastructures have helped companies deploy mission-critical workloads on FlexPod with confidence. These infrastructures are powered by Cisco UCS Servers, Cisco Nexus switches, and NetApp storage arrays, and a large portfolio of Cisco Validated Designs and NetApp Verified Architectures (NVAs). These CVDs and NVAs cover all major data center workloads and are the result of continued collaborations and innovations of NetApp and Cisco on FlexPod platform solutions. Incorporating extensive testing and validations in their creation process, these CVDs and NVAs provide reference solution architecture designs and step-by-step deployment guides to help partners and customers deploy and adopt FlexPod solutions. By using these CVDs and NVAs as the guides for design and implementation, businesses can reduce risk, reduce solution downtime, and increase the availability, scalability, flexibility, and security of the FlexPod solutions they deploy.As an example, Figure 1 shows the validation topology of the FlexPod Datacenter with VMware vSphere 7.0 CVD. This design consists of the Cisco fourth-generation fabric interconnects, UCS B-Series blade servers, UCS C-Series rack servers, UCS C4200 rack server chassis with C125 server nodes, Nexus switches, and the NetApp AFF A400 storage controllers.
Figure 1) FlexPod Datacenter with VMware vSphere 7.0 CVD validation topology
Benefits of FlexPod MetroCluster IP solutions
FlexPod MetroCluster IP solutions combine the proven FlexPod converged infrastructure with the capabilities of ONTAP MetroCluster IP to synchronously replicate data between sites. This means protection for your enterprise applications and business-critical workloads such as databases, virtual desktop infrastructures, artificial intelligence, and machine learning against a disaster that results in a complete site outage.Here are highlights of the benefits to customers of adopting FlexPod MetroCluster IP solutions:
- FlexPod solutions are highly available, highly flexible, and highly scalable. A large portfolio of CVDs and NVAs covers solution designs and detailed implementations.
- The MetroCluster IP solution synchronously replicates data between sites and offers zero RPO and low RTO to protect against data loss and enable fast recovery.
- The MetroCluster IP solution is an active-active solution. Both MetroCluster IP sites can serve workloads and applications locally and also function as DR sites for their partners.
- The MetroCluster IP solution supports deployment distances of up to 700 km, when latency and other network requirements are met, to enable a wide range of site considerations.
- The MetroCluster IP solution can protect business-critical data services against site disasters and a variety of single-point-of-failure scenarios at each site due to data replication between sites and the redundant solution architecture design.
- The application and workload data written to a MetroCluster IP solution is automatically protected without any additional application- or workload-level configuration. The switchover and switchback operations are transparent to applications and workloads.
MetroCluster IP solution architecture with compliant switches
With ONTAP 9.6 and earlier releases, the MetroCluster IP solution requires dedicated switches that are validated and provided by NetApp. Beginning with ONTAP 9.7, MetroCluster IP solutions for some platforms can support switches that are not validated by NetApp if they are compliant with NetApp specifications. Using the Nexus switches (which are already part of a FlexPod solution) as compliant switches reduces the cost and complexity of the solution and increases the usage of the switches.Figure 2 shows an AFF A700 cluster for one of the sites of a MetroCluster IP solution deployed without dedicated switches. In this deployment configuration, the two storage controller nodes (HA Pair) at one site are connected back to back for the intracluster traffic and the MetroCluster IP interfaces are connected to the compliant switches (not shown in the figure). The MetroCluster replication data travels from the controller nodes to the compliant switches and the intersite links to reach the cluster and storage at the other site.
Figure 2) Intracluster and MetroCluster IP fabric of an AFF A700 MetroCluster IP solution with compliant switches at one site.
The compliant switches deployment architecture can use the Nexus switches, which are already part of a typical FlexPod design, to carry both client data and MetroCluster IP storage traffic. Here are the general requirements for deploying a MetroCluster IP solution with compliant switches:
- Only platforms that support MetroCluster IP and provide dedicated ports for switchless cluster interconnects are supported, including AFF A300, A320, A700, A800, FAS8200, and FAS9000.
- The MetroCluster IP interface can be connected to any switch port that can be configured to meet the requirements.
- The speed of the switch ports required depends on the platform. For example, it is 25Gbps for AFF A300, 40Gbps for AFF A700, and 40/100Gbps for AFF A800.
- The ISLs must be 10Gbps or higher and must be sized appropriately for the load on the MetroCluster configuration.
- The MetroCluster configuration must be connected to two networks, and each MetroCluster node must be connected to two network switches.
- The network must meet additional requirements for ISL sharing, cabling, and required settings on intermediate switches.
- The MTU of 9216 must be configured on all switches that carry MetroCluster IP traffic.
- The switches must support QoS/traffic classification, explicit congestion notification (ECN), L4 port-vlan load-balancing policies to preserve order along the path, and L2 Flow Control (L2FC).
- The cables connecting the nodes to the switches must be purchased from NetApp and supported by the switch vendor.
ONTAP Mediator support for automated unplanned switchover
New ONTAP Mediator software is included with ONTAP 9.7 for the MetroCluster IP solution. The ONTAP Mediator enables the solution to perform an automated unplanned switchover (AUSO). The best practice is to deploy the Mediator software at a third site, as shown in Figure 3.Figure 3) ONTAP Mediator deployed at a third site provides support for AUSO.
The ONTAP Mediator also allows the AUSO to be disabled when the two sites encounter a failure in mirroring data between them. Preventing an automatic switchover when the intersite links are down allows the administrator to decide if it is appropriate to switch over.
To install the ONTAP Mediator service in a MetroCluster configuration, make sure that the following network requirements are met:
- The links between the Mediator service and the MetroCluster configuration must have at least 1Gbps of bandwidth.
- The round-trip latency must be no more than 25ms.
- The MTU size must be at least 1400.
- The packet loss on the network must be less than or equal to 0.01%.
FlexPod MetroCluster IP solutions architectures with compliant switches
The following two solution architectures illustrate how FlexPod platforms and MetroCluster IP solutions using compliant switches can be implemented together at different scales for sites with various compute, storage, and performance requirements to protect data. These FlexPod MetroCluster IP solutions provide compute resources and storage for each site. Moreover, the MetroCluster IP solutions use the network infrastructure within each site and between sites to synchronously replicate data from one site to the other. The solution architectures ensure no data loss, zero RPO, low RTO, and fast restoration of services to achieve business continuity objectives despite a single-site outage scenario.For a small site with limited compute and storage requirements, a FlexPod MetroCluster IP solution built with the UCS C-series rack servers, Nexus 9K switches, and AFF A300 storage arrays provides a small but scalable platform.
Figure 4 shows two identical FlexPod configurations, one at each site, that are connected by the intersite links (ISLs). The ONTAP MetroClsuter IP sites can be separated by up to 700 km, if the latency and other network requirements detailed in ONTAP documentation are met. There are two types of connections between the Nexus switches and the storage controllers, and they are used for data traffic and MetroCluster data replication between the two ONTAP clusters.
Figure 4) FlexPod MetroCluster IP solution architecture for a small site with compliant switches and AFF A300.
The FlexPod Datacenter platform offers the scalability and performance needed for a data center that supports many applications and workloads. The FlexPod MetroCluster IP solution architecture example illustrated in Figure 5 takes advantage of the compliant switches’ deployment architecture and a FlexPod Datacenter configuration. The configuration includes Cisco UCS B-Series and C-Series servers, fourth-generation UCS fabric interconnect 6454, Nexus 9K switches, and NetApp AFF A700 storage controllers at each site.
Figure 5) FlexPod MetroCluster IP solution architecture for a large site with compliant switches and AFF A700.
The ONTAP Mediator shown in the examples is deployed at a third site to monitor the ONTAP storage clusters at the two MetroCluster sites. It also provides the AUSO capability to automatically perform a switchover operation when a site experiences outage so that the data services can quickly resume from the storage at the surviving site. To scale and grow the solution, additional servers and SSD shelves can be added as needed. Finally, NetApp recommends routine monitoring and testing of the disaster scenarios to verify that the solution has been properly configured and can survive simulated and real disaster scenarios.
Conclusion
FlexPod MetroCluster IP solutions combine the highly available, flexible, and scalable FlexPod platforms and the capabilities of ONTAP MetroCluster IP solutions. The combination helps companies to ensure the continued availability of business-critical data services that are essential to their business success.There are many supported FlexPod and ONTAP MetroCluster IP configurations that can be implemented together to create a variety of FlexPod MetroCluster IP solutions. Different companies and different sites will have different solution requirements for compute and storage capacities or performance. By using the example FlexPod MetroCluster IP solution architectures presented in this blog post as a guide, companies can adapt these configurations to meet their requirements.
For a small site, deploying a configuration combination of AFF A300, Nexus 9K switches, and UCS C-series servers offers a small yet scalable solution architecture. For a large site, deploying the FlexPod Datacenter architecture with the latest-generation fabric interconnect, UCS B-Series and C-Series servers, Nexus 9K switches, and AFF A700 provides the required scale and performance. Over time, additional servers and SSD storage shelves can be added to grow the solution to accommodate additional applications, workloads, and data.
In summary, companies can deploy the NetApp ONTAP MetroCluster IP solution using the simplified compliant switches configuration in a proven FlexPod converged infrastructure and automating the unplanned switchover operation with the ONTAP Mediator. With this configuration, companies can ensure the continued availability of data services and mitigate a site-wide disaster and many other single-point-of-failure scenarios to achieve their business continuity objectives.
Where to Find More Information
For more information about the FlexPod architectures, MetroCluster IP installation and configuration details, and the supported hardware platforms, firmware releases, and other related information, refer to the following websites and documents. Cisco FlexPod Design Guides Cisco UCS Hardware and Software Compatibility Cisco Validated Design Overview FlexPod Datacenter with Cisco Intersight and NetApp ONTAP 9.7 FlexPod Datacenter with Cisco ACI Multi-Pod, NetApp MetroCluster IP, and VMware vSphere 6.7 Design Guide FlexPod Datacenter with VMware vSphere 7.0 FlexPod MetroCluster IP Solutions NetApp Hardware Universe (HWU) NetApp Interoperability Matrix Tool (IMT) ONTAP 9 Documentation Center ONTAP 9 MetroCluster IP Installation and Configuration Guide ONTAP 9 MetroCluster Management and Disaster Recovery Guide TR-4036 FlexPod Datacenter Technical Specifications TR-4293 FlexPod Express Technical Specifications TR-4689 MetroCluster IP Solution Architecture and Design TR-4705 NetApp MetroCluster Solution Architecture and Design VMware vSphere 5.x, 6.x, and 7.x support with NetApp MetroCluster (2031038)
Jyh-shing Chen
Jyh-shing Chen is a Senior Technical Marketing Engineer with NetApp. His current focus is on Converged Infrastructure solution enablement, validation, and deployment / management simplification with Ansible automation. Jyh-shing joined NetApp in 2006 and worked in several other areas previously, including storage interoperability with Solaris and VMware vSphere operating systems, qualification of ONTAP MetroCluster solutions and Cloud Volumes data services. Before joining NetApp, Jyh-shing’s engineering experiences include software and firmware development on cardiology health imaging system, mass spectrometer system, Fibre Channel virtual tape library, and the research and development of microfluidic devices. Jyh-shing holds B.S. and M.S. degrees from National Taiwan University, PhD from Massachusetts Institute of Technology, and MBA from Meredith College.
