What is high performance computing or HPC?

High performance computing (HPC) is the ability to process data and perform complex calculations at high speeds. To put it into perspective, a laptop or desktop with a 3 GHz processor can perform around 3 billion calculations per second. While that is much faster than any human can achieve, it pales in comparison to HPC solutions that can perform quadrillions of calculations per second. 

One of the best-known types of HPC solutions is the supercomputer. A supercomputer contains thousands of compute nodes that work together to complete one or more tasks. This is called parallel processing. It’s similar to having thousands of PCs networked together, combining compute power to complete tasks faster.

It is through data that groundbreaking scientific discoveries are made, game-changing innovations are fueled, and quality of life is improved for billions of people around the globe. HPC is the foundation for scientific, industrial, and societal advancements.

As technologies like the Internet of Things (IoT), artificial intelligence (AI), and 3-D imaging evolve, the size and amount of data that organizations have to work with is growing exponentially. For many purposes, such as streaming a live sporting event, tracking a developing storm, testing new products, or analyzing stock trends, the ability to process data in real time is crucial.

To keep a step ahead of the competition, organizations need lightning-fast, highly reliable IT infrastructure to process, store, and analyze massive amounts of data.

HPC solutions have three main components:

• Compute
• Network
• Storage

To build a high performance computing architecture, compute servers are networked together into a cluster. Software programs and algorithms are run simultaneously on the servers in the cluster. The cluster is networked to the data storage to capture the output. Together, these components operate seamlessly to complete a diverse set of tasks.

To operate at maximum performance, each component must keep pace with the others. For example, the storage component must be able to feed and ingest data to and from the compute servers as quickly as it is processed. Likewise, the networking components must be able to support the high-speed transportation of data between compute servers and the data storage. If one component cannot keep up with the rest, the performance of the entire HPC infrastructure suffers.

An HPC cluster consists of hundreds or thousands of compute servers that are networked together. Each server is called a node. The nodes in each cluster work in parallel with each other, boosting processing speed to deliver high performance computing.

Deployed on premises, at the edge, or in the cloud, HPC solutions are used for a variety of purposes across multiple industries. Examples include:

• Research labs. HPC is used to help scientists find sources of renewable energy, understand the evolution of our universe, predict and track storms, and create new materials.
• Media and entertainment. HPC is used to edit feature films, render mind-blowing special effects, and stream live events around the world.
• Oil and gas. HPC is used to more accurately identify where to drill for new wells and to help boost production from existing wells.
• Artificial intelligence and machine learning. HPC is used to detect credit card fraud, provide self-guided technical support, teach self-driving vehicles, and improve cancer screening techniques.
• Financial services. HPC is used to track real-time stock trends and automate trading.
• HPC is used to design new products, simulate test scenarios, and make sure that parts are kept in stock so that production lines aren’t held up.
• HPC is used to help develop cures for diseases like diabetes and cancer and to enable faster, more accurate patient diagnosis.

The NetApp HPC solution features a complete line of high-performance, high-density E-Series storage systems. A modular architecture with industry-leading price/performance offers a true pay-as-you-grow solution to support storage requirements for multi-petabyte datasets. The system is integrated with leading HPC file systems, including Lustre, IBM Spectrum Scale, BeeGFS, and others to handle the performance and reliability requirements of the world’s largest computing infrastructures.

E-Series systems provide the performance, reliability, scalability, simplicity, and lower TCO needed to take on the challenges of supporting extreme workloads:

• Performance. Delivers up to 1 million random read IOPS and 13GB/sec sustained (maximum burst) write bandwidth per scalable building block. Optimized for both flash and spinning media, the NetApp HPC solution includes built-in technology that monitors workloads and automatically adjusts configurations to maximize performance.
• Reliability. Fault-tolerant design delivers greater than 99.9999% availability, proven by more than 1 million systems deployed. Built-in Data Assurance features help make sure that data is accurate with no drops, corruption, or missed bits.
• Easy to deploy and manage. Modular design, on-the-fly (“cut and paste”) replication of storage blocks, proactive monitoring, and automation scripts all add up to easy, fast and flexible management.
• Scalability. A granular, building-block approach to growth that enables seamless scalability from terabytes to petabytes by adding capacity in any increment—one or multiple drives at a time.
• Lower TCO. Price/performance-optimized building blocks and the industry’s best density per delivers low power, cooling, and support costs, and 4-times lower failure rates than commodity HDD and SSD devices.