What is High Throughput Computing?

In computing, throughput is a measure of the number of computing tasks a system can complete over time.

High Throughput Computing (HTC) is an approach to computing that focuses on completing as much work as possible, integrated over weeks or months, by running independent tasks across multiple computers as they become available.

Animation showing independent computing tasks being distributed across multiple available computers.

In high throughput computing, independent tasks are distributed across available computers to complete more work over time.

What kinds of computations work well with the HTC approach?

Throughput Computing approaches work particularly well with workloads that can be expressed as groups of independent tasks. Tasks that do not rely on one another can run efficiently across available computing resources.

Workload Examples Why HTC?
🧬 Genomics Samples can be analyzed independently.
🖼️ Image Processing Each image can be processed separately.
🤖 Machine Learning Training runs or model settings can be tested at the same time.
🔬 Parameter Sweeps Each set of parameters can run as its own job.

Where did HTC come from?

The concept of High Throughput Computing (HTC) was developed by researchers at the University of Wisconsin-Madison in the 1990s.

Members of the HTCondor team in 1998

The HTCondor team in 1998, during the early development of High Throughput Computing.

At the time, much of the computing community focused on High Performance Computing (HPC) and measuring how quickly a computer could perform calculations using a metric called floating point operations per second (FLOPS).

Researchers recognized that some scientists cared less about the number operations the environment can provide them per second or minute and more about how many operations can be completed per month or per year. This idea led to the development of High Throughput Computing (HTC).

In 1996, researchers first explained the difference between High Throughput Computing (HTC) and High Performance Computing (HPC) during a seminar at NASA’s Goddard Space Flight Center and CERN. In 1997, HPCWire published an interview on High Throughput Computing.

Ongoing work at CHTC

Today, the Center for High Throughput Computing (CHTC) continues to build on the principles of High Throughput Computing and help researchers around the world accomplish more scientific work. CHTC is the home of the HTCondor Software Suite and Pelican Platform, two technologies that support HTC on large collections of heterogeneous computing resources that are distributed in both location and ownership. CHTC was also the original home of the term Research Computing Facilitation, a methodology developed to support end-users of computing approaches like HTC.