Physical Society Colloquium

An Execution Model for Exascale Computing

Thomas Sterling

Indiana University, Bloomington

For nearly two decades MPPs and commodity clusters have excelled due to the exponential growth of processor performance and the ubiquity of the MPI programming interface that mapped applications on to resources. Processor core performance is no longer increasing significantly due to the flat-lining of clock rates because of power limitations and the saturation of ILP through exhaustion of processor complexity. With heterogeneous node architectures comprising multicore sockets in combination with GPU accelerators, MPI and its underlying Communicating Sequential Processes (CSP) execution model are no longer sufficient or even appropriate for parallel application programming of future HPC systems. Ad hoc methods of combining different levels of programming models provide short-term examples of the potential of such systems but at the expense of programmability, portability, and efficiency. Throughout the remarkable trillion X history of digital electronic computing, the field has experienced at least five distinct phases; CSP the most recent. Each was driven by technology trends requiring new architectures and new programming models; often with a change in underlying paradigm as well. Clusters and MPPs that dominate the field of HPC are entering such a phase shift requiring a new paradigm for guiding the programming of new parallel algorithms and governing the design of future cluster nodes. This presentation identifies the challenges facing HPC architecture and programming and describes a new execution model, ParalleX, which may provide a strategy for addressing them. Using the experimental runtime software system, HPX-3, which serves as a reference implementation for ParalleX, results from several parallel applications have exhibited dramatic scaling characteristics through advanced flow control methods. The implications for a future HPC paradigm will be considered that may enable Exascale computing by the end of this decade.