As a part of the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, a testbed prototype called “Arthur” at Sandia National Labs is helping to pioneer the future of many-integrated core (MIC) supercomputers.
Sandia National Labs aims to reach the goal of exascale supercomputers one thousand times more powerful than the petascale supercomputers used today. Its strategy is to co-design future supercomputers with experimental testbeds, architectural simulators and proxy applications that together accurately predict the performance future supercomputers.
Codesign is being accomplished with its Structural Simulation Toolkit (SST) and its Mantevo proxy applications, which can be calibrated to mimic every performance aspect of new large-scale supercomputers. Thus, they can be used for benchmarking large-scale applications even running on testbed-sized systems.
Sandia National Labs has developed many compact kernels specifically designed for massive parallel processors, and is currently porting its latest lightweight compute-node operating system (OS)–called Kitten–to the Xeon Phi.“We are using [our MIC-based Arthur system] to collect application performance data on our proxy applications,” explained Jim Ang, Manager of Scalable Computer Architectures of Sandia National Laboratories. “We’ve also been exploring a number of programming model implementations for those applications…and we’re trying to model the Knight’s Ferry coprocessor with our architectural simulations.”
“Kitten has very good support for memory subsystem access, which is better suited for high-performance computing,” said Ang. “We are trying to understand what the state-of-the-art is for these new many-core technologies…with the next step being predicting new capabilities that don’t exist today, and perhaps give feedback to Intel and Appro on the design trade offs that could be made for the next generation.”
Arthur is currently a 42-node MIC-cluster using two six-core Xeon 5600 processors with 24-Gbytes of Double Data Rate (DDR3-1600 MHz) memory, connected to two 30-core Knight’s Ferry advanced prototype chips–each with 2-Gbytes of memory (DDR5-1800MHz) and connected by a Quad Data Rate (QDR) Mellanox Infiniband IV. The prototype Knights Ferry chips will soon be replaced with Xeon Phi chips on Knight Corner coprocessor boards connected by PCIe.
Arthur is based on the Appro Xtreme-X Supercomputer Cluster architecture. Appro has been an Intel partner throughout the development of the Knight’s Ferry and Knight’s Corner, and has also installed MIC-clusters at the National Institute for Computational Sciences (NICS), a joint effort of the University of Tennessee and Oak Ridge National Lab.
Sandia National Labs ultimate goal is low-power exascale supercomputers, which it plans to design using an iterative process starting with the simultaneous co-design of experimental testbeds, architectural simulators and proxy applications. The design loop will then be closed by performance evaluations and analysis that results in recommended changes to the next generation of experimental testbeds, architectural simulators and proxy applications. By iterating this design loop, Sandia National Labs hopes to evolve new hardware architectures that meet the demanding performance and power requirements for economical exascale supercomputers.