MIC-Powered Grid-Engine Rivals Supercomputer Share your comment!

Cluster computers today can already make use of Univa’s Grid Engine distributed resource management software to schedule and manage parallel applications. But with the debut of Intel’s teraFLOP-caliber supercomputer-on-a-card–the Xeon Phi–Univa has added support for Intel’s many-integrated-core architecture (MIC). And Grid Engine users are already reporting supercomputer-caliber performance from desktop systems accelerated by Xeon Phi coprocessors.

“We already have Grid Engine customers who are raving about the Xeon Phi,” says Fritz Ferstl, Grid Engine’s original programmer and now chief technology officer at Univa (Munich, Germany). “One system in a test environment has just four Xeon Phi coprocessors [240 cores]–about a $10,000 system–but they say it is about one-fourth the speed of a $10 million IBM Blue Gene supercomputer with 64,000 cores.”

CAPTION: Grid Engine works with Univa’s analytics and reporting metrics which track processor utilization for maximizing speed and energy efficiency of cluster computers. SOURCE: Univa 

Univa is partially funded by Intel Capitol. Nevertheless, Grid Engine has been adapted to work with other brands of accelerators, including nVidia’s GPUs, giving Ferstl a unique perspective on parallel processing. Systems using different types of processors are inevitable today, explains Ferstl; even systems using a single brand of processor seldom upgrade all cores at the same time, leaving software like Grid Engine to deal with version issues. Grid Engine keeps track of all such details about system resources, utilization schedules, and their health, then dispatches the right kind of workloads to the right kind of processing nodes in a cluster.

“Univa’s Grid Engine becomes your operating system for managing serial as well as parallel jobs, which can range from automotive customers using just a dozen or so cores, to oil and gas exploration customers whose parallel jobs use hundreds of cores, to scientific researchers who routinely run parallel jobs using as many as 100,000 cores,” he says.  

With native support for Intel’s MIC and the Xeon Phi, Grid Engine now tracks coprocessor availability in resource maps allowing users to specify exactly which Intel Xeon Phi coprocessor card a job will run on. Resource topology selection then matches the requested Xeon Phi coprocessor to the closest free main processor ensuring maximum performance by placing its workload as close as possible on the PCI Express bus to the Intel Xeon Phi coprocessors card running the parallel job. Finally, load and metric collection tools ensure that coprocessor jobs run on the least loaded Xeon Phi in the cluster.

Besides processing more parallel workloads per unit time, according to Ferstl, Univa’s Grid Engine also provides metrics about processor core utilization on Intel Xeon Phi coprocessors, tallying memory and power usage to enable IT to measure and control the energy consumption and other costs associated with running workloads on Intel Xeon Phi coprocessors.

Posted on December 21, 2012 by R. Colin Johnson, Geeknet Contributing Editor