Welcome to Go Parallel – a site and community, sponsored by Intel and developed in partnership with Geeknet, devoted to learning, practicing, and advancing the disciplines of parallel programming. Parallel programming is the next big sea-change in programming practice – the place programming needs to go (at least for a while, and perhaps forever) in order to achieve greater application performance, increased efficiency, and more harmonious interoperation of components within increasingly complex and sophisticated systems.
Since the early-to-middle part of the last decade, CPU clock speeds have been coming up against a resilient limit of around 4GHz, imposed by the difficulty of dissipating heat from a given silicon area. Chipmakers, of course, could see this limit coming, and began adding various forms of parallelism into single-core chips as the 90s drew to a close. Starting around 2002, ‘Climax’ single-core CPUs like Intel’s own (Foster) Xeons and Pentium 4s (Northwood), featured hyperthreading (on-chip symmetric multiprocessing support for concurrent thread execution), SSI (streaming SIMD instructions for vectorization), and the implicit parallelism of deep instruction pipelining.
Most application programmers – those outside scientific and performance-critical computing, anyway – could ignore this for a while, because it was managed by CPUs, SMP-capable operating systems, and in limited ways, by compilers. But by 2004, when multiple cores debuted in the mainstream, the writing was on the wall. As C++ language expert Herb Sutter, long-time convener of the ISO C++ standards committee, wrote in a seminal 2004 article called “The Free Lunch is Over” (http://www.gotw.ca/publications/concurrency-ddj.htm), parallel programming was set to become – soon or late – a major preoccupation of mainstream developers.
Since then, core-counts have increased on servers and desktops. GPU cores have proliferated both in their own slots and increasingly, paired with multicore CPUs in SOC arrangements. And cloud computing, in the most elemental, Hardware-as-a-Service models, is coming on strong. In 2012, we can expect to see the last single-core bastions fall as multi-core chips and on-chip GPUs go mainstream on smartphones, tablets, and ultralight laptops.
Meanwhile, the new “next big thing” in hardware is exemplified by Intel’s MIC (Many Integrated Core) architecture, set for commercial release late this year. MIC-type designs exploit massively parallel arrays of (themselves, four-way hyperthreading) Xeon cores to achieve TeraFLOP performance goals at lower clock frequencies. But MIC also holds the promise of higher efficiency through on-chip networking, message-passing, heterogeneous node function, and dynamic workload allocation: functioning like a cloud datacenter on a chip. And this, in turn, would seem to presage a future – maybe five years off? – where a single massively parallel heterogeneous computing model governs from the chip on the device to the cloud the device connects to.
The inevitable conclusion is that developers need to learn how to leverage these advanced technologies, which means learning to use the tools that make this fast-expanding power accessible, and help keep software exploiting parallelism future-viable. Here at Go Parallel, expert contributors will weigh in on parallel programming’s major themes, from SIMD vectorization to threaded programming and beyond. We’ll look at pragmatic toolkits like Cilk Plus, which adds vectorization-specific array notations and other expressive tools to C/C++, and Intel Threading Building Blocks. We’ll look at parallel programming from a workflow perspective, sharing tips and best-practice advice for development, optimization, testing, and validation. We’ll talk about new processor architectures and what they mean for coding. And we’ll visit a host of projects, from advanced 3D gaming to scientific and supercomputing – both in business and academia – to meet the people pushing the envelope, and gain insight into their work.
Keep reading, try the tools, and join in the conversation.
John Jainschigg is a Geeknet contributing editor, and is CEO of World2Worlds, Inc., a digital agency focused on immersive technology and gaming. John’s initial intro to concurrency was via interrupt and re-entrancy programming at the assembler level on Z80 and 68000-based systems. He wrote concurrent, time-critical packet-switching applications on HP-UX RISC machines in the late 1980s, and since then has worked up and down the client-server stack in Java, C++, PHP, and other conventional and scripting languages, and more recently, in task-specific, state-based, radically concurrent languages like LSL.
