Moore’s Law correctly predicted that processor transistor density would increase year after year. All those extra transistors have been used to make powerful hardware capabilities like multiple cores and hardware extensions that improve performance. Intel processors have extensions that support SIMD (single instruction, multiple data) parallelism with Intel® SSE and Intel® AVX(2). These instructions operate on a vector of data in parallel. The vector width, and therefore the number of elements that can be accessed in parallel, continues to expand as new processor technologies are introduced. Applications need to be vectorized to take advantage of SIMD instructions that utilize the expanded vector width.

All those extra transistors have been used to make powerful hardware capabilities like multiple cores and hardware extensions that improve performance. Intel processors have extensions that support SIMD (single instruction, multiple data) parallelism with Intel® SSE and Intel® AVX(2). These instructions operate on a vector of data in parallel.

There are some pretty easy ways to vectorize your code, such as  autovectorization, that requires no changes to code. Next are using libraries that utilize both threading and vectorization to improve performance. For some applications, more advanced techniques may be needed to provide information to the compiler, including using special build logs and features such as Intel® Cilk™ Plus.

A great place to start is by downloading the Vectorization CodeBook. You will find simple, yet powerful vectorization techniques that can be used by just about any application developer using Intel® Compilers and libraries.

Learn lots more about vectorization at this great resource site http://software.intel.com/en-us/intel-vectorization-tools/

Download Intel Inspector XE 2013 one-sheeter here (pdf.)

While it’s important to master the tools used in parallel development such as Parallel Composer, Parallel Amplifier and Parallel Inspector, it’s good now and then to step back and let the work of others inspire your work.  Today we’ll see how a company called Flow Science used the full Intel® Cluster Studio XE tool suite to enhance the serial and parallel performance of its FLOW-3D app.   

Beyond Theory and Technique 

In parallel development, verification is important. While developing parallel algorithms might at first seem easy, it’s skill that must be honed. Intel Parallel Studio helps out by making suggestions and verifying your parallel code is correct.

Here on Go Parallel, we’ve provided lots of information about using various tools to perform verification at different levels. For example, in this blog I showed how to use Parallel Amplifier to do verification, even though most people think of using Parallel Inspector. Both tools can be used for different aspects of the development process. Inspector is usually used for verification and memory leaks, Amplifier for other types of verification.

It’s easy to get wrapped up in theory and technique. But how well do these tools perform in the real world? And how can we get new ideas about ways to maximize our development efforts.

When a big company like Intel publishes case studies, they’re usually targeted at management. As a part-time writer, I find myself reading a lot of case studies to gather facts and figures for articles. But as a full-time software engineer, I’ve found that reading cases really helps my own development.   

So let’s look at a real case focusing on parallel verification. We’ll start with a look at the basic problems. (Impatient types can skip to the next section.)

The challenge: Fluid Dynamics and Parallel Programming

If you’ve taken any physics classes or, better, are fortunate to work in a physics laboratory, you’ve learned about fluid dynamics, the study of how fluids flow. But more specifically, it’s about how fluids flow through and around things, relevant to many fields including hydraulics, aerodynamics, and even climatology and meteorology (since the Earth’s atmosphere is a gaseous fluid). The mathematics is pretty amazing, even using the “partial derivatives” we suffered in Calculus class.

Typically fluid dynamics occurs in three dimensions. Modeling such systems takes advanced computing power. Consider, for example, a weather system. Depending on the method of calculation (because Earth’s atmosphere doesn’t end abruptly), the effective height of the atmosphere is 8.2 kilometers. For the U.S. alone, that means the atmosphere takes up about 80 million cubic kilometers. Suppose you want to model not just the entire U.S. atmosphere, but the motions at point at every cubic meter. You’re dealing with an enormous amount of data. The points alone are 8.0 x 10^¹⁶ data positions, and that’s just positions. Add to that complete calculations at each point. It’s pretty obvious you’ll need more than a simple quad-core processor for this kind of work.

Similarly, if you’re dealing with a small enclosure with fluid flowing say 1 meter wide, deep, and tall. You’re dealing with 1 million date points if you’re only looking at the millimeter level. Switch to the micron, which is one millionth of a meter, and you’re dealing with 1×10^²⁷ data points.

The calculations in fluid dynamics are hard enough. But now take the established algorithms and split them out to run in parallel across 61 cores or more. Using full vectorization on each core, then things get messy very quickly.  But persist and you’ll get code that works. But then we must ask: Does it use reduction correctly? Does it accurately pull in all data for the final results?

Flow Science: Masterful Use of Verification Tools

Now we look to the real world answer. A company called Flow Science Flow, which supports a worldwide customer base of commercial, academic, and government users, has created a fluid dynamics application using Parallel Studio (see the full case study here). Flow Science used the full Intel® Cluster Studio XE tool suite to enhance the serial and parallel performance of its FLOW-3D app. 

Although at Go Parallel we usually focus on the C++ Compiler, Flow Science used the Intel Fortran Compiler. Parallel Studio’s verification tools were key. Building such a product was certainly no easy task, no matter how large the team of programmers and engineers. The code has to be right, and there are only so many tests you can perform. That’s where Parallel Inspector and Parallel Amplifier both come to save the day.

Challenge: The company’s customers face ever-larger data sets. And they continue
 to demand accurate solutions in less time. Moreover, introduction of multicore architecture makes parallelization difficult, since the computational load keeps changing throughout the simulation.

So the initial challenge was to maintain the accuracy and consistency of results, while greatly reducing simulation time. The team decided to extend the shared memory parallelism of FLOW-3D to a hybrid MPI*-OpenMP* version.

They soon discovered that introducing a distributed memory approach made debugging difficult. Furthermore, once errors were corrected, obtaining good speedups or scaling on a higher number of cores was difficult. So the next challenge was addressing scalability and parallel performance.

Developers used the Intel® MPI Library to enable distributed memory performance. By just switching runtime environment variables, company engineers and our customers have been able to achieve maximum interconnect performance. This feature enabled Flow Science to provide the Intel MPI runtime toolkit as part of the user installation for a seamless user experience.

For Flow Science, the primary benefit of using the Intel Cluster Studio XE suite was improved customer satisfaction due to better speedups for larger, more complex problems. Other benefits included reduced development time and costs. Bottom line: Flow Science enabled faster, more accurate simulation with Intel® software development tools, delivering improved results even as customer data sets grow larger.

Conclusion

I doubt every the programmer on the fluid dynamics project had gigantic Xeon Phi processors for their desktop development machines, but that’s okay. With Parallel Studio, you can develop and debug on a simple quad-core, and then let the runtime automatically scale your program up to a Many Integrated Core (MIC) architecture. And with the help of the correct verification tests on your machine, you can be assured your code is correct. For real.

But what about “Benign” data races. Are they actually harmful? How do you suppress them?

Consider this “innocent” code:

int op_count;
...
op_count++;  // Executed by several threads, it’s OK if it’s not 100% precise.

To learn what could go wrong and what you can do, read the rest of the post here…

Read more here.

Key tips for programming a high degree of parallelism, while using familiar programming methods and the latest Intel® tools supporting the Intel® Xeon Phi™ coprocessor.

Read the complete posting here. 

Read the full article from Intel’s Parallel Universe magazine here.

.NET developers can boost application performance and increase the code quality and reliability needed for high-performance computing and enterprise applications with the help of tools like Intel® Inspector XE and Intel® VTune™ Amplifier XE.

Read the full article from Intel’s Parallel Universe magazine here.

You can also visit The Parallel Universe archive to access past issues here.

Let’s suppose we have an algorithm that’s going to do a character-by-character comparison of two strings of the same length. It will build a third string of the same length as the other two, and fill in the letter T where the first two strings match, and a letter F where the first two strings differ. For example, given these two strings:

“parallel”
“paxallxl”

the function will detect differences at the third and seventh positions, and generate the string

“TTFTTTFT”

A function like this is certainly a good candidate for a parallelism. Here’s a first take of the function in serial form:

std::string finddiffs(std::string s1, std::string s2) {
 if (s1.length() != s2.length()) {
 return "";
 }
 
 int len = s1.length();
 std::cout << len << std::endl;
 char * x = new char[len + 1];
 for (int i=0; i<len; i++) {
 if (s1[i] == s2[i]) {
 x[i] = 'T';
 }
 else {
 x[i] = 'F';
 }
 }
 x[len] = '\0';
 
 std::string result = std::string(x);
 delete[] x;
 return result;
} 

The function takes two strings as parameters. It checks that their lengths are the same; if not, it just returns an empty string. Next, it allocates an array of characters of the size in question. It begins looping through the two strings, comparing character-by-character. With each iteration, if the two strings have matching characters, it fills a T in the target array; otherwise it fills in an F. It then adds a null-terminator ‘\0’ to the end of the array, and finally creates a new std::string instance from the array.

I wanted to find out how long a function like this takes before creating the parallel version. To do that, I used the timing tools in Parallel Amplifier. Because I wanted to really test it out, I made two enormous strings (114 million characters each). After running the code, Parallel Amplifier told me the function took 333 milliseconds to run.

Converting this to a parallel function is easy; initially all we need to do is try changing our for loop to a cilk_for, like so:

std::string finddiffs2(std::string s1, std::string s2) {
 if (s1.length() != s2.length()) {
 return "";
 }
 
 int len = s1.length();
 std::cout << len << std::endl;
 char * x = new char[len + 1];
 cilk_for (int i=0; i<len; i++) {
 if (s1[i] == s2[i]) {
 x[i] = 'T';
 }
 else {
 x[i] = 'F';
 }
 }
 x[len] = '\0';
 
 std::string result = std::string(x);
 delete[] x;
 return result;
}

Running the code again under Parallel Amplifier with the same 114-million-character strings on a processor with four cores shows an improvement. The time has been reduced to 77 milliseconds. But is this code correct? I used Parallel Inspector’s Inspect Threading Errors function, with Analysis Time set all the way to the top. The test seemed to go okay, except that a warning appeared:

Warning: Cross-thread stack access

A cross-thread stack access warning occurs when your threads are writing to data owned by other threads. We have exactly that situation happening here; we’re writing to the array that was allocated. This can sometimes be a problem, but not necessarily. It can especially be a problem if a loop relies on the results of previous loop, where “previous” refers to the previous one when running serially. In parallel, there’s no guarantee on the order that your loops run.

For example, if we have an array like we do in our example, but each thread reads the value of a different slot in the array, and that slot was calculated from a previous loop serially, then we would have a problem when running in parallel.

The fundamental test is this: Can the loops be run in a different order and still give you the same results? In our case, even though we’re sharing data among the threads (resulting in the warning), we can see that we’re not relying on other iterations. As such, we can run any iteration of our loop in any order and get the same results. And that means we can safely ignore this warning. (It is, after all, just a warning, not an error.)

In summary, we have successfully converted our serial algorithm to parallel, and we received an improvement in performance.

Have you run into similar verification situations when coding? Please tell me about it in the Comments section below.

 _______________________________________________________________

Jeff Cogswell is a Geeknet contributing editor, and is the author of several tech books including C++ All-In-One Desk Reference For Dummies, C++ Cookbook, and Designing Highly Useable Software. A software engineer for over 20 years, Jeff has written extensively on many different development topics. An expert in C++ and JavaScript, he has experience starting from low-level C development on Linux, up through modern web development in JavaScript and jQuery, PHP, and ASP.NET MVC. 

Applications employing several types of parallelism are intrinsically very complex, and can be hard to debug. So Intel offers tools like Intel Inspector XE – an upgrade to Thread Checker – that are designed for use early in development cycles, to analyze source code and resolve threading and memory errors – both predictively and in early, small-scale test runs. At the node level, tuning can be improved, and issues quickly resolved with Intel VTune Amplifier XE, which does profiling, hotspot identification, and lets you drill down to graphically and numerically assess locks and waits, thread balance, and transitions, enabling resolution to single lines of code and individual functions. For performing similar graphically enhanced analysis of applications on the target cluster, Intel has also introduced a tool called Intel Trace Analyzer & Collector, which instruments and traces execution across clustered machines, monitors all the MPI traffic, and offers a wide range of drill-down tools to help fix bugs, identify bottlenecks, and optimize MPI pattern choices (e.g. blocking, non-blocking, etc.) for best performance.

For some mission critical applications, though – especially those aimed at healthcare, scientific, aerospace, military, and other markets where software faults can be catastrophic – further assurance is required, both that programs have been written according to specifications, and that they will perform reliably given all combinations of input data and conditions. This is the domain of “V&V” – Validation and Verification. While a V&V process exists around any coding project, ramping up criticality and scale necessitates more deliberate, formal, and reliable treatment.

Eventually – typically sooner in the domain of hardware than in software, at least today – you enter the zone of “formal verification:” a mathematical and logical discipline used in abstracting and describing systems (hardware, software, etc.) in ways that can be verified by theorem-proving. In effect, the program (or other system under test) becomes a theorem predicting its end-states, and proving this theorem complete and correct means the program works as advertised. A reasonable place to start exploring this area is a presentation by Intel’s John Harrison (http://www.cl.cam.ac.uk/~jrh13/slides/lics-22jun03.pdf) , delivered in 2003 at the Ottawa LICS conference, which shows clearly why programs can be hard to test, why assumptions about them may be very wrong, why testing with normal inputs provides no assurance that any other input won’t break the software, and why formal verification is thus useful. The presentation goes remarkably deeply (and with virtually no math) into explaining why formal verification is hard, and how it can be made easier.

One way Intel and others make it easier is by using automated and interactive theorem-proving via tools like HOL (which stands for Higher Order Logic) Light, a proof-checker written in CAML (Categorical Abstract Machine Language), a derivative of Standard Machine Language. CAML and its sibling MLs are functional languages that help you map formal logic into code and treat programs as proofs – the underlying notion that programs actually are proofs articulated as the famous Curry-Howard-Lambek correspondence, showing the three-way isomorphism among intuitionistic logic, typed lambda calculus, and Cartesian closed categories. CH correspondence has also recently been recommended as a method for partitioning search-spaces explored by genetic algorithms, where each node (called a “species”) is indexed by its Curry-Howard isomorphic proof. Now hold that thought.

The trouble is that – as software and the parallel components and systems it runs on get individually denser and more numerous – all this austere and beautiful math gets bigger and harder for machines to model and exhaustively verify, given finite resources and time, and with the understanding that most formal verification is carried out interactively, with highly trained mathematicians tweaking the theorem-proving software every step of the way. This is a huge problem for organizations like the military, who want to develop and run perfectly reliable systems at epic scales.

So, late last year, DARPA initiated a program, and created a community around research into CSFV – Crowd Sourced Formal Verification. Basically, the DARPA CSFV program is looking for ways to take units of software, turn them into collections of theorem templates in a logical calculus, then programmatically create games based on these templates, have people play these games to explore the search space and statistically prove the individual theorems, then compile back the results of many plays to make progress towards verifying the program property under evaluation. If it works for protein-folding, why not for software?

If you’re interested in joining the effort, check out the program homepage at http://www.darpa.mil/Our_Work/I2O/Programs/Crowd_Sourced_Formal_Verification_(CSFV).aspx. The Proposer’s Day Briefing PDF by program manager Dr. Drew Dean of DARPA provides a full explanation of the program, its goals, and the range of skills and capabilities required to participate (plenty of room in this program, by the way, for people who aren’t Turing Fellows – they also need webdevs and cool kids to make these games fun, after all). Anyone qualified can join the program’s online community.

Meanwhile, folks with more of a hunger to grasp the mathematical and logical connections between game theory and problems in formal verification should check out the elegant and erudite review modules (http://pub.ist.ac.at/gametheory/) for the course in Game Theory in Formal Verification, taught in 2010 by Krishnendu Chatterjee at the new Institute for Science and Technology in Klosterneuburg, Austria.

Do you have an interesting verification anecdote? I’d love to hear about it so please let me know in the Comments section below.

 ______________________________________________________________

John Jainschigg is a Geeknet contributing editor, and is CEO of World2Worlds, Inc., a digital agency focused on immersive technology and gaming. John’s initial intro to concurrency was via interrupt and re-entrancy programming at the assembler level on Z80 and 68000-based systems. He wrote concurrent, time-critical packet-switching applications on HP-UX RISC machines in the late 1980s, and since then has worked up and down the client-server stack in Java, C++, PHP, and other conventional and scripting languages, and more recently, in task-specific, state-based, radically concurrent languages like LSL.