“For the beta, we are allowing access to Intel’s parallel development tools even if new Xeon Phi buyers don’t, currently, own them,” explained James Reinders, Intel’s Software Products Director and Multi-core Evangelist. “But a lot of workstation users already own Parallel Studio for Windows, so all they have to do to access the Xeon Phi is make sure they are using the latest version.”

The configuration of Intel’s Many-Integrated-Core (MIC) Testing Lab for Windows OS

Intel unveiled the Xeon Phi 60-core coprocessor last year as the first member of its many-integrated core (MIC) parallel processor architecture. Since then it has been marketing the Xeon Phi coprocessor almost exclusively to high-performance computer (HPC) users, who almost universally run the Linux operating system (OS).

“Today the Xeon Phi is mostly used by HPCs, which are already running highly parallel workloads in a Linux environment,” Reinders continued in an interview.” But now that Windows users are learning to take advantage of multi-core processors, they too can take advantage of the Xeon Phi’s massively parallel processors.”

With nearly a year of experience under its belt, Intel is confident its current beta-release is stable and that users will find they can get real work done using their Xeon Phi coprocessor with a Windows OS. In fact, Reinders estimates it will only be a few months before Intel is confident enough to do a general release of Parallel Studio XE for Windows.

“It is a beta, but we have a decade of experience in supporting Windows with our toolset, and we have gained a lot of experience supporting the Xeon Phi under Linux, which helped us in preparing the Windows release,” said Reinders. “However, we are releasing it as a beta to get feedback from users, who might even find a few bugs. Then probably this fall we will do a full release of our Xeon Phi toolset for Windows.”

Linux Inside Windows

The Xeon Phi co-processor card runs Linux–even when hosted on a Windows-based workstation or server–but the Windows user does not have to hassle with Linux, because it is downloaded to the card automatically when the user installs the device drivers for the Xeon Phi coprocessor. Thereafter, the programmer can offload any parallel program running on a Xeon host to the Xeon Phi coprocessor.

“The host Xeon processor can run Windows or Linux, but the Xeon Phi coprocessor card will still run Linux,” said Reinders. “However, our software allows Windows applications to off-load jobs to the coprocessor at any time.”

Intel expects Windows parallel programmers will make user of the new Xeon Phi coprocessor for applications in the fields of engineering, computer-aided design (CAD), virtual prototyping, modeling, stress analysis, life sciences, drug analysis, oil exploration, energy generation, financial analysis and big-data analytics.

Intel’s Parallel Studio XE  runs under Microsoft Windows 7 Enterprise SP1 (64-bit), Windows 8 Enterprise (64-bit), Windows Server 2008 R2 SP1 (64-bit) and Windows Server 2012 (64-bit). Reinders also claims that the programming techniques described in his new book, Intel Xeon Phi Coprocessor High-Performance Programming, can be used by Windows programmers because its examples are in C++ or Fortran.

Those who have experience with parallel programming know it’s mostly true with “traditional” approaches that incorporate parallel constructs into mainstream programming languages like C++ or FORTRAN. In light of decades of research in parallel computing, this is an irritating situation – and more so now that multi-core systems have been mainstream for years.  

You probably guessed already that I don’t think it needs to be that painful, but let’s defer that for the moment. So what it is that makes traditional approaches so painful?

Pain is Implicit

The pain is in what they require the programmer to think about and more importantly what they leave implicit. The programmer is required to explicitly state what computations should go in parallel.

However, the programmer must implicitly make sure that no race-conditions can occur; it is up to the developer to guarantee that the computations s/he declared as parallel do not depend on each other.

Without offering a way to specify/define dependences (if one computation depends on the other we say there is a dependence between the two) it is simply required (or assumed) that none exist. Prominent examples of tools for C/C++ that make you guarantee the non-existence of dependences are OpenMP, MPI, TBB and Cilk. Without becoming too philosophical,  it should be obvious that proving the non-existence of something is very hard (and in some cases impossible). Not uncommonly this approach brings programmers to the verge of despair by making them think about what doesn’t exist.

The Secret? Inversion

That said, the key to less pain with parallelism becomes apparent. It’s not in syntax, more powerful parallel constructs, automatic parallelization or new programming languages. As long as dependences are implicit, getting parallelism won’t become significantly simpler.

However, “inverting” the problem simplifies it: it’s much easier to tell what exists than proving what doesn’t. If we define what can *not* go in parallel, e.g. which (dependent) computations need to go in order, then we also know what can go in parallel: everything else.

So why not just letting the programmer define the dependencies (or the required orderings) between computations? It is a simple thought process; simple mostly because the necessary information is known by the developer anyway: when writing a computation kernel the programmer knows what input/information it needs, so it is clear what it depends on. If all such dependences are declared, executing non-dependent pieces in parallel is an almost straight forward task.

Make Relevant Information Explicit!

In other words, the key is making the relevant information explicit rather than tediously working around implicit information. Serial languages (like C/C++) hide such information even though it is known to the programmer. They have no capability to express the necessary information directly and natively. Tools for automatic parallelization then try to uncover what’s not defined explicitly – in general that’s a futile endeavor.

 So what’s needed is a program structure to explicitly express the dependences which are actually needed to find a semantically correct execution order of compute kernels. Even a runtime can take care of the parallelization and things like “parallel_for”, pipelining-constructs etc. become superfluous. Expressing the dependences avoids the requirement to make parallelism explicit or even to think about it but still exposes the available parallelism in the program.

Inverting the traditional approach of thinking about parallelism, e.g. thinking about what needs to be serial/ordered, makes parallelism not only easier but also uncovers more of it (because no explicit construct limits it to a specific type of parallelism).

There are different approaches to tackle this, like streaming and functional languages. In a series of articles I will introduce Intel’s C++ library/runtime implementation of CnC* which directly implements the above idea. Without specializing to a certain domain it provides purity in design and proven scalability from single-core to multi-cores systems up to clusters of multicore-workstations. I will demonstrate that the thought process is simple and even detecting errors is usually easy, because the runtime has the information to actually issue meaningful warnings and messages.

For the impatient: Here is CnC’s homepage with the free download, papers, talks and tutorials. 

http://software.intel.com/en-us/blogs/2013/01/31/about-the-pain-of-parallel-programming

Here’s how it usually happens: Neural networks are trained, and an application uses them to make decisions. Unfortunately, this overlooks the real-life fact that circumstances change, invalidating network training. What makes the situation particularly serious is that retraining a network takes time and may not be practical for real-time applications. But the parallel processing technology offers a solution that makes retraining neural networks in real time an option.

Two examples:

Imagine a robot scanning the ocean floor to gather geological data. It makes movement and navigation decisions based on trained neural networks. But if it encounters situations for which it was not trained, the robot’s behavior may be unpredictable, or downright wrong. If such a robot could retrain in real time, as do flesh and blood animals, it has a chance of surviving and completing the mission.

Let’s say that a financial trading program has neural networks that have been trained for all known trends. It predicts investment decisions based on the trained neural networks it contains. Here again, if a new trend surfaces, the predictions may be inaccurate and may result in a loss of investment capital. What this program needs is the ability to respond to new trends, and retrain itself so that predictions can be accurate.

Demo Program for a Smarter Solution

To illustrate a possible solution, I wrote a demonstration program using Intel parallel processing technology. The program generates four pre-calculated data sets that can be used to train and test neural networks. Four neural network objects are intended to dovetail with the parallel processing technology. The following methodology is used whenever a new data set is selected.

1. A derived data set for training is created. This is important because neural networks, which are trained on sequential data, do not learn as well as those, which are trained on data that has been reordered. A network trained by sequential (or ordered) data is said to be over-fitted, and the results are usually less than optimal.

2. Three neural networks are trained. The first is trained with only 100 training epochs so that it will be ready to use quickly. The reason for this is that in an authentic situation, even a poorly trained network is better than one that is trained for another purpose.

3. The two other networks are then trained with progressively more thorough training, each with greater amounts of training epochs. In this way, a network will eventually emerge that makes good decisions.

The Magic Sauce

In order to train the three networks without abruptly stopping the program is to spin up parallel threads. The program uses cilk_spawn. Using it could not be easier, the following code shows how to spin up the three training methods.

cilk_spawn TrainFirst();

cilk_spawn TrainSecond();

cilk_spawn TrainThird();

The three functions are extremely simple. They each train a neural network object, and then assign the values to a neural network object, which is being used to draw the predictions on the application. The first trains with 100 epochs, the second with 200 epochs, and the third with 300 epochs. When you run the application you can easily see the user interface reflect the status of each neural network.

Gotchas and Caveats

I discovered a few gotchas and caveats when writing the app. The major one is that you cannot put any user interface code within the function that cilk_spawn references. If you do, the function call will block, meaning that the calling code won’t get control back until the spawned method finishes (or I should say the allegedly spawned method). What I ended up doing was creating some semaphore variables, which are examined from an independent thread. When the semaphore variables are set, then the independent thread can make the user interface calls.

Another warning is in order. I originally wrote the application with three consecutive uses of cilk_spawn. While this worked, the CPU usage on my six-core development machine went to the point of being sluggish and almost non responsive. I would suggest that you experiment with spawning simultaneous threads that perform CPU intensive tasks. You might find that it is not an issue, but you should make sure you have thoroughly tested your application. My final solution was to wait until each training function finished before launching another training function.

If you use cilk_spawn from within a spawned thread, you won’t gain any benefit. It is as if you simply performed a normal function call from within the spawned thread.

The Results

The application (which can be downloaded here) allows users to select the data set, have the application retrain without missing a beat, and display a graphical representation as shown in Figure 1.

The application uses whichever neural network has the most thorough training to make predictions.

Now that I have explored neural networks within the parallel processing world, I am ready to look for new and innovative applications of neural networks. Now that real-time retraining is an option, the applications for neural networks have increased very dramatically.

For the last eight years, the Programming Without Coding Technology project has been working on ways of developing, well, programming without coding. The initiative’s General-Purpose Visual Programming tool is aimed for clue-free newbies, expert developers and everyone in between. Interesting, but doubly so because of how the effort might quicken adoption of parallel programming.  Read how.

First, a quick explanation.  Project head Mahmoud Fayed laid out the basics in a recent Source Forge guest post:

“PWCT is not a Wizard for creating your application in 1-2-3 steps… A novice programmer can use PWCT to learn programming concepts like Data Structure, Control Structure, Programming Paradigm..etc.  An expert programmer can use PWCT to develop large and/or complex software.”

He continues:

“Using PWCT, we developed a textual programming language Compiler and Virtual Machine without writing a single line of code where the programming process done using the PWCT visual components. This language called Supernova and it’s free-open source (http://supernova.sourceforge.net).

 Many database, multi-media, network, AI, simulation & math applications developed using PWCT.”

That sounded pretty interesting. So I contacted Fayed and asked what I hoped was not a dumb question: “Could the code produced by PWCT be parallelized?”

His rapid reply:

“Yes, PWCT comes with more than one VPL (Visual Programming Language) like HarbourPWCT, PythonPWCT, C#PWCT & SupernovaPWCT and you can extended PWCT to support any textual programming language.”

And you can …

1 – Use Threads

2 – Use the Super Server Programming Paradigm (Embedded in HarbourPWCT) to develop network applications (Client-Server & Distributed )

3 – Support new textual programming language for concurrent programming like ErLang.

“The idea of PWCT is to create new VPLs powerful as the textual programming languages we are using for programming tasks. 
And we have 11 lessons to help developers extend PWCT.”  You’ll find these at here. 

So… insofar as Go Parallel is devoted to helping people, well, go parallel, this seems like good news, right? After all, one of the problems with programming in general — and parallel programming in particular – is the lack of programmers.

So what if some basic, workaday programming could be handled by talented-enough non-specialists? Wouldn’t that free more expert developers to focus on higher-value tasks, including parallel programming?

Taking the idea further, what if the “commodity” programs created by non-experts could be easily parallelized? That would be a good thing too, right?

I frankly don’t know if trying to parallelize non-coded PWCT programs would be worth the trouble. Or what the quality would be. Or how/if they could be handed off into an Intel software environment for parallelization.

This last point is important not just because Intel sponsors the Go Parallel site, published in partnership with Slashdot Media. But also because the company’s parallel programming suite and tools represent the gold standard. That’s a fact that can’t be ignored.

Still, the whole idea raises provocative questions. A few weeks ago at the annual Davos Economic Forum, WWW Demigod Tim Berners-Lee issued a passionate call for ordinary users to become more literate in basic programming concepts and skills. 

PWCT seems to answer that call pretty well. Could it, or similiar approaches, end up climbing the evolutionary tree of programming and play (or adapt) well enough with deeply established apex companies like Intel? Or are we looking at a coding Cro-Magnon, at least where parallel is concerned?

I honestly don’t know the answer. I raise the question, because part of our site’s mission is “thought leadership. What’s your take? Please weigh in below. Not because I say so, but because Sir TB-L think it’s a topic worth whacking round.

Singleton instances have their place in programming, but can present a problem in multithreaded programming. You only want a single thread to create the initial instance and to perform the initialization. With the help of the atomic type in Threading Building Blocks, you can safely create your instance.

Last time we met up, we talked about the inner workings of the atomic template in Threading Building Blocks. That template helps you with some high-performance mutexes and adding and swapping by making use of some assembler-level operations. These operations use what’s called fencing to make such operations possible. The end result is that you can create fully thread-safe classes that, for example, include a usage count.

Quick Theory: Atomic Template

When might this type of template might be useful? If you want to have some initialization take place only after an object is accessed the first time, for example, then some cleanup when the object is no longer used—all while being multi-thread, multi-core safe. The good news is that you don’t have to write a single line of assembler code. The template already has done that part for you.

Without such a template, you might use operating-system level mutexes. But then you end up taking a performance hit. By letting the template work at the assembler level, time spent is minimal.

Here’s the idea: When you need to increment a value, you want to grab the current value, add one to it, and store it back in. Without any kind of mutex in place, two competing threads might both grab the current value at almost the exact same time, thus getting the same value. They both add one to it, and store the new value back in. So instead of incrementing by two as it should, the value only increments by one, resulting in a bug. At the assembler level, the processor allows a single thread to grab the value, add on to it, and write it back on as an atomic operation (hence the name of the template, atomic). This fencing happens extremely fast, so the wait time for other threads is very minimal. That’s far more efficient than creating a mutex at the operating-system level.

And Quick Practice…

Let’s try it out. Remember, we’re dealing with templates, so we’ll be using the template as a starting point for our own class. We’ll create a singleton class. The first time the instance is needed, we’ll create the instance. To keep this simple, we won’t do a decrement or cleanup; feel free to try that part yourself and discuss it in the comments, and we’ll look at in the future if there’s interest.

Here’s some code that does it. This isn’t the only approach, but it’s one way that works:

class Singleton
{
    static tbb::atomic<Singleton *> inst;
    int x;
    int y;
    Singleton(): x(10), y(20) { }
public:
    int getX() {
        return x;
    }
    int getY() {
        return y;
    }
    static Singleton &getInst() {
        if (inst == 0) {
            Singleton* temp = new Singleton();
            if (inst.compare_and_swap(temp, NULL) != NULL) {
                delete temp;
            }
        }
        return *inst;
    }
};
tbb::atomic<Singleton *> Singleton::inst;
 
int _tmain(int argc, _TCHAR* argv[])
{
 
    cilk_for(int i=0; i<8; i++) {
        Singleton& s = Singleton.getInst();
        cout << s.getX();
    }
}

Before diving into this code, I need to point out something about static initialization. In C++, the static line inside the class defines the member, but doesn’t actually declare the member. (In other words, it sets up a name for it, but doesn’t create the storage for it.) You create the storage for it after the class with a line like this:

tbb::atomic<Singleton *> Singleton::inst;

The C++ compiler will then allocate the storage. But we have a bit of a problem. I’d like to initialize this, but I can’t. The reason is the static initialization treats the initialization as a call to a copy constructor (which doesn’t exist), instead of using the overloaded = operator to assign the value.

The solution is a bit odd, but it’s the officially sanctioned solution from Intel. The compiler initializes the data to 0, so that’s what we’ll go with. In fact, that’s exactly what we want, an initial value of 0 (which is the same as NULL). So it works as is, without us initializing the data. It seems a little odd, and for purists it might seem outright dangerous, but we’re fine. We have a guarantee from Intel that their compiler will initialize it to 0.

Now for the important part of the code. If the instance exists, we just return it in the getInst function. If it doesn’t exist, we attempt to create it. This is where the thread-safe part comes in. We don’t just create it and shove it into the private inst variable, because there’s a slight possibility that another thread could be doing the same thing at the same time. Instead, we create an instance and save it in a temporary variable. Then in a single atomic operation, we check if the private inst variable is 0, and if so, save the new instance into the variable. We use the compare_and_swap function to do that. In the event another thread managed to create the instance in between the time it took to check if the inst variable is 0, and when we run the compare_and_swap, we’ll be safe; that other thread did it in an atomic operation, and basically beat us to it. So in that case, we’ll get back a non-NULL value, in which case we just delete the temporary variable.

This means you won’t want to do any initialization in the constructor, of course; put that in a separate function, add an else statement and do it there. 

In the end, you’ll have a singleton instance that you can access in your threads with only a single thread guaranteed to create the final instance used.

Learn more here. 

Download the Intel® Software Development Kit (SDK) for OpenCL Applications here. 

Threading Building Blocks includes a template class called atomic that helps you accomplish some atomic-level tasks. Under the hood these tasks are actually implemented at the processor level using assembly language. But you don’t need to do any assembly programming yourself. Let’s take a look at this interesting class. 

Buried down inside the Threading Building Blocks documentation is a template class called atomic. At first glance it appears simple  and nothing special, providing “read, write, fetch-and-add, fetch-and-store, and compare-and-swap.” Not much there, but in fact, it’s these are useful and highly optimized.

In my article about performing lazy initialization, I briefly mentioned this little template. Let’s take a closer look at it and see why it’s optimized for parallel programming.

Fencing Memory is Key

At the heart of the atomic type is a technique called memory fencing. There are a lot of highly technical explanations across the web of what exactly memory fencing is. But for a quick explanation, just think of it as a forced ordering. Two operations run in separate cores, with one putting up a fence to block other operations from accessing memory until the first thread is finished. Typically this means doing a couple of operations that must happen together as a single, atomic operation. For example, a thread might need to read the contents of some memory, add one to it, and write it back into memory, with no other threads making the change. This can actually be accomplished at the assembler-level, essentially implementing a very low-level mutex.

Threading Building Blocks uses a minimal amount of assembly language to accomplish parallel tasks. The atomic template is one such place, making use of fencing features of the Intel processor. Down in the source code are a few directories with processor-specific assembly code for accomplishing fences at the assembler level. The header file atomic.h includes several macros that make calls to this assembly code. What this means is that the code used in the atomic template is about as optimized as it can get. So I encourage you to make use of this deceptively simple template. 

The atomic template includes a member function called compare_and_swap. I briefly mentioned this function in the article about lazy initialization. Here’s more of the story:

compare_and_swap

When you look at the lazy initialization algorithms, specifically the one that doesn’t use mutexes, you might get a bit concerned that by not using mutexes, you could still end up with a race condition. In fact, you don’t, because template’s compare_and_swap function makes use of the assembly code that implements the fencing. (Check out the other blog for more info on what exactly compare_and_swap does.)

Intel processors provide built-in support for fencing through the use of a LOCK opcode that precedes certain other opcodes, including one called CMPCHG. This CMPCHG opcode does exactly what the compare_and_swap function does, but at the assembly level. TBB uses this opcode and adds in the LOCK instruction so that the compare_and_swap member function runs at the assembly level with a single instruction. And the LOCK opcode provides the hardware-level mutex we need. I’d say that’s pretty optimized! And it has minimal—if any—performance bottlenecks.

fetch_and_add

Another function the atomic template provides is fetch_and_add. This is also implemented at the assembly level with a LOCK instruction, and with a single instruction with two opcodes that does the work.

The idea behind fetch_and_add is to retrieve the value of a variable, add something to it, and store it back in. This is important in parallel processing because you want the whole thing to happen without another thread intervening before the task is complete.

For example, if two threads are using the same variable, and both threads need to add to the variable, you want the final value to be the original value plus both added-on values. But if one thread grabs the value, and the other thread grabs it too, before the first thread can write the new value back, then the final value will be wrong.

A common example of this is in reference counting. When a thread needs to access a variable, and the variable needs to keep a reference count by adding one to an internal integer, you don’t want a race condition to cause the increment to be wrong. The great thing here is that the processor can handle this at the assembler level, meaning it’s fast and easy to do. That’s how the fetch_and_add function works, and why it’s a key part of the atomic template. Again, this one also uses the LOCK opcode to implement a mutex of sort. 

Assembly Level Locking = Great Thread Performance

Next time we’ll look at some actual code examples of the atomic template, and how you can implement various techniques such as a reference counter. Since the atomic template makes uses of assembly level locking, it’s efficient and has excellent thread performance. And since it’s a template, you can use it to build your own types, resulting in code that’s both easy to use and highly optimized—all within a parallel environment. 

Intel graphics SW aces will describe how to apply the heterogeneous parallel compute capabilities of standardized OpenCL into applications.

Intel OpenCL lets applications utilize both CPU and GPU power while balancing the work load between devices for better performance. At the same time, advanced OpenGL and DirectX interoperability extensions allow instant data sharing between rendering and compute domains, with efficient synchronization techniques. Intel OpenCL SDK provides various samples to learn from. Adding OpenCL code to an application gets even easier with CLU library.

We’ll resist the temptation to say “ make the scene.” But you should. If you can’t stay tuned here for session materials.

Explore the range of Intel OpenCL tools here: http://software.intel.com/en-us/vcsource/tools/opencl-sdk

This article takes a look at how to build an effective on-the-job training curriculum for new software engineers and developers. But be warned: one size does not fit all. As you design new training programs, you must take into account the skill level and background for each candidate.

Read the full article here: http://slashdot.org/topic/bi/training-software-engineers-start-with-a-plan/

About the Author

Catherine has spent the last ten years working throughout engineering, including development, test, support, and product management. She focuses on agile team management and effective software delivery, building high-performance multi-functional teams that work effectively with business needs. Catherine’s projects also include non-dogmatic agile training for teams just starting up or looking to move to agile methods. Past experience includes an enterprise storage system, a tablet solution for restaurants, a mobile data synchronization platform, a marketing analytics platform, and several web-based applications.