<p>I've been writing a lot recently about Parallel computing and programming and I do notice that there are a lot of patterns that come up when it comes to parallel computing. Noting that Microsoft already has released a library along with the Microsoft Visual C++ 2010 Community Technical Preview (named Parallel Patterns Library) I'm wondering what are the common parallel programming patterns you have been using and encountering that may be worth remembering? Do you have any idioms you follow and patterns that you seem to keep popping up as you write parallel programs with C++?</p>

First you have to chose between shared-memory computing, and shared-nothing computing. Shared memory is easier, but doesn't scale that well - you will use shared-nothing if you either
a) have a cluster, rather than a multiprocessor system, or
b) if you have many CPUs (say, > 60), and a high degree of non-uniform memory
For shared-memory, the common solution is to use threads; they are easy to understand as a concept, and easy to use in the API (but difficult to debug).
For shared-nothing, you use some kind of messaging. In high-performance computing, MPI is established as the messaging middleware.
You then also need to design an architecture for the parallel activities. The most common approach (again because it's easy to understand) is the farmer-worker-pattern (a.k.a. master-slave).