In parallel programming, multiple calculations are executed simultaneously. The computations are typically CPU-intensive and perform either different operations on the same data set or perform the same operation on parts of a data set.
- To be run using multiple processors
- A problem is broken into discrete parts that can be solved concurrently
- Each part is further broken down to a series of instructions
- Instructions from each part execute simultaneously on different processors
- An overall control/coordination mechanism is employed
In concurrent programming, programs are designed as computations that interact. The computations may be executed on multiple threads, but also using just a single thread.
- Multiple threads of control
A number of sequential programs programs work together to achieve a goal.
- Inter-process Communication
Shared variables and Message passing
- Mutual exclusion
processes must execute their critical sections one at a time.
- Conditional synchronization
processes wait until a condition is true.
The term asynchronous is used when describing I/O operations that do not block a thread while waiting for completion.
It's common to look at asynchronous processes as non-blocking (the system won't lock up, will continue). Asynchronous programming was typically implemented using callbacks or events.