Parallel, Concurrent and Asynchronous Programming

Parallel Programming

In parallel programming, multiple calculations are executed simultaneously. The computations are typically CPU-intensive and perform either different operations on the same data set or perform the same operation on parts of a data set.

• To be run using multiple processors
• A problem is broken into discrete parts that can be solved concurrently
• Each part is further broken down to a series of instructions
• Instructions from each part execute simultaneously on different processors
• An overall control/coordination mechanism is employed

Concurrent Programming

In concurrent programming, programs are designed as computations that interact. The computations may be executed on multiple threads, but also using just a single thread.

• Multiple threads of control

A number of sequential programs programs work together to achieve a goal.

• Inter-process Communication

Shared variables and Message passing

• Synchronization

1. Mutual exclusion

processes must execute their critical sections one at a time.

1. Conditional synchronization

processes wait until a condition is true.

Asynchronous Programming

The term asynchronous is used when describing I/O operations that do not block a thread while waiting for completion.

It's common to look at asynchronous processes as non-blocking (the system won't lock up, will continue). Asynchronous programming was typically implemented using callbacks or events.