Communication Models
Shared Memory Space Model
- Threads communicate by reading/writing to shared variable
- Shared variable are like a big whiteboard that everyone can see and write on it
Uniform Memory Access (UMA)
Every thread (or processor) can access every memory space with the same latency
Non-Uniform Memory Access (NUMA)
Any processor can directly reference to any memory location, but the cost of memory access is different for different processors
more scalable
Distributed Memory
Concept
- Each processor has its own local memory space
- Programmer can specify which processors and when should they communicate to exchange information
- Each processor is connected with a network (i.e. Ethernet)
Message Passing Model
- Thread operates within their own private address space
- Threads communicates each other by sending/receiving message by protocols