General Model of Communication
Terminology
Overhead
Time spent by the sender’s processor on communication tasks
Occupancy
Time for data pass through the slowest component of the system
Message Communication
Inherent Communication
Concepts
- Some algorithm simply need to share data between processes to compute the correct results
- This told us “if the hardware is perfect and unlimited, how much communication does this algorithm needs”
Communication-to-Computation Ratio
- “Arithmetic Intensity” = 1 / communication-to-computation ratio
- We want this ratio to be as low as possible
Artifactual Communication
Concepts
- All the other communication because of practical system limitation
Examples
- System might have minimum granularity of transfer
- System must communicate more data than we need to transfer
- Additional communication overhead
- Poor placement of data in distributed memories
- Process 0 frequently accessing certain array but it is stored in Process 1’s memory
- Finite capacity
- Cache is too small, we can’t store all the required data on cache