Download High Performance Computing Lecture 1 - Lecture Slides | CMSC 714 and more Study notes Computer Science in PDF only on Docsity! 1 CMSC 714 High Performance Computing Lecture 1- Introduction http://www.cs.umd.edu/class/fall2005/cmsc714 Alan Sussman 2CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth Introduction Class is an introduction to parallel computing – topics include: hardware, applications, compilers, system software, and tools Counts for Masters/PhD Comp Credit Work required – small programming assignments (two) - MPI/OpenMP – midterm – classroom participation • Everyone will have to present (1 or 2) papers for 1 class • For each class with readings, everyone has to send me one question to discuss in class – group project (2-3 students per group) Will be asked to participate in a study of parallel programming 2 3CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth What is Parallel Computing? Does it include: – super-scalar processing (more than one instruction at once)? – client/server computing? • what if RPC calls are non-blocking? – vector processing (same instruction to several values)? – collection of PC’s not connected to a (fast) network? For this class, parallel computing requires: – more than one processing element – nodes connected to a communication network – nodes working together to solve a single problem 4CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth Why Parallelism Speed – need to get results faster than possible with sequential • a weather forecast that is late is useless – could come from • more processing elements (P.E.’s) • more memory (or cache) • more disks Cost: cheaper to buy many smaller machines – this is only relatively recently true due to • VLSI • commodity parts 5 9CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth SPMD Processors Program Counter Program Counter Program Counter Program Program Program Program 10CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth I2 I3I1 Dataflow instruction instruction I4 6 11CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth Communication Networks Connect – PE’s, memory, I/O Key Performance Issues – latency: time for first byte – throughput: average bytes/second Possible Topologies – bus - simple, but doesn’t scale – ring - orders delivery of messages PE MEM MEM MEM PE PE MEM PE 12CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth Topologies (cont) – tree - needs to increase bandwidth near the top PE PE PEPE PE PE PE PE PE PE PEPE PE PE PE PE PE PE PEPE PEPE PEPE –mesh - two or three dimensions –hypercube - needs a power of number of nodes 7 13CMSC 714 – F05 Alan Sussman & Jeffrey K. Hollingsworth Memory Systems Key Performance Issues – latency: time for first byte – throughput: average bytes/second Design Issues – Where is the memory • divided among each node • centrally located (on communication network) – Access by processors • can all processors get to all memory? • is the access time uniform? – UMA vs. NUMA
