Parallel Programming: Understanding Shared Memory Model vs Message Passing Model, Slides of Parallel Computing and Programming

Download Parallel Programming: Understanding Shared Memory Model vs Message Passing Model and more Slides Parallel Computing and Programming in PDF only on Docsity! Parallel Programming * Shared Memory Model vs Message Passing model * MIMD platforms Prog Models - MPMD vs SPMD docsity.com 1.3 • Sequential programming language and ask parallelizing compiler to convert it into parallel executable code. (Implicit or explicit) • Sequential programming language with added syntax to declare shared variables and specify parallelism. Example UPC (Unified Parallel C) - needs a UPC compiler.(explicit) • Parallel programming language with syntax to express parallelism - compiler creates executable code for each processor. (explicit) Programming Parallel Machines docsity.com Message Passing Programming: Generic Program Concepts • Send & Receive Primitive • Blocked Send/ Receive: Synchronous • Send/ Receive: Asynchronous docsity.com Example: Array Addition on 2-processor Message Passing Parallel Computer docsity.com Two-processor addition of array Program for Processor0 Array A[1:N]; begin for i :=1toN do Read A[i] end for; Send(1,A[(N/2 + 1] ,N/2) Sum0O := 0; for i := 1 toN/2 do Sum0O := Sum0 + A[i] end for; Receive(1, Suml1, 1); Sum := Sum0O + Suml1; Write Sum Program for Processor! Array B[1:N/2]; begin Receive (0,B[1) ,N/2); Suml := 0; for i := 1 to N/2 do Suml := Suml + B[i] end for; Send(0, Suml, 1) end. docsity.com SPMD program for addition of array begin Sum_local, sum_rcvd, increment; Array A[1:N]; if procid = 0 then for i:=1 to N do Read A[i] end for end if increment := N div p docsity.com if procid = 0 then for j:=1 to (p-1) do Send (j, A{j*increment +1], increment) end for else Receive (0, A[1], increment) end if sum local := 0; for i:=1 to increment do sum local := sum local + Ali] end for ® docsity.com if procid # 0 then Send (0, sum local, 1) else for j:=1 to p-1 do Receive (j, sum rcvd, 1); Sum-local := sum_local + sum_revd endfor _ Write sum local end if end, ® docsity.com Program for Processor(p-1) Array B[1:last_inc]; begin Receive (0, B[1], last inc); Sum[(p-1] :=0; for i :=1 to last inc do Sum[p-1] := Sum[p-1] + B[i] end for; Send (0, Sum[p-1], 1) end. docsity.com (N > p) & (N NOT div by p) docsity.com LE procid = 0 Met ‘for j:=1 to (p-2) do | Send (j, A{j*increment +1], increment) ‘end for ‘Send (p-1, Al(p-1) *increnent + 1}, increment + 3 mod pl Receive (0, A{1], increment) end if sum local :=0; for i:=1 to increment do sum local := sum local + Ali] end for ® docsity.com