Σχόλια 0

Το κείμενο του εγγράφου

Before starting to work on parallel programming, we need to setup a working environment. Our labcomputers have this environment, but student may need to work at home with the same settings. Wepreferred to use Linux Operating System on our computer. If you are familiar with linux, you maychoose your preferred distribution. If not, we advice you install Ubuntu Desktop 10.10,

whoseinstallation CD's are distributed in the lab. Linux Operating System is completely Open Source and isfree of charge. You can even use it with code modifications. In this tutorial we will explain how toinstall needed softwares on Ubuntu Desktop distribution of GNU/Linux Operating System.

If you already have linux installed on your computer, you may skip to “Installing OpenMPI” section.

Installing Ubuntu Desktop

We advice you use Ubuntu Documentation about the installation. For simplest installation guide goto download page of ubuntu (www.ubuntu.com) and at the buttom of page there is 4th

item saying“Install it!”, click the “Show me how” button there. You will see the simplest tutorial there. But beforedoing this read the below first.

You can also find a good enhanced tutorial at

http://www.ubuntuguide.org

NOTE : Before installation, you are advised to take full backup of your computer.

On our lab computer, we have dual boot mechanism, so that both Windows and Linux are installedcan be booted from the same computer. If you would like to create the same configuration, your firststep must be cleaning up your disk (remove unnecessary files), and defragmenting the disc. This iscrutial because usually windows uses whole of your computer discs and we need to “shrunk” inorder to create free space for linux OS.

After than, insert your Ubuntu CD and follow up the installation tutorial to install it.

The initial Ubuntu installation do not contain C++ and Fortran compilers. So the next step will beinstalling C++ and Fortran compiler. But before installing needed packages it would be better toupdate your initial system.

Updating Ubuntu

Open a terminal and issue the following commands;

>sudo apt-get update

This will collect the latest package informations from the repositories. And then,

>sudo apt-get upgrade

will check all updated packages and will show you how many package is being updated. After youconfirm, it will start to download and update process.

3

Installing C++ and Fortran Compilers

To do so, open a terminal and issue the following commands there;

>sudo apt-get install build-essential

This package depends on many other packages which contains C++ compiler, and realted items.After you confirm, all those packages will be installed on your computer.

If you would like to test if C++ is installed, just enter

>c++

command. It's output would be similar to;

c++: no input files

> sudo apt-get install gfortran

This will install GNU Fortran Compiler on your computer.

After those compiler installations, we are ready to install OpenMPI now.

Installing OpenMPI

We need openmpi libraries, runtime environments, and development files to create parallel C, C++and Fortran applications which are based on MPI. There is of course other MPI packages that can beused, but we prefer to use OpenMPI packages.

Execute following command in a terminal;

>sudo apt-get install openmpi-bin openmpi-dev

Afteryou confirm the selected packages, all needed packages would be installed on your computerand after this process our computer is ready for compilation and running of parallel program basedon MPI. To test our environment let us create the simplest “HelloWorld” parallel application in C,compile and run on our computer.

Open Text Editor from Application-> Accessories-> Text Editor menu. Copy the following codesfrom here and paste to editor content;

#include <stdio.h>

#include <mpi.h>

int main(int argc, char *argv[]) {

int numprocs, rank, namelen;

char processor_name[MPI_MAX_PROCESSOR_NAME];

MPI_Init(&argc, &argv);

MPI_Comm_size(MPI_COMM_WORLD, &numprocs);

MPI_Comm_rank(MPI_COMM_WORLD, &rank);

4

MPI_Get_processor_name(processor_name, &namelen);

printf("Process %d on %s out of %d\n", rank, processor_name, numprocs);

MPI_Finalize();

}

Save this file as “testmpi.c” in your home directory. When you click on “Save” button the defaultlocation is your home directory. So you simply write the file name and click on Save in this dialogwindow. Here is a screenshot after save operation.

Now you have a simple parallel C program saved in your home directory. Open a new terminal. Theworking directory of this terminal is initially would be your home directory. So just see if your file isthere by issuing “ls-l” command, which will list the files and folder names in current directory.

5

If your file is there then compile it using the command;

>mpicc testmpi.c-o testmpi

This will compile thecode and will produce an executable file named “testmpi” in the currentdirectory. Don't worry, no output means successfull compilation. If you like you can list your filesand see if the executable file is there. Executable file names are displayed in different color, mostprobably in green.

Now let us run this application on 4 processor. To do so, issue the following command;

>mpirun-np 4 ./testmpi

The output will be similar to;

This shows us our computer is ready to develop parallel application

in C, C++, or Fortran using MPI,compile and run on out machine.

6

It would be better to have an Integrated Development Environment (IDE) for our preferredprogramming language. This would speed up writing codes, compilation and running process.

Installing IDE

There is a range of development environment can be found on Linux, from the simplest one such as“Geany” to the advanced such as “Eclipse”, “Netbeans”, etc..

For our purpose we will show you how to install both “Geany” and “Eclipse” on your computer andwill show you how to configure them to compile and run applications.

Before installing IDE's let me give you a brief information about how we install application onUbuntu. You may find other ways that are not mentioned here. But I prefer to use

command line tool“apt-get” and GUI tool “Ubuntu Software Center”. Up to now, we have used “apt-get” many times toinstall packages.

On the other hand, there is a GUI application named “Ubuntu Software Center” residing inApplications menu on top of your computer. Ubuntu development team categorised most of theapplication, so that we may find them easily using this GUI tool. Here is a screenshot;

As you can see there is a “Developer Tool” department. Just click on it, you will se sub departmentsfrom

where you will see a department named “IDEs”. If you click on “IDEs” you will see the list ofapplication that can be installed. If there is a green check on the application icon, that means it isalready installed on your computer. Here is a screenshot of “IDEs” department content. I have alsoclicked on “Geany” there.

7

As you see from screenshot, when you click on “Geany” it becomes highlighted and shows a shortdescription of application together with “More Info” and “Install” button. If you click on

“More Info”you will have another window, showing detailed information about the application and there willalso be an “Install” button. If you highlight already installed application, instead of “Install” button,there will be “Remove” button, which uninstall the application.

Now clik on “Install” button of Geany. After completion you will see a green check on Geany's iconand it is ready for use. Geany can be found under “Application->Programming” menu.

8

As you can see from the menu list, I have already installed many IDEs on my computer. You can alsoinstall and try them. You can uninstall whenever you want :)

Let us start using geany. After you click on that menu item, geany will be opened. Click on File->Open menu item and select the “testmpi.c” file from your home directory. From the Build menu ofGeany, there is a “Set Build Command” menu item. Click on it and you will be able to set compilerand runtime commands.

Since geany knows GNU C compiler, there would be commands for non-parallel applications, so weneed to update those commands.

Here is the screenshot after change of commands. You just need to replace “gcc” command with“mpicc” in Compile and Build edit boxes. And for running application, you insert “mpirun-np 4” infront of the command already given in “Execute” edit box.

That's all we need to setup :) You can compile and run your application. Let us see screenshots ofsetting window before and after commands update.

9

Afterupdate itwilllooklike;

Thats all forGeany. You

cancompile andExecute yourapplication. Have anice coding withGeany :)

InstallingEclipse andParallel Tool Platform for C, C++ and Fortran

This part will be comleted later... Check this tutorial regularly..

10

Basic MPI Commands

MPI Fortran Commands

MPI_INIT(err)

Initiate an MPI computation

MPI_COMM_RANK( MPI_COMM_WORLD,rank, ierr )

Determine my process identifier.

MPI_COMM_SIZE( MPI_COMM_WORLD,size, ierr )

Determine number of processes.

MPI_FINALIZE(err)

Terminate a computation

MPI C Commands used

MPI_Init( &argc, &argv )

Initiate an MPI computation

MPI_Comm_rank( MPI_COMM_WORLD,&rank )

Determine my process identifier

MPI_Comm_size( MPI_COMM_WORLD,&size )

Determine number of processes.

MPI_Finalize()

Terminate a computation

Example 1.

Fortran

PROGRAM first472

INCLUDE ‘mpif.h‘

INTEGER err

CALLMPI_INIT(err)

CALLMPI_COMM_RANK( MPI_COMM_WORLD,rank, ierr )

CALLMPI_COMM_SIZE( MPI_COMM_WORLD,size, ierr )

PRINT*, 'I am ', rank, ' of ', size

CALLMPI_FINALIZE(err)

END

C

include <stdio.h>

#include <mpi.h>

void main (int argc, char * argv[])

{

int rank, size;

MPI_Init( &argc, &argv );

MPI_Comm_rank( MPI_COMM_WORLD,&rank );

MPI_Comm_size( MPI_COMM_WORLD,&size );

printf( "I am %d of %d\n", rank, size );

MPI_Finalize();

return 0;

}

Types of Point-to-Point Operations



MPI point-to-point operations typically involve message passing between two, and only two,different MPI tasks. One task is performing a send operation and the other task is performinga matching receive operation.



There are different types of send and receive routines used for different purposes. Forexample:

o

Synchronous send

11

o

Blocking send / blocking receive

o

Non-blocking send / non-blocking receive

o

Buffered send

o

Combined send/receive

o

"Ready" send



Any type of send routine can be paired with any type of receive routine.



MPI also provides several routines associated with send-

receive operations, such as thoseused to wait for a message's arrival or probe to find out if a message has arrived.

Buffering



In a perfect world, every send operation would be perfectly synchronized with its matchingreceive. This is rarely the case. Somehow or other, the MPI implementation must be able todeal with storing data when the two tasks are out of sync.



Consider the following two cases:

o

A send operation occurs 5 seconds before the receive is ready-

where is the messagewhile the receive is pending?

o

Multiple sends arrive at the same receiving task which can only accept one send at atime-

what happens to the messages that are "backing up"?



The MPI implementation (not the MPI standard) decides what happens to data in these typesof cases. Typically, asystem buffer

area is reserved to hold data in transit. For example:



System buffer space is:

o

Opaque to the programmer and managed entirely by the MPI library

o

A finite resource that can be easy to exhaust

o

Often mysterious and not well documented

o

Able to exist on the sending side, the receiving side, or both

o

Something that may improve program performance because it allows send-

receiveoperations to be asynchronous.



User managed address space (i.e. your program variables) is called theapplication buffer.MPI also provides for a user managed send buffer.

Blocking vs. Non-blocking:

12



Most of the MPI point-to-point routines can be used in either blocking or non-blocking mode.



Blocking:

o

A blocking send routine will only "return" after it is safe to modify the applicationbuffer (your send data) for reuse. Safe means that modifications will not affect the dataintended for the receive task. Safe does not imply that the data was actually received-

it may very well be sitting in a system buffer.

o

A blocking send can be synchronous which means there is handshaking occurringwith the receive task to confirm a safe send.

o

A blocking send can be asynchronous if a system buffer is used to hold the data foreventual delivery to the receive.

o

A blocking receive only "returns" after the data has arrived and is ready for use by theprogram.



Non-blocking:

o

Non-blocking send and receive routines behave similarly-

they will return almostimmediately. They do not wait for any communication events to complete, such asmessage copying from user memory to system buffer space or the actual arrival ofmessage.

o

Non-blocking operations simply "request" the MPI library to perform the operationwhen it is able. The user can not predict when that will happen.

o

It is unsafe to modify the application buffer (your variable space) until you know for afact the requested non-blocking operation was actually performed by the library. Thereare "wait" routines used to do this.

If a sender sends two messages (Message 1 and Message 2) in succession to the samedestination, and both match the same receive, the receive operation will receiveMessage 1 before Message 2.

o

If a receiver posts two receives (Receive 1 and Receive 2), in succession, and both arelooking for the same message, Receive 1 will receive the message before Receive 2.

o

Order rules do not apply if there are multiple threads participating in thecommunication operations.



Fairness:

o

MPI does not guarantee fairness-

it's up to the programmer to prevent "operationstarvation".

o

Example: task 0 sends a message to task 2. However, task 1 sends a competing messagethat matches task 2’s receives. Only one of the sends will complete.

13

MPI Message Passing Routine Arguments

MPI point-to-point communication routines generally have an argument list that takes one of thefollowing formats:

Blocking sends

MPI_Send(buffer,count,type,dest,tag,comm)

Non-blockingsends

MPI_Isend(buffer,count,type,dest,tag,comm,request)

Blocking receive

MPI_Recv(buffer,count,type,source,tag,comm,status)

Non-blocking receive

MPI_Irecv(buffer,count,type,source,tag,comm,request)

Buffer

Program (application) address space that

references the data that is to be sent or received. Inmost cases, this is simply the variable name that is be sent/received. For C programs, thisargument is passed by reference and usually must be prepended with an ampersand:&var1

Data Count

Indicates the number of data elements of a particular type to be sent.

Data Type

For reasons of portability, MPI predefines its elementary data types. The table below liststhose required by the standard.

C Data Types

Fortran Data Types

MPI_CHAR

signed char

MPI_CHARACTER

character(1)

MPI_SHORT

signed shortint

MPI_INT

signed int

MPI_INTEGER

integer

MPI_LONG

signed longint

MPI_UNSIGNED_CHAR

unsigned char

14

MPI_UNSIGNED_SHORT

unsignedshort int

MPI_UNSIGNED

unsigned int

MPI_UNSIGNED_LONG

unsigned longint

MPI_FLOAT

float

MPI_REAL

real

MPI_DOUBLE

double

MPI_DOUBLE_PRECISION

doubleprecision

MPI_LONG_DOUBLE

long double

MPI_COMPLEX

complex

MPI_DOUBLE_COMPLEX

doublecomplex

MPI_LOGICAL

logical

MPI_BYTE

8 binary digits

MPI_BYTE

8 binary digits

MPI_PACKED

data packedor unpackedwithMPI_Pack()/MPI_Unpack

MPI_PACKED

data packed orunpacked withMPI_Pack()/MPI_Unpack

Notes:



Programmers may also create their own data types (seeDerived Data Types).



MPI_BYTE and MPI_PACKED do not correspond to standard C or Fortran types.

An argument to send routines that indicates the process where a message should be delivered.Specified as the rank of the receiving process.

Source

An argument to receive routines that indicates the originating process of the message.Specified as the rank of the sending process. This may be set to the wild cardMPI_ANY_SOURCE to receive a message from any task.

Tag

Arbitrary non-negative integer assigned by the programmer to uniquely identify a message.Send and receive operations should match message tags. For a receive operation, the wild cardMPI_ANY_TAG can be used to receive any message regardless of its tag. The MPI standardguarantees that integers 0-32767 can be used as tags, but most implementations allow a muchlarger range than this.

15

Communicator

Indicates the communication context, or set of processes for which the source or destinationfields are valid. Unless the programmer is explicitly creating new communicators, thepredefined communicator MPI_COMM_WORLD

is usually used.

Status

For a receive operation, indicates the source of the message and the tag of the message. In C,this argument is a pointer to a predefined structure MPI_Status (ex. stat.MPI_SOURCEstat.MPI_TAG). In Fortran, it is an integer array of size MPI_STATUS_SIZE (ex.stat(MPI_SOURCE) stat(MPI_TAG)). Additionally, the actual number of bytes received areobtainable from Status via the MPI_Get_count routine.

Request

Used by non-blocking send and receive operations. Since non-blocking operations may returnbefore the requested system buffer space is obtained, the system issues a unique "requestnumber". The programmer uses this system assigned "handle" later (in a WAIT type routine)to determine completion of the non-blocking operation. InC, this argument is a pointer to apredefined structure MPI_Request. In Fortran, it is an integer.

16

Lab2(Week#8)

Blocking Message Passing Routines

The more commonly used MPI blocking message passing routines are described below.

MPI_Send

Basic blocking send operation. Routine returns only after the application buffer in the sendingtask is free for reuse. Note that this routine may be implemented differently on different

systems. The MPI standard permits the use of a system buffer but does not require it. Someimplementations may actually use a synchronous send (discussed below) to implement thebasic blocking send.

MPI_Send (&buf,count,datatype,dest,tag,comm)

MPI_SEND(buf,count,datatype,dest,tag,comm,ierr)

MPI_Recv

Receive a message and block until the requested data is available in the application buffer inthe receiving task.

MPI_Recv(&buf,count,datatype,source,tag,comm,&status)

MPI_RECV (buf,count,datatype,source,tag,comm,status,ierr)

MPI_Ssend

Synchronous blocking send: Send a message and block untilthe application buffer in thesending task is free for reuse and the destination process has started to receive the message.

MPI_Ssend (&buf,count,datatype,dest,tag,comm)

MPI_SSEND (buf,count,datatype,dest,tag,comm,ierr)

MPI_Bsend

Buffered blocking send: permits the programmer to allocate the required amount of bufferspace into which data can be copied until it is delivered. Insulates against the problemsassociated with insufficient system buffer space. Routine returns after the data has beencopied from application buffer space to the allocated send buffer. Must be used with theMPI_Buffer_attach routine.

MPI_Bsend (&buf,count,datatype,dest,tag,comm)

MPI_BSEND (buf,count,datatype,dest,tag,comm,ierr)

MPI_Buffer_attach

MPI_Buffer_detach

Used byprogrammer to allocate/deallocate message buffer space to be used by theMPI_Bsend routine. The size argument is specified in actual data bytes-

not a count of dataelements. Only one buffer can be attached to a process at a time. Note that the IBMimplementation uses MPI_BSEND_OVERHEAD bytes of the allocated buffer for overhead.

MPI_Buffer_attach (&buffer,size)

MPI_Buffer_detach (&buffer,size)

17

MPI_BUFFER_ATTACH (buffer,size,ierr)

MPI_BUFFER_DETACH (buffer,size,ierr)

MPI_Rsend

Blocking ready send. Should only be used if the programmer is certain that the matchingreceive has already been posted.

MPI_Rsend (&buf,count,datatype,dest,tag,comm)

MPI_RSEND (buf,count,datatype,dest,tag,comm,ierr)

MPI_Sendrecv

Send a message and post a receive before blocking. Will block until the sending applicationbuffer is free for reuse and until the receiving application buffer contains the receivedmessage.

Write an MPI program in which each process displays its rank followed as follows:

Hi! I’m process number xxx.

****Ziya****

Problem 2.****Ziya****

19

Lab3(Week#11)

Example 3. Parallel Numerical Integration

program

main

use

mpi

double precision

starttime, endtime

double precision

PI25DT

parameter

(PI25DT = 3.141592653589793238462643d0)

double precision

mypi, pi, h, sum, x, f, a

double precision

starttime,

endtime

integer

n, myid, numprocs, i, ierr

f(a) = 4.d0 / (1.d0 + a*a)! function to integrate

call

MPI_INIT(ierr)

call

MPI_COMM_RANK(MPI_COMM_WORLD, myid, ierr)

call

MPI_COMM_SIZE(MPI_COMM_WORLD, numprocs, ierr)

10

if

( myid .eq. 0 )then

print

*,'Enter the number of intervals: (0 quits) '

read(*,*) n

endif

starttime = MPI_WTIME()

! broadcast n

call

MPI_BCAST(n,1,MPI_INTEGER,0,MPI_COMM_WORLD,ierr)

!

check for quit signal

if

( n .le. 0 )goto

30

! calculate the interval size

h = 1.0d0/n

sum = 0.0d0

do

20 i = (n*myid)/numprocs+1, n*(myid+1)/numprocs

x =

h * (dble(i)-

0.5d0)

sum = sum + f(x)

20

continue

mypi = h * sum

! collect all the partial sums

call

MPI_REDUCE(mypi,pi,1,MPI_DOUBLE_PRECISION,MPI_SUM,0, &

MPI_COMM_WORLD,ierr)

! node 0 prints the answer.

endtime = MPI_WTIME()

if

(myid .eq. 0)then

print

*,'pi is ', pi,'Error is ',abs(pi-

PI25DT)

print

*,'time is ', endtime-starttime,' seconds'

endif

go to

10

30

call

MPI_FINALIZE(ierr)

stop

end

Problem 3.Convert the code above to C. Test the program with different number of processes anddifferent number of intervals (n).

Example 4.

Design for a parallel program to sum an array

The

code below shows a common program structure for including both master and slave segments inthe parallel version of the example program just presented. It is composed of a short set-up sectionfollowed by a singleif...elseloop where the master process executes the statments between the

20

brackets after the if statement, and the slave processes execute the statements between thebracketsafter the else statement.