3 Cooperating ProcessesIndependent process is a process that cannot affect or cannot be affected by the execution of another process. A process that does not share data with another process is independentCooperating process is a process that can affect or can be affected by the execution of another process. A process that share data with other process is a cooperating processAdvantages of Cooperating processes:Information sharingComputation speed upModularityConvenience3/25/2017CMP320 PUCIT Arif Butt

4 Inter Process CommunicationIPC is a mechanism for processes to communicate and to synchronize their actions. There are two fundamental models of IPC:Shared Memory – A region of memory that is shared by cooperating processes is established. Processes can then exchange information by reading and writing data to the shared regionMessaging system – Communication takes place by means of messages exchanged between the cooperating processes. Processes communicate with each other without resorting to shared variables. (pipes, FIFO, sockets, message queues, semaphores, signals)3/25/2017CMP320 PUCIT Arif Butt

7 Producer-Consumer Problem (cont…)A producer process produces information that is consumed by a consumer processTo allow producer and consumer run concurrently we must have a buffer that can be filled by the producer and emptied by the consumer. Buffer can be bounded or unboundedUnbounded BufferPlaces no practical limit on the size of the bufferThe consumer may have to wait for new items (if the buffer is empty), but the producer can always produce new itemsBounded BufferAssumes a fixed size bufferThe consumer must wait if the buffer is empty, and the producer must wait if the buffer is full3/25/2017CMP320 PUCIT Arif Butt

9 Solution 2 - Bounded Buffer Problem#define BUFFER_SIZE 5typedef struct{ ---- } item;item buffer[BUFFER_SIZE];int in = 0; //points to location where next item will be placed, will be used by producer processint out = 0; //points to location from where item is to be consumed, will be used by consumer processint ctr = 0;Can you do it withoutusing the ctr variable?Producer Processitem nextProduced;while(1){nextProduced = getItem();while(ctr == BUFFER_SIZE); //do nothingbuffer[in] = nextProduced;in = (in + 1) % BUFFER_SIZE;ctr++; }Consumer Processitem nextConsumed;while(1){while(ctr == 0); //do nothingnextConsumed = buffer[out];out = (out + 1) % BUFFER_SIZE;ctr--;}Instead using the ctr variable. Initialize the entire buffer with -1. While consuming an item remember to place a -1 where out is pointing to.While(in == out && buff[in] != -1) // means the buffer is full, so a producer cannot produceWhile(in == out && buff[in] == -1) // means the buffer is empty, so a consumer cannot consume3/25/2017CMP320 PUCIT Arif Butt

10 Messaging SystemMessage passing provides a mechanism to allow processes to communicate and to synchronize their actions without sharing the same address spaceIt is particularly useful in a distributed environment, where the communicating processes may reside on different computers connected by a networkFor example; a chat program used on the wwwMessage Passing is one solution to the communication requirement. The added bonus is: It works with shared memory and with distributed systems3/25/2017CMP320 PUCIT Arif Butt

12 Messaging System (cont…)Direct CommunicationProcesses must name each other explicitly:send (P, message) – send a message to process Preceive(Q, message) – receive a message from process QSend primitive includes a specific identifier of the destination processReceive primitive could know ahead of time which process a message is expectedReceive primitive could use source parameter to return a value when the receive operation has been performedExample is a letter mailed via Pakistan Postal ServiceProperties of communication link in Direct Comm.:A Link is established automaticallyA link is associated with exactly one pair of communicating processesBetween each pair there exists exactly one link3/25/2017CMP320 PUCIT Arif Butt

13 Messaging System (cont…)Indirect CommunicationMessages are directed and received from mailboxes (also referred to as ports)Each mailbox has a unique idProcesses can communicate only if they share a mailboxWe can think of a mailbox as an abstract object into which a message can be placed in or removed fromProperties of communication link in Indirect Comm.:Link established only if processes share a common mailbox.A link may be associated with many processesEach pair of processes may share several communication links3/25/2017CMP320 PUCIT Arif Butt

14 Messaging System (cont…)Indirect Communication (cont…)Operationscreate a new mailbox.send and receive messages through mailbox.destroy a mailbox.Primitives are defined as:send(A, message) – send a message to mailbox Areceive(A, message) – receive a message from mailbox AMailbox SharingP1, P2, and P3 share mailbox AP1, sends; P2 and P3 receiveWho gets the message?SolutionsAllow a link to be associated with at most two processes.Allow only one process at a time to execute a receive operation.Allow the system to select arbitrarily the receiver. Sender is notified who the receiver was.3/25/2017CMP320 PUCIT Arif Butt

15 Indirect Process Communication (…)Animated Slide – each item below is magnified for instructor to address separately1) A one-to-one relationshipallows a private communications link to be set up between two processes.This insulates their interaction from erroneous interference from other processes.2) A many-to-one relationship is useful for client/server interaction;one process provides service to a number of other processes.In this case, the mailbox is often referred to as a port.3) A one-to-many relationship allows for one sender and multiple receivers;it is useful for applications where a message or some information is to be broadcast to a set of processes.4) A many-to-many relationshipallows multiple server processes to provide concurrent service to multiple clients.The association of processes to mailboxes can be either static or dynamic.Ports are often statically associated with a particular process; that is, the port is created and assigned to the process permanently.Similarly, a one-to-one relationship is typically defined statically and permanently. When there are many senders, the association of a sender to a mailbox may occur dynamically. Primitives such as connect and disconnect may be used for this purpose3/25/2017CMP320 PUCIT Arif Butt

16 General Message FormatThe format of the message depends on the objectives of the messaging facility and whether the facility runs on a single computer or on a distributed system.This is a typical message format for operating systems that support variable-length messages.The message is divided into two parts:a header, which contains information about the message.The header may contain an identification of the source and intended destination of the message, a length field, and a type field to discriminate among various types of messages.additional control information, e.g. pointer field so a linked list of messages can be created; a sequence number, to keep track of the number and order of messages passed between source and destination; and a priority field.a body, which contains the actual contents of the message.3/25/2017CMP320 PUCIT Arif Butt

17 Messaging System (cont…)SynchronizationMessage passing may be either blocking or non-blockingBlocking is considered synchronousBlocking send. The sending process is blocked until the message is received by the receiving process or by the mailbox. (Producer process blocks when the buffer is full)Blocking receive. The receiver blocks until a message is available. (Consumer process blocks when the buffer is empty)Non-blocking is considered asynchronousNon-blocking send. The sending process sends the message and resumes operationNon-blocking receive. The receiver receives either a valid message or a null3/25/2017CMP320 PUCIT Arif Butt

18 Messaging System (cont…)BufferingWhether communication is direct or indirect, messages exchanged by communicating processes reside in a temporary queue. Such queues can be implemented in three ways:Zero Capacity. The queue has a length of zero. Thus the link cannot have any messages waiting in it. The sender must block until the recipient receives the messageBounded Capacity. The queue has finite length n. Sender must wait if link is fullUnbounded Capacity. The queue is of infinite length. So any number of messages can wait in it. Thus the sender never blocks3/25/2017CMP320 PUCIT Arif Butt

19 UNIX / LINUX IPC ToolsPipes are used for communication between related processes (parent-child-sibling) on the same UNIX system.P1P2PipeUNIX/Linux System3/25/2017CMP320 PUCIT Arif Butt

20 UNIX / LINUX IPC ToolsNamed pipes (FIFO) are used for communication between related or unrelated processes on the same UNIX system.P1P2p1p2FIFOUNIX/Linux SystemCMP320 PUCIT Arif Butt3/25/2017

21 UNIX / LINUX IPC ToolsSockets are used for communication between related or unrelated processes on the same or different UNIX systems.P1P2Network ConnectionSocketSocketComputer 1Computer 2CMP320 PUCIT Arif Butt3/25/2017

23 Cmd Line use of Pipes cmd1 | cmd2 | cmd3 | … | cmdNThe UNIX system allows stdout of a command to be connected to stdin of another command using the pipe operator |.cmd1 | cmd2 | cmd3 | … | cmdNStdout of cmd1 is connected to stdin of cmd2, stdout of cmd2 is connected to stdin of cmd3, … and stdout of cmdN-1 is connected to stdin of cmdN.Collaboration allows faults to propagate…cmd1pipecmd2pipepipecmdNCMP320 PUCIT Arif Butt3/25/2017

24 Cmd Line Use of PipesExample 1. Write a command that displays the contents of /etc/ directory, one page at a time.$ ls –l /etc/ 1> temp$ less 0< tempThis will display the contents of file temp on screen one page at a time. After this we need to remove the temp file as well.$ rm tempSo we needed three commands to accomplish the task, and the command sequence is also extremely slow because file read and write operations are involved.Collaboration allows faults to propagate…Lets use the pipe symbol$ ls –l /etc/ | lessThe net effect of this command is the same. It does not use a disk to connect standard output of ls –l to standard input of less because pipe is implemented in the main memory.CMP320 PUCIT Arif Butt3/25/2017

26 Cmd Line Use of Pipes (cont…)Pipes and redirection operators alone cannot be used to redirect stdout of a command to a file as well as another command in a single commandtee command copies stdin to stdout and at the same time copies it to one or more files$ tee tout1.txt tout2.txttee command is used to redirect the stdout of a command to one or more files, as well as to another commandcmd1 | tee file1 … fileN | cmd2Stdout of cmd1 is connected to stdin of tee, and tee sends its input to files file1, … fileN and as stdin of cmd2$ who | tee who.txtSarwar page = 320CMP320 PUCIT Arif Butt3/25/2017

27 System Call - pipe() int pipe(int pipefd[2]); Call Fails when:pipefd[0] and pipefd[1] are read and write descriptors respectively.Will return -1 on failureBounded buffer, maximum data written is PIPE_BUF <linux/param.h>, 4096 bytesCall Fails when:Opening a pipe is equivalent to opening two files, so at least two slots need to be empty in the PPFDT, otherwise, the call will fail.Buffer space not available in the kernel.File table is full.3/25/2017CMP320 PUCIT Arif Butt

28 System Call - pipe() fork P2 P1 pipe parent childChild Process will send a string to Parent Processand Parent Process will print that on screen.forkP2P1parentchildChild writes in the pipe. It blocks if the pipe is full.Parent read from the pipe. It blocks if the pipe is empty.pipeRead endWrite endProcess P1 will create a pipe (having two ends, a read end and a write end).Then the Parent Process P1 will create a child process P2.There are two descriptors associated with the pipe. The writer process (child) will use the write descriptor of the pipe. The reader process (parent) will use the read descriptor of the pipe.The child process will write data onto the pipe and the parent process will read that data at the read end and will display it on the screen.3/25/2017CMP320 PUCIT Arif Butt

32 Named Pipes / FIFOsFirst In First Out are special file types in UNIX used for communication between related or unrelated processes on a computer.Processes communicating with pipes must be related to each other through a common ancestor process that you execute, processes communicating with FIFOs do not have to have this kind of relationship. They can be independently executed programs on a system.A pipe is a main memory buffer and has no name, while FIFO is created on disk and has a name like a file name.So like files and unlike a pipe, a FIFO has to be created and opened before using it for communication.We can use mkfifo command to create a FIFO file in a shell session. We can use mknod() system call or mkfifo() library call to create a FIFO in a process.UNIX system calls read(), write(), close(), etc. can be used with FIFOs.Major differences between pipes and FIFOspipe is an in-memory buffer, while FIFOs are mostly created on disk.After creating a pipe, you don’t need to explicitly open it before using it.After creating a FIFO, you need to open it and you need to tell the system whether you are opening it for reading or writing or both. After you are done you need to close them. (Just as you do I/O with files)CMP320 PUCIT Arif Butt3/25/2017

33 Named Pipes / FIFOs Two common uses of FIFOs are:Used by shell commands to pass data from one shell pipeline to another, without creating temporary filesIn client server applications, FIFOs are used to pass data between a server process and client processesCollaboration allows faults to propagate…CMP320 PUCIT Arif Butt3/25/2017

34 Cmd Line Use of FIFOsExample. Write a command that will count the number of lines in the man page of ls.$ man ls | wc –l239$Example. Write a command that will count only the number of lines containing string “ls” in the man page of ls.Collaboration allows faults to propagate…$ man ls | grep ls | wc –l9$CMP320 PUCIT Arif Butt3/25/2017

35 Cmd Line Use of FIFOs (cont…)$ mkfifo fifo1$ prog3 0< fifo1 &$ prog1 0< infile | tee fifo1 | prog2First line will create an empty FIFO named fifo1.In second line, prog3 is trying to read its input from fifo1. Since fifo1 is empty therefore prog3 will block at the read operation. More so the & shows that it will run in the back ground.In third line, prog1 will read its input from infile and send its o/p to tee utility and tee will do two things:Write the o/p to fifo1 as well as to the stdout which will be sent to prog3Through the pipe will give the o/p to prog2$ mkfifo fifo1man ls 1> ls.dat$ cat 0< fifo1 | grep ls | wc –l &[1] 21108$ sort 0< ls.dat | tee fifo1 | wc –l9239wc -lgrepfifo1PipeThe no 31 is the count of lines in ls.dat containing the string ls, while 239 is the total count of lines in ls.datsortteeinfilePipewc -lCMP320 PUCIT Arif Butt3/25/2017

36 Client Server Comm with FIFOsServer will open a well known FIFO, which it will useto read the request of client.Each client will open a client FIFO and send its name to the server process,so now the server knows via which client FIFO it can send reply to a particular client3/25/2017CMP320 PUCIT Arif Butt

37 System Call - mknod() Call Fails when: #include <sys/types.h>#include <sys/stat.h>int mknod(const char * pathname, mode_t mode, dev_t dev);mode should be permission mode ORed with S_IFIFO flag.dev is set to 0 for a FIFO, i.e. we are not making a device file.Call Fails when:File with the given name already existsPathname too longA component in the pathname not searchable or non existentDestination directory is read onlyNot enough memory space available3/25/2017CMP320 PUCIT Arif Butt

39 Example - FIFO FIFO1 FIFO2 Display Screen Server Client ReadingWritingFIFO1ServerClientFIFO2WritingReadingServerCreate FIFO1 and FIFO2Open FIFO1 for reading.Open FIFO2 for writing.Make a blocking read() system call on FIFO1Once the client writes in FIFO1 (sent msg to server), the server will display that msg on screen.Then server will send a msg to client “Hello Class”.CLIENTOpen FIFO1 for writing.Open FIFO2 for reading.Write “Hello World” msg in FIFO1 (this will make server come out of the blocked read() system call)Client will read from FIFO2Once client will receive the msg sent by server from FIFO2 and will display that msg on screen.Display Screen3/25/2017CMP320 PUCIT Arif Butt

47 Shared Memory SystemShared memory allows two or more processes to share a given region of memory.This is the fastest form of IPC because the data does not need to be copied between the client and server.The only trick in using shared memory is synchronizing access to a given region among multiple processes.If the server is placing data into a shared memory region, the client shouldn't try to access the data until the sever is done.Often semaphores are used to synchronize shared memory access. We can use record locking as well.3/25/2017CMP320 PUCIT Arif Butt

48 Shared Memory System (cont…)Data 2CodeDataHeapStackSharedCodeDataHeapStackSharedStack 1Heap 1Code 1Stack 2Data 1Prog 2LogicalAddressSpace 2Prog 1LogicalAddressSpace 1Heap 2Code 2SharedIn shared memory we declare a given section in the memory as one that will be used simultaneously by several processes.Typically a shared memory region resides in the address space of the process creating the shared memory segment.Other processes that wish to communicate using this shared memory segment must attach it to their address space.3/25/2017CMP320 PUCIT Arif Butt

50 char * shared_memory = (char*) shmat(segment_id, NULL, 0);System Call shmat()Processes that wish to access a shared memory segment must attach it to their address space using the shmat() system call. (SHared Memory ATach)On success, shmat() returns a pointer to the first byte of shared memory.On failure, it returns -1.char * shared_memory = (char*) shmat(segment_id, NULL, 0);void* shmat(int shmid, const void *shmaddr, int flag); First parameter, shm_id, is the shared memory identifier returned from shmget. Second parameter is a pointer location in memory indicating where the shared memory will be attached. NULL pointer allows the system to choose the address at which the memory appears. Third parameter identifies the mode, a 0 means read-write.3/25/2017CMP320 PUCIT Arif Butt

51 shmdt(shared_memory);System Call shdt()When a process no longer requires access to the shared memory segment, it detaches the segment form its address space.This does not remove the identifier and its associated data structure from the system. The identifier remains in existence until some process specifically removes it by calling shmctl() with a command of IPC_RMID.To detach a region of shared memory, the process can pass the pointer of the shared memory region to the shmdt() system call. (SHared Memory DeTach)shmdt(shared_memory);int shmdt(void * addr);The addr argument is the value that was returned by a previous call to shmat.3/25/2017CMP320 PUCIT Arif Butt

52 shmctl(segment_id, IPC_RMID, NULL);System Call shctl()Shared memory segment can be removed from the system with the shmctl() system call. (SHared Memory ConTroL)Returns 0 if OK, -1 on error.shmctl(segment_id, IPC_RMID, NULL);void* shmctl(int shmid, int cmd, struct shmid_ds * buf); First parameter, shmid, is the shared memory identifier returned from shmget(). Second parameter can take five different values. IPC_RMID removes the shared memory segment set from the system. Third parameter is normally a NULL.3/25/2017CMP320 PUCIT Arif Butt

55 We’re done for now, but Todo’s for you after this lecture…Go through the slides and Book Sections: 3.1 to 3.4, 3.51Solution 2 to the bounded buffer problem is using an additional variable ctr. Suggest another solution that do not use any additional variable (do not mention the one discussed in class).Write down the programs discussed in class in vim editor and execute them. Make variations in the programs and understand the underlying details.If you have problems visit me in counseling hours3/25/2017CMP320 PUCIT Arif Butt