Archive

It’s a typical exercise, but we always see it theoretically, let’s bring it to practice. We’re developing with semaphores. As we can see here and here, semaphores, among other things will be used to block a process or thread which is trying to access an exclusive resource which is already being used by other process. A typical example is when you go to a public toilet: when the door isn’t locked, you go in and lock the door, when you finish whatever you were doing there, you unlock the door and exit. The same way, when a process tries to use a resource, if the semaphore is open, closes it, use the resource and reopens the semaphore when finish.
But, here we can create as many semaphores as we want, and we are free to do whatever we want with them, so we can create tons of situations.

But, what if we have three processes (P1, P2 and P3), and P1 is waiting for P2, P2 is waiting for P3 and P3 is waiting for P1? We’ll be waiting forever.

What I’m about to code is:

We have two processes (P1 and P2)

We have two resources (R1 and R2) which are exclusive (only one process can access each moment)

P1 wants to access R1, so closes its semaphore

P2 wants to access R2, so closes its semaphore

P1 wants to access also R2, so waits for its semaphore to be open

P2 wants to access also R1, so waits for its semaphore to be open

As P1 is waiting P2 to open R2 semaphore and P2 is waiting P1 to open R1 semaphore, both processes will be waiting indefinitely.

Don’t make a mess

Use the least amount of semaphores, if we can do the same using one semaphore, just use one. We can do it in some cases.

Use the resource if available, skip if not

We can use sem_trywait(), if the resource is busy, it’ll return an error without blocking the application. We only have to do it in one process, but this process will enter fewer times in the critic section:

We’ve been practicing sharing variables between child processes, but when there are some processes trying to access a shared resource, we need a mutex to make it safer. This time to implement the mutex we’ll use semaphores. This semaphores must be also shared variables to work properly.

First, think about semaphores as variables which can be 0 or 1. So if the semaphore is 1, it’s open and we will close (0 value) it after we pass; if it is 0, we’ll wait until it goes 1 (it’s not like a while (semaphore==0); because the operating system will deactivate the process and reactivate it when the semaphore is open and we can use our system resources for anything else).
But, let’s go further, semaphore’s value can be whatever, not just 0 or 1, but if it’s positive, and we want to pass, we will decrement it and it won’t block our process, but if it’s zero or less, our process will block. So we can say a mutex is a semaphore with 1 and 0 values, used to protect a resource.

To use semaphores we must have in mind three basic functions (there are some more):

sem_init(semaphore, pshared, value): Initialize the semaphore with a known value, pshared can be 0 if we want it to be shared between threads of the process, or another value if we want it to be shared between processes. In this case we will put a 1 here.

sem_post(semaphore): Increment the semaphore, it’s what we do to free the resource

sem_wait(semaphore): Decrement the semaphore, if its value is less than zero, blocks the process until we have a value greater or equal than zero. We’ll use this to check if the resource is locked.

In the next example, we’ll increment a number, but to make it a bit more difficult, it will be stored in a string, each increment must be done by a different child process. The final value of x must be 20. On the other hand, I’ve inserted some random waits to simulate a heavy process and provoke a race condition.

We can change SEMAPHORES constant value from 1 to 0 to see how this program behaves in each case:

Each time a process wants to enter our critic section, it will be written on screen by its process Id, so we can see when a process is accessing the resource, and we can detect if two or more processes are accessing simultaneously (and we don’t want it). Remember, then final x value must be 20 and without semaphores we may or may have not this value, it isn’t under our control.

Another way of facing concurrency in the wonderful world of doing several task at the same time is using child processes with fork(). The main difference between fork and threads is that forked process are full process, they have their own memory for code, for data and stack, while threads share code and data, and have their own stack.

But, what about sharing variables in forked processes? If we try to make it as simpler as in thread examples, it’s not going to work, because as they are different processes, they are using different memory zones for their data. For example:

In our ideal world, the MAIN process can see the values written by CHILD and vice versa, but the value the MAIN process can access is completely different than the value the CHILD can access, even using malloc before calling fork(). fork() copies the values of all variables before fork(), but the physical memory address is different.

To do that we have to work with shared memory, have to use mmap instead of malloc:

In this example, the main program waits for a second (with Thread.sleep()) while the tasks is scheduled, so we will see the output running, and after that second, the process stops, but not exits because the Timer is still active. We can write “timer.cancel()”, the task is cancelling too and the program exists.

The problem comes when resumin this task after a while, because we have to create the TimerTask object again, if we try to reuse the old TimerTask (task) created, we’ll get an exception.

But, as we must define the inner TimerTask class again, it’s interesting to create a separate derivate class. But let’s go further, we’ll make a TimerTask derivate that cancels and resumes itself, so our TimerTask (MyTimerTask) must know the Timer object, this way it can also re-schedule the task at different frequencies:

But, as we can see, we must re-create the task object when re-scheduling, what about passing information between the old task to the new task? We may want that, for example, to know what has happened before the new task is created. To do that, we can create a store class, where we can use the attributes to store useful information between schedules, and we can pass this variable to our MyTimerTask constructor, but to avoid external classes to modify the values of our store class, this new overloaded constructor can be private.

In the next example, we just store an integer (we can store as many things as we want), and with this object we will control the times the task was canceled, and make it exit after 5 times:
MyTimerTaskInfo.java

If we’re creating a multi-thread application and we’re also sharing information between the main thread and the secondary thread, or between threads, you must have in mind the type of access to that information.
For example, if we will only allow one thread to write on a variable and the other will just read we won’t have any problem in most cases, but if any thread can write a value at any time, we must be careful. If some threads are willing to write a variable at almost the same time, only the last value written will remain.

Another example, we have a film collection software, and at the moment we have 50 films stored. Another thread is going to synchronize an Internet server, but while the synchronization is running, we add 3 films more. The thread synchronizing may see 50 films, but the films sent can be a mix of the old and new films, so the server will think we’ve removed some films and we will have a problem.
In this case, we must protect the access to the critic section (our film list), so when we are adding data, the other thread can’t sync, and when the other thread is syncing, we must wait before adding anything. We will use for that mutual exclusion or mutex.

To try to make a visible example, we’re incrementing numbers, but we will insert a CPU eater task between the read of the value and the writing of the new value. The CPU-eater task can finish in a variable time interval, so one threads will finish this task before others. The desired result is the number incrementing to 10, but the real one differs a bit:

What has happened? Some threads read the variable when it was 0 (two occasions), so they both incremented to 1, others read the value when it was 1 (another two ones), and incremented to 2, in other cases, the variable was 2 and was incremented to 3…
So, several threads read the same value and when writing the new value, we didn’t have in mind the value could have changed by another thread while we were working. That is the race condition.

How can we fix that? The solution is coding structures that block access to the resource when it’s being used. For example, it some other thread has read the value of the variable, no other can, until a new value is written.
Do we lose performance? Yes, a bit, because we are waiting for other tasks instead of working together. But we avoid undesirable situations like the example before. But the threads may do also some other things outside the critical section, and this work can be done simultaneously. We will only block the critical section (when working on a number), when we will block other threads with a mutex.

I recently talked about using TimerTask in Java. Today we will pass variables or attributes to that TimerTask. We saw in the last article variable passing to an implicit or inner class, but now we’re creating a subclass and passing arguments through the constructor.

In this example, we’ve passed two arguments: 10, “Start: “, to the TimerTask (MyTimerTask) constructor, they will be the number of times the task must be launched before writing “TEN! ” on a String. On the other hand, we’ve implemented the method toString in MyTimerTask to write the value directly with System.out.println().

Easy, but, what about passing a callback to execute the System.out.println() ¿?

Let’s write an interface for the Callback (MyTimerCallback), then we will implement MyTimerTask and then we will make TimerEx implement MyTimerCallback, so we can put this function in the main class:

A few days ago, we talked about building our first multi-thread programs using POSIX threads. But soon a need arises: share data between the main thread and the recently launched thread, at least to indicate what we want it to do.
So we can use the last argument of pthread_create(), this is a void pointer we can use for whatever we want and pass the variable type we want. For example an integer:

123456789101112131415161718192021222324252627282930

#include <stdio.h>#include <stdlib.h>#include <pthread.h>

void*newtask(void*_number){int number =*(int*)_number;printf("The number I was asked for: %d\n", number);
pthread_exit(NULL);}

printf("Main process about to finish.\n");/* Last thing that main() should do */
pthread_exit(NULL);}

Remember to compile using pthread:

$ gcc -o simplepass simplepass.c -lpthread

This way, the thread can read the value we’ve passed him, as we can see if we run this little program. Just after starting newtask() we extract the number from the void pointer to an integer variable. But working a bit on this code we can realize this variable can be bi-directional, in other words, we can write a value from the secondary thread, and the main thread can read it. (Instead of doing this, we can use global variables, but I don’t like it much):

printf("Main process about to finish.\n");/* Last thing that main() should do */
pthread_exit(NULL);}

In older and slower computers, we may have to cary the value passed to usleep(), I’m just simulating a wait, it could be the time taken by additional computation. But using a sleep could be a problem, depending on the computer capabilities this value must change, or we could give a high value to make it compatible with more computers, but it could be a waste of time for modern computers. We can solve it many ways, but this first one will be a demo of what you shouldn’t do making an active wait:

printf("Main process about to finish.\n");/* Last thing that main() should do */
pthread_exit(NULL);}

In this case, we have a new variable (vars->ready), when it’s 1 it means the values are set by the secondary thread, so the main one can read them. The main thread, to wait for vars->ready is asking all the time for this value while it’s 0:

1

while(!vars->ready);

But it has a great disadvantage: the process is eating CPU while the condition is not met. We can see it clearer writing sleep(20) just before the secondary thread set ready to 1, and use a task manager to see how CPU use is going, our process will use a big percentage of CPU for nothing, just waiting. If we think about it and the secondary thread is doing any type of complex calculation, the main thread will be stealing CPU time obstructing the secondary one, making it slower, so we have to use another type of solution, like semaphores, signals, mutex, etc. But I’ll talk about them in another post.
Photo: OakleyOriginals (Flickr) CC-by

If you are creating a program which interacts with a server, this may interest you. First, we’re going to make a simple TCP client, we connect to a server and it will allow us to send whatever we write on our keyboard. We can’t receive anything from the server right now, it’s just a first step to reach our goal:

We’ll analyze the main() function directly, because usage() just display text and panic() makes the program exit on an error.
The first thing we do is to determine the server and port we will connect to, they both will be the first and second arguments to our program. Then we create the socket and build the client address. The port is 0 to let the system pick any port, we don’t mind which port is picked. Then bind() will assign the address to the socket.

Then, create the server address. We will use getbyhostname() before because we may resolve the server address first (if it comes with a hostname, or even if it comes as a IP address we will have to translate this address to a suitable format), so let’s connect() and begin sending messages.

In this case we’re using fgets() to ask the user for information through the standard input (usually keyboard) and then send() it, we must set the size to strlen(mens)+1 to include the string terminator ‘\0′ at the end of the string being sent.

Note: fgets() will send the line feed at the end when we press return. We can use a trim() function or filter it with:

Reception is similar to sending, but this time we don’t know the size of the message, so we provide a maximum size (or the size of our buffer), if the size of the data we are receiving is greater than our buffer, we can continue the message calling again and again recv().
As this message may not have a terminator, to display it properly on screen with printf() we must put the terminator manually at the end of the string.

It’s easy. The biggest problem, comes when we want the application to send and receive information whenever there is data on each side (sending or reception), making our application full duplex. With this code, right now, we just can give turns, make the user send information sometimes, and sometimes the application will receive data.
We will make a more complex example using select(), this function will detect whenever there is data waiting on some descriptors, and we can later choose whether to read the socket or the keyboard. But the most interesting is the timeout, we can give select() a wait timeout, when the function will stop waiting and the program can continue, it won’t block indefinitely waiting for data.

do{/* We must set all this information on each select we do */
FD_ZERO(&readmask);/* empty readmask *//* Then we put all the descriptors we want to wait for in a *//* mask = readmask */
FD_SET(socketfd,&readmask);
FD_SET(STDIN_FILENO,&readmask);/* STDIN_FILENO = 0 (standard input) *//* Timeout, we will stop waiting for information */
timeout.tv_sec=0;
timeout.tv_usec=100000;

/* The first parameter is the biggest descriptor+1. The first one
was 0, so every other descriptor will be bigger.*//* readfds = &readmask *//* writefds = we are not waiting for writefds *//* exceptfds = we are not waiting for exception fds */if(select(socketfd+1,&readmask, NULL, NULL,&timeout)==-1)
panic("Error on SELECT");

One of the biggest faults of an array is that we have to define it’s size in coding time. Sometimes it’s the easiest thing to do and it’s ok, we may waste some bytes (As in a little string when we create a 100 bytes array); sometimes we choose the right size (for example when creating an array with the names of the months). But many times we have no idea what the size it must have, if it is very small we probably will fail short and if it is very large, we will wast a lot of memory.

But one of the tools C has is the dinamic memory allocation (malloc function and derivates), but it can allocate memory for an structure, or an unidimensional array, if we want to allocate memory for a two-dimensional, three-dimensional array or so, we must allocate memory for each of the arrays composing the multi-dimensional array,

To have a dynamic array, we must have a double pointer, so it can be declared as:

1

int**myArray;

To allocate memory for that variable, first we must allocate all rows, and then, inside each row, we must allocate the columns. For example for a 3×3 array, we must:

Allocate 3 rows

Allocate 3 columns for 1st row

Allocate 3 columns for 2nd row

Allocate 3 columns for 3rd row

The next example is using calloc() function to allocate memory (we can do it with malloc with no difficulties:

The first function (calloc2d()) will allocate dinamically bi-dimensional arrays indicating rows and columns, free2d() will free memory allocated only indicating rows.
The function randomValues() assign random integer values to the array, to use the allocated memory, so the memory will be really used.

The program does three pauses, one before anything (press enter, to continue), another one when the memory is allocated, and the other one when everything in freed. We can use a memory analysis program, or even top to see what’s going on.

Timers are a useful tool to launch a task periodically, for example, if we are connected to a server, we can send status information to avoid disconnections, or launch a cron task, or even to run an animation (as the time goes by, we are changing a draw or a frame).

This program writes TIC and TOC alternatively each second, as the task is programmed each 1000ms. Is extremely easy, first, we must create a TimerTask object defining the task we want to launch inside the run() method (which we must @Override). Then, load a Timer and running the method schedule() we can call the desired task defining the time we want to wait before launching the first time and then the time we must wait before launching again the task.

If we only give one argument, the task will execute just once in N milliseconds. We can also specify Date objects in these arguments.

We can also call another schedule method called scheduleAtFixedRate, the difference is noticed when there are delays in executions (the tasks may be delayed due to other things in execution, instead of running each 1000ms, one task may be launched in 1500ms (because of these delays), scheduleAtFixedRate() will try to run the next task in 500ms to compensate lost time, but schedule() will run it after 1000ms, no matter if the task before was delayed.

The TimerTask in the example was created implicitly, it was an inner class, it’s absolutely valid for an example, or simple tasks. If we have something bigger it’s recommended to create a separate class for this task extending TimerTask superclass.

But, now the question is, how can we call an external variable? (External to the inner TimerTask we’ve created), call a variable of the main method of TimerEx. It has nothing to do with Timers but it’s interesting to tell:

In this example, we’ve called the task ten times, and then str String will be written on screen. So inside inner or nested classes we can access final variables. But we can’t write them, because they are final. There is an scape, we can create a class allowing us to modify it’s attributes, so the value we are interested in can be changed. Let’s create a class TicClass to get tic attribute out of the nested TimerTask