Archive for the Category: '
Bash '

As you know, the variables in bash are global by default (their scope is from the initialisation (usually when you start using it), to the destruction of the variable (usually at the end of the script). The variable will be available through all the functions within the script.

Now, if you want to reduce the scope of a variable, and tie it to a specific function, you usually render it unique (by starting with an underscore for instance), then initialise it at the beginning of your function, then destroy it (unset) at the end of the function.

There is another option : you can use the keyword “local” like in other languages.

Here is an example on how to use it :

#!/bin/bash

MyFunction () {

local TheVar

TheVar=1

echo “In The Function ${TheVar}”

}

declare -i TheVar

TheVar=99

echo “Before the Function ${TheVar}”

MyFunction

echo “After the Function ${TheVar}”

Here is the output :

Before the Function 99

In The Function 1

After the Function 99

And if you comment out the “local” part :

Before the Function 99

In The Function 1

After the Function 1

Another note on this.

According to my tests done on my bash version (GNU bash, version 3.2.51(1)-release), you can achieve the same results by simply declaring your variable within a function.

e.g.

#!/bin/bash

MyFunction () {

declare -i TheVar

TheVar=1

echo “In The Function ${TheVar}”

}

declare -i TheVar

TheVar=99

echo “Before the Function ${TheVar}”

MyFunction

echo “After the Function ${TheVar}”

Has the following results :

Before the Function 99

In The Function 1

After the Function 99

Note also that, if you re-use the same variable name (as per our example) once you declare a local variable, the content of the global variable of the same name is not accessible within the function anymore !

e.g.

#!/bin/bash

MyFunction () {

#local TheVar

echo “Before declaration : ${TheVar}”

declare -i TheVar

echo “After declaration, before assignement : ${TheVar}”

TheVar=1

echo “After assignement : ${TheVar}”

}

declare -i TheVar

TheVar=99

echo “Before the Function ${TheVar}”

MyFunction

echo “After the Function ${TheVar}”

Gives the following results :

Before the Function 99

Before declaration : 99

After declaration, before assignement :

After assignement : 1

After the Function 99

Think about this the next time you do recursive functions, this might help. Counters (while [ i -lt 200 ];) should always be declared local within a function. Keep in mind also that “local” keyword is only available within a function.

If you happen to write some scripts for sftp, ftp (…) files transfer, you might want to check if the remote host does accept the connection on the port you specify before triggering anything.

What’s the point of doing this test? – it can be discussed, actually. But one might prefer to handle the case where the host is not responding or refusing the connection otherwise than by parsing the answer provided through the command line by the sftp, ftp (sftp..) binary.

That’s what I wanted to do and my next challenge was to find a way to do this test without using telnet, ftp, sftp, ssh (etc.). The thing is most of the example you can find on the web are using these commands.

The trick is to use the bash sockets…And especially this one: /dev/tcp/. We can try to write something in the socket

exec 3>/dev/tcp/REMOTE_DEST/REMOTE_PORT

and depending on the success or failure assume we could connect… or not. For example, let’s do the following:

doLog, quit, doVerbose are functions within the same script dedicated to handle the log files, manage exits and finally manage the verbose mode, if triggered while executing the script.

${REMOTE_DEST} and ${REMOTE_PORT} are variables set outside the fuction.

To go further:

/dev/tcp is one of the pseudo-device unix like systems have. We are more familiar with /dev/null when for example we are only interested with the STDOUT stream and not STDERR (the famous 2>/dev/null) but there are actually a few of them: /dev/zero, /dev/random, /dev/full for example.

You can read the wikipedia page on device files that is quite interesting.

Imagine you have a script, that will need deactivation. You do not want someone to find the deactivator easily (if you want to “lock” a script for example).

There are may ways to do this, one easy one is to create a script with name ” ” (space) and to execute it in your main script.

To create your space script :

cat >\
echo ” You should not start this script”
exit 0

and then, you add the following line somewhere, for example after the functions.

. ./\

The script will exit with the message at that place, and it is very difficult for the user to find the cause.

Another option can be to encrypt the script using crypt (you can install it if you do not know where it is) so that the user cannot use grep on all files to find the blocking one. You simply decrypt the file before running it (this can be done by your “space” script.

Oh, yeah, and a quick important note, on a related but different issue.

If you want to deactivate the following function :

function removeall {
rm -Rf .
}

DO NOT try to comment it out like this :

#function removeall {
rm -Rf .
}

Because this will simply remove everything every time we try to load your library !

Do it like this :

function removeall {
return 0
rm -Rf .
}

This is all for today, tomorrow we will talk about the misuse of the tr function.

The last post before a long week-end (I will not be able to post tomorrow, and on Monday and Tuesday. I’ll see you on wednesday.

Today, the topic is “The difference between find -exec and find | xargs”

The situation

You have to look into a specific directory and subdirectories for all the files named “*log” because you are quite sure one of them contains the word “error”.

You have two ways to do this :

1 – using -exec

This is a parameter to the find commands that allows you to do an extra command on the found files. The syntax is as follow :

-exec <command> {} \;

<command> replaces your command

{} will be replaced by the name of the found file.

\; terminates the command

i.e. you type :

find . -name \*log -exec grep -i error {} \;

and it will return all the lines containing error, regardless of the case.

2 – using xargs.

xargs is a command that allows the piped data to be passed as parameter to another command. The syntax is as follow :

| xargs <command>

xargs simply puts at the end of the command the piped data. i.e. you type :

find . -name \*log | xargs grep -i

3 – Using a while loop.

Yes, you can do like that, but it is not the topic of this discussion.

The difference

What is the main difference between the two?

-exec is going to take each file, and execute the command with it. Using this, you will get a list of all the lines, but not the name of the file, as grep assumes you know which file he talks about, as you have passed the name as parameter !

A sub process will be spawn for each file to be checked.

xargs, on the other end, is going to pass the complete list of files to grep. The name of the files will be shown in the result list.

The separator for the parameters will be the space, and this is OK as long as there is no space in the names of the files (but who puts spaces in the names of the files ? Ah, ok, users…).

In fact, the fail proof way to deal with this specific request is to make the while loop.

The conclusion

Both ways can be useful, depending of the request and the situation. It is important that you understand the different ways those tools work, so that you can choose which one is the best for your usage.

Thank you for reading, and see you next wednesday ! (told you, long week-end for me 🙂 )

is there to define the variables i and j as integer (-i) (equivalent to int i,j; in c)

The next line is parsing the file, and looping each line between the “do” and “done”. The line is put in the variable “line”

If you had written

cat tempo1 | while read JamesTKirk

then the variable containing the line would have been JamesTKirk, which is a bit long to type and also a bit of an overkill to have one of the best Starfleet Captain as your variable.

Next line speaks by itself :

if echo $line | grep COMP >/dev/null 2>&1; then

means “if the result of the command “echo $line | grep COMP” is 0, then execute the following” and of course, this will only be the case if the line in question contains the word “COMP” somewhere. We could have done maybe with “grep -e ‘^COMP'”, or with the funny test we’ll see tomorrow, but let’s not split the hair.

i=`expr \`echo $line | cut -f2- -d’ ‘\“

expr allows you to do basic arithmetical calculation. This line means “Take everything after the first field on the line, space being the separator, then compute it and store it in the variable “i”“.

let j=j+i

let is another way to do arithmetical operations, using integer variables.

and voila, done.

Should you require more assistance, feel free to post in the comments or to man the command you want to know about.

Thank you for reading, and see you tomorrow for a special test operator.

You receive a new requirement from your customer (I did receive it a couple of times). You need to develop a program that will :

– Not consume CPU or disk until it is necessary.

– Read the content of a directory, then process each file in a directory A (move it to another directory B ) one by one. Not two files at the same time.

– Once the file has been “taken” by another process (no file in directory B), the next file can be published.

– You can’t have the system polling every minute or so, because the files need to be copied as fast as possible.

– The process that provides the files can start another process, but can’t wait more that a millisecond before giving the hand back.

Good luck…

The Solution

Well, at least the one I found, was to have a main process going to sleep, then being awoken by another one (triggered by the “file-provider”), and starts polling the directory A until it is empty. Then it goes back to sleep.

How ? well, like this :

First, we need to trap the “KILLUSR” signal :

trap “wakeup” 30

Then, we need to create a script, on the fly, that will be called to signal the main script that there is something to process (the “trigger” process)

As you know, Unix uses signals to communicate with processes. A process receives a signal, then takes an action. If no action is defined, the default action for most signals is to kill the process… which is maybe something you want to avoid.

The problem.

You have written the best script in the world, that does… let’s say… make ice-cream (we can dream).

The problem is that if someone kills your script (i.e. kill -15, SIGTEM, which is the default one), you process will simply stop, and throw ice-cream everywhere on the walls. (Ice-cream is here an image for temporary files).

How can you catch this message and close the ice-cream tap and clean-up a bit ?

The Solution.

Use trap.

At the beginning of your script, you can add the following line (after the shebang) :

trap 15 cleanup

This will trap the default SIGTERM, then start the procedure “cleanup” ( you need to define the “cleanup” procedure somewhere).

You can define a function that will do the cleanup for you, maybe add a message in the log, kill sub-processes, send mail, whatever is needed.

The only thing, is that you need to fail proof your function, in order to be sure that your process is not going in an infinite loop, because the way to kill it afterwards is to use another kill.

Note also that not all the signals might be trapped. Some of them, like the signal 9 (SIGKILL) is not trappable in a script.

Thank you for reading, and if I find some time, I’ll talk tomorrow about an inventive way to use the trap function.

So today, we’ll see how to pass a variable (and not the content of a variable) to a function.

The Problem

You need to create a function that updates one or multiple variables given as parameter. For example, let’s say you want to create a “to_upper” function, that will take a variable as parameter, then modify it.

If you do it like this :

to_upper $mytext

You will just receive the content of the variable. Then, what do you want to do with it ? Print it ? You can not return it to the variable, as you do not know it.

From there, you can execute everything, like SQL queries, and return them in the calling variable or in another variable… Of course, you still need to protect your function, checking that the given parameter is a name of a variable, and not the variable itself.