I provide some pointers for people to learn programming on the Debian system enough to trace the packaged source code. Here are notable packages and corresponding documentation packages for programing.

Online references are available by typing "man name" after installing manpages and manpages-dev packages. Online references for the GNU tools are available by typing "info program_name" after installing the pertinent documentation packages. You may need to include the contrib and non-free archives in addition to the main archive since some GFDL documentations are not considered to be DFSG compliant.

Warning

Do not use "test" as the name of an executable test file. "test" is a shell builtin.

Caution

You should install software programs directly compiled from source into "/usr/local" or "/opt" to avoid collision with system programs.

12.1.1. POSIX shell compatibility

Many system scripts may be interpreted by any one of POSIX shells (see Table 1.13, “List of shell programs”). The default shell for the system is "/bin/sh" which is a symlink pointing to the actual program.

bash(1) for lenny or older

dash(1) for squeeze or newer

Avoid writing a shell script with bashisms or zshisms to make it portable among all POSIX shells. You can check it using checkbashisms(1).

Table 12.2. List of typical bashisms

Good: POSIX

Avoid: bashism

if [ "$foo" = "$bar" ] ; then …

if [ "$foo" == "$bar" ] ; then …

diff -u file.c.orig file.c

diff -u file.c{.orig,}

mkdir /foobar /foobaz

mkdir /foo{bar,baz}

funcname() { … }

function funcname() { … }

octal format: "\377"

hexadecimal format: "\xff"

The "echo" command must be used with following cares since its implementation differs among shell builtin and external commands.

Avoid using command option "-e" and "-E".

Avoid using any command options except "-n".

Avoid using escape sequences in the string since their handling varies.

Note

Although "-n" option is not really POSIX syntax, it is generally accepted.

Tip

Use the "printf" command instead of the "echo" command if you need to embed escape sequences in the output string.

12.1.2. Shell parameters

Special shell parameters are frequently used in the shell script.

Table 12.3. List of shell parameters

shell parameter

value

$0

name of the shell or shell script

$1

first(1) shell argument

$9

ninth(9) shell argument

$#

number of positional parameters

"$*"

"$1 $2 $3 $4 … "

"$@"

"$1" "$2" "$3" "$4" …

$?

exit status of the most recent command

$$

PID of this shell script

$!

PID of most recently started background job

Basic parameter expansions to remember are followings.

Table 12.4. List of shell parameter expansions

parameter expression form

value if var is set

value if var is not set

${var:-string}

"$var"

"string"

${var:+string}

"string"

"null"

${var:=string}

"$var"

"string" (and run "var=string")

${var:?string}

"$var"

echo "string" to stderr (and exit with error)

Here, the colon ":" in all of these operators is actually optional.

with ":" = operator test for exist and not null

without ":" = operator test for exist only

Table 12.5. List of key shell parameter substitutions

parameter substitution form

result

${var%suffix}

remove smallest suffix pattern

${var%%suffix}

remove largest suffix pattern

${var#prefix}

remove smallest prefix pattern

${var##prefix}

remove largest prefix pattern

12.1.3. Shell conditionals

Each command returns an exit status which can be used for conditional expressions.

You may wish to create launcher on the desktop with command set something like "/usr/local/bin/gmkrs02 %d".

12.2. Make

Make is a utility to maintain groups of programs. Upon execution of make(1), make read the rule file, "Makefile", and updates a target if it depends on prerequisite files that have been modified since the target was last modified, or if the target does not exist. The execution of these updates may occur concurrently.

Here " [TAB] " is a TAB code. Each line is interpreted by the shell after make variable substitution. Use "\" at the end of a line to continue the script. Use "$$" to enter "$" for environment values for a shell script.

Implicit rules for the target and prerequisites can be written, for example, by the following.

%.o: %.c header.h

Here, the target contains the character "%" (exactly one of them). The "%" can match any nonempty substring in the actual target filenames. The prerequisites likewise use "%" to show how their names relate to the actual target name.

Here, "-lm" is needed to link library "/usr/lib/libm.so" from the libc6 package for sqrt(3). The actual library is in "/lib/" with filename "libm.so.6", which is a symlink to "libm-2.7.so".

Look at the last parameter in the output text. There are more than 10 characters even though "%10s" is specified.

The use of pointer memory operation functions without boundary checks, such as sprintf(3) and strcpy(3), is deprecated to prevent buffer overflow exploits that leverage the above overrun effects. Instead, use snprintf(3) and strncpy(3).

12.4. Debug

Debug is important part of programing activities. Knowing how to debug programs makes you a good Debian user who can produce meaningful bug reports.

12.4.1. Basic gdb execution

Primary debugger on Debian is gdb(1) which enables you to inspect a program while it executes.

Let's install gdb and related programs by the following.

# apt-get install gdb gdb-doc build-essential devscripts

Good tutorial of gdb is provided by "info gdb" or found elsewhere on the web.
Here is a simple example of using gdb(1) on a "program" compiled with the "-g" option to produce debugging information.

Many gdb(1) commands can be abbreviated. Tab expansion works as in the shell.

12.4.2. Debugging the Debian package

Since all installed binaries should be stripped on the Debian system by default, most debugging symbols are removed in the normal package. In order to debug Debian packages with gdb(1), corresponding *-dbg packages need to be installed (e.g. libc6-dbg in the case of libc6).

If a package to be debugged does not provide its *-dbg package, you need to install it after rebuilding it by the following.

Bump package version to one which does not collide with official Debian versions, e.g. one appended with "+debug1" when recompiling existing package version, or one appended with "~pre1" when compiling unreleased package version by the following.

You need to check build scripts of the package and ensure to use "CFLAGS=-g -Wall" for compiling binaries.

12.4.3. Obtaining backtrace

When you encounter program crash, reporting bug report with cut-and-pasted backtrace information is a good idea.

The backtrace can be obtained by the following steps.

Run the program under gdb(1).

Reproduce crash.

It causes you to be dropped back to the gdb prompt.

Type "bt" at the gdb prompt.

In case of program freeze, you can crash the program by pressing Ctrl-C in the terminal running gdb to obtain gdb prompt.

Tip

Often, you see a backtrace where one or more of the top lines are in "malloc()" or "g_malloc()". When this happens, chances are your backtrace isn't very useful. The easiest way to find some useful information is to set the environment variable "$MALLOC_CHECK_" to a value of 2 (malloc(3)). You can do this while running gdb by doing the following.

$ MALLOC_CHECK_=2 gdb hello

12.4.4. Advanced gdb commands

Table 12.12. List of advanced gdb commands

command

description for command objectives

(gdb) thread apply all bt

get a backtrace for all threads for multi-threaded program

(gdb) bt full

get parameters came on the stack of function calls

(gdb) thread apply all bt full

get a backtrace and parameters as the combination of the preceding options

(gdb) thread apply all bt full 10

get a backtrace and parameters for top 10 calls to cut off irrelevant output

(gdb) set logging on

write log of gdb output to a file (the default is "gdb.txt")

12.4.5. Debugging X Errors

If a GNOME program preview1 has received an X error, you should see a message as follows.

The program 'preview1' received an X Window System error.

If this is the case, you can try running the program with "--sync", and break on the "gdk_x_error" function in order to obtain a backtrace.

12.4.6. Check dependency on libraries

Use ldd(1) to find out a program's dependency on libraries by the followings.

12.5. Flex — a better Lex

You need to provide your own "main()" and "yywrap()". Otherwise, your flex program should look like this to compile without a library. This is because that "yywrap" is a macro and "%option main" turns on "%option noyywrap" implicitly.

%option main
%%
.|\n ECHO ;
%%

Alternatively, you may compile with the "-lfl" linker option at the end of your cc(1) command line (like AT&T-Lex with "-ll"). No "%option" is needed in this case.

12.7.1. Compile and install a program

Do not overwrite system files with your compiled programs when installing them.

Debian does not touch files in "/usr/local/" or "/opt". So if you compile a program from source, install it into "/usr/local/" so it does not interfere with Debian.

$ cd src
$ ./configure --prefix=/usr/local
$ make
$ make install # this puts the files in the system

12.7.2. Uninstall program

If you have the original source and if it uses autoconf(1)/automake(1) and if you can remember how you configured it, execute as follows to uninstall the program.

$ ./configure "all-of-the-options-you-gave-it"
# make uninstall

Alternatively, if you are absolutely sure that the install process puts files only under "/usr/local/" and there is nothing important there, you can erase all its contents by the following.

# find /usr/local -type f -print0 | xargs -0 rm -f

If you are not sure where files are installed, you should consider using checkinstall(8) from the checkinstall package, which provides a clean path for the uninstall. It now supports to create a Debian package with "-D" option.

12.8. Perl short script madness

Although any AWK scripts can be automatically rewritten in Perl using a2p(1), one-liner AWK scripts are best converted to one-liner Perl scripts manually.

Let's think following AWK script snippet.

awk '($2=="1957") { print $3 }' |

This is equivalent to any one of the following lines.

perl -ne '@f=split; if ($f[1] eq "1957") { print "$f[2]\n"}' |

perl -ne 'if ((@f=split)[1] eq "1957") { print "$f[2]\n"}' |

perl -ne '@f=split; print $f[2] if ( $f[1]==1957 )' |

perl -lane 'print $F[2] if $F[1] eq "1957"' |

perl -lane 'print$F[2]if$F[1]eq+1957' |

The last one is a riddle. It took advantage of following Perl features.

The whitespace is optional.

The automatic conversion exists from number to the string.

See perlrun(1) for the command-line options. For more crazy Perl scripts, Perl Golf may be interesting.

12.9. Web

Filling and clicking on the form entries sends one of the following URL string with encoded parameters from the browser to the web server.

"http://www.foo.dom/cgi-bin/program.pl?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3"

"http://www.foo.dom/cgi-bin/program.py?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3"

"http://www.foo.dom/program.php?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3"

"%nn" in URL is replaced with a character with hexadecimal nn value.

The environment variable is set as: "QUERY_STRING="VAR1=VAL1 VAR2=VAL2 VAR3=VAL3"".

CGI program (any one of "program.*") on the web server executes itself with the environment variable "$QUERY_STRING".

stdout of CGI program is sent to the web browser and is presented as an interactive dynamic web page.

For security reasons it is better not to hand craft new hacks for parsing CGI parameters. There are established modules for them in Perl and Python. PHP comes with these functionalities. When client data storage is needed, HTTP cookies are used. When client side data processing is needed, Javascript is frequently used.