Calculating Averages

The Simple Maths post seems to be the most popular article in the so-far short life of this blog.

It’s also something that I have received a few emails about recently, so I feel like posting a bit more on the subject.

I think that the code can speak for itself… We implement a loop, which calls the builtin read function (I’m not sure the “-p” flag, to provide a prompt, is universal. It does work with the Bash builtin. If it doesn’t work on your *nix, it’s really only for show, so you can live without it.

Because read works on standard input (aka “stdin”), it will work interactively from the keyboard, or direct from a file (one number per line).

We use two methods of doing maths in the shell:

expr, because it’s a simple and easily-read way to do simple maths: n=`expr $n + 1`

bc, because it is more powerful. Do have a play with bc interactively, it can do a lot... see below.

So, we can write a fairly simple script (read down, it's only actually 11 lines of code without the comments), which is actually quite versatile - it can do running averages, it can be interactive or run from cron, called from another script, even used as a function.

So, here's the code. It should be fairly self-explanatory, but do have a look at the interactive bc sample session below, to see what we are doing with bc. Also, do play with bc (some Linux distros have dropped it from the default install recently, so you'll have to yast -i bc, or equivalent)

The Script - Calculate Averages

#!/bin/sh
# Calculate mean (average) of integer data
# Initialise the variables
n=0 # n being the number of (valid) data provided
sum=0 # sum being the running total of all data
# Note that by using ^D (aka "EOF") to quit, this
# script will work just as well interactively, as
# when provided with a file containing the data.
while read -p "Enter a number (^D to quit): " x
do
# expr is useful for simple maths
sum=`expr $sum + $x`
# If this fails, it was non-numeric input
if [ "$?" -eq "0" ]; then
# Okay, it was valid input.
n=`expr $n + 1`
# We can provide a "running average" here;
# I'll comment it out for now.
# echo "Running Average:"
# echo "scale=2;$sum/$n" | bc
# echo
fi
done
# Okay, we've done the loop.
# Present the data.
echo "Overall Average:"
# bc is more useful than expr for
# more involved maths, though its
# syntax, particularly in a script,
# is possibly less obvious.
# Using bc interactively is easier
# than using it in a shell script
echo "scale=2;$sum/$n" | bc

I ran this script on 10’s of thousands of numbers and it takes a *long* time. That’s because you spawn an expr not once but twice for every number you average. Here is the script with your expr statement changed to native bash arithmetic expansion.

Thanks Bruce. There are lots of things that Bash can do that Bourne can’t do.

These days, most *nix boxes have Bash available, but /bin/sh still points to the Bourne shell. Indeed, Debian is going back from /bin/sh being bash, to dash. Ubuntu has already replaced /bin/sh with dash.

/bin/sh in Linux is on most distros a symlink to bash (or dash) and when run it actually runs “Bash in POSIX compatibility mode”, which should be like Bourne Shell, but neither Bash nor Dash is good for actually testing if your script runs on real Bourne Shell.

For those wanting to make sure their script is Bourne Shell compatible but you don’t have actual Bourne Shell available, Heirloom Bourne Shell is something you may want to look into – comes in source package, no configure script, you just have to edit very simple modifications to very simple Makefile, compile and install and you have Bourne Shell with NO extensions whatsoever – with that I’ve learned a lot on how much imagination and crazy hacks you may have to achieve to do things that in Bash are quite normal things to do :) It may be fun, I for one like to do some hobby projects “just to see if I can” where I try to do something in Bourne Shell – like transforming script relying on recursive function calls into Bourne Shell which has no function local variables, only script global ones (hint: subprocess inside function can help, of course the things you can do are limited, subprocess can have it’s own variables, but you can’t change variables outside it, he-he).