All Entries in the "Bash Scripting" Category

Bellow I will present script checking Load Average on a Linux server which will send report if it becomes to high.
It is using Linux command ‘uptime’ which is pulling out server’s uptime as well as its Load Average:

Shell

1

2

# uptime

21:53:51up43days,2:19,3users,load average:1.63,0.70,8.29

In my script bellow I am using not the last minute load average, but the last five minutes (the second) one.
If you are not aware you can interpret a load average of “1.63, 0.70, 7.89” on a single-CPU system as:

– during the last minute, the system was overloaded by 63% on average (1.63 runnable processes, so that 0.73 processes had to wait for a turn for a single CPU system on average).

– during the last 5 minutes, the CPU was idling 30% of the time on average.

– during the last 15 minutes, the system was overloaded 698% on average (7.89 runnable processes, so that 6.98 processes had to wait for a turn for a single CPU system on average).

I have chosen the five minute interval as sending mails every minute is too aggressive in case of a server load. Also it could be something too short and handled by the server without notification.

This tutorial contains 301 redirect checker script which I have created after spending more than half an hour searching the Internet for something suitable I can use for my needs. While I was searching I found only separate web pages with fields where you can check one – two URLs or heavy applications that can be installed, and yet not covering my needs. Certainly, there were paid 301 redirect checker solutions which can be used to check 301 Permanent Redirect, but I am not sure whether it does worth to buy such. The purpose of this checker in most cases is only to see if a particular pages redirect permanent covering the SEO (Search Engine Optimization) requirements.

In my case part of the OnlineHowTo.net is moving over Free Tutorials Submit dot com, and from SEO perspective, the tutorials moved have to be with 301 redirect to their new URLs.

I decided to write it as Linux bash 301 redirect checker script because it can be run on almost any Linux machine with curl installed and easily added to the Crontab. I believe the script is rather easy to be understood and used, even by not so experienced administrators.

What the redirect script is doing is to gather the URLs you want to check from an external file (in my case: /opt/scripts/301URLs.txt ), where every URL is placed on a separate line like this:

I chose this way as most of the get-all-urls-from-domain applications provide the listing like this, but certainly you can choose (and then modify a little bit my script) different format.
Then using simple bash loop it reads every line one by one, processing it with CURL extracting the headers (curl -I option).
Every header contains information similar to:

XHTML

1

2

3

4

5

6

HTTP/1.1301MovedPermanently

Date:Thu,18Aug201117:23:57GMT

Server:Apache/2.2.3(CentOS)

Location:http://www.freetutorialssubmit.com/ftp-voyager/109

Connection:close

Content-Type:text/html;charset=iso-8859-1

Once we have this information we can easily egrep for the 301 Moved Permanently part and if it exist, the 301 redirect checker script continues with the next URL. If the redirect is something different it writes down the URL into the /tmp/301report.txt file.

Finally the script checks whether the /tmp/301report.txt file exist – it will be created only if there is even one different from 301 Moved Permanently redirect – and will send it via e-mail to the address you have set in the EMAIL variable. If there are no such file, the script will end with no report.

Here is the whole script which you can freely use and modify as per your needs:

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

#!/bin/bash

#The subject of the mail message - can be anything.

#For example something that triggers your e-mail client filters.

SUBJECT="301 Redirect Compromised"

#Here you must change with your e-mail address.

EMAIL="mail-to-report-to@yourdomain.com"

#Temporary file I am using to store non 301 redirected URLs.

LINKS=/tmp/301report.txt

#Here I make sure this file does not exist before processing the URLs.

rm-f/tmp/301report.txt

#All URLs which should be checked are in /opt/scripts/301URLs.txt one per line.

cat/opt/scripts/301URLs.txt|whilereadline;do

L=`curl-I-s$line|egrep'^HTTP\/1\.1 301'`

sleep5

if[-z"$L"]

then

echo$line>>$LINKS

fi

done

#Check if there were non 301 redirects stored and e-mail them if any

if[-f/tmp/301report.txt]

then

mail-s"$SUBJECT""$EMAIL"<$links

fi

#EOF

To be honest, I am using a little bit modified 301 redirect checker script, which is also checking whether the redirected URL is the one I need, but it is more complicated and may confuse you if you have to modify it.
Also, more elegant solution will be if you decide to use array, but this redirect checker script fits my needs perfect.
Also I have put sleep command as if the number of the URLs is too big, it may lead to server abuse and blocking the IP you are checking the redirect from. If you do not need the script to wait 5 seconds for every URL, just remove that line.

Hope this scrip will cover your needs as well. Your comments and suggestions are welcome.

Shell

1

/opt/scripts/301URLs.txt

Tags

This tutorial will show you how to create a test file under RedHat-like Linux operating systems. It will be useful when you need to test FTP, Mail, Samba or any protocol and you do not have big file handy to send.

To create a test file simply copy the following line in your Linux shell:

This tutorial will show you how, with one raw command to rename multiple files using bash scripting.
It is written for Linux/Unix bash shell and will be useful when you want to rename files with same extension.