All Entries in the "Web forwarding" Category

This tutorial contains 301 redirect checker script which I have created after spending more than half an hour searching the Internet for something suitable I can use for my needs. While I was searching I found only separate web pages with fields where you can check one – two URLs or heavy applications that can be installed, and yet not covering my needs. Certainly, there were paid 301 redirect checker solutions which can be used to check 301 Permanent Redirect, but I am not sure whether it does worth to buy such. The purpose of this checker in most cases is only to see if a particular pages redirect permanent covering the SEO (Search Engine Optimization) requirements.

In my case part of the OnlineHowTo.net is moving over Free Tutorials Submit dot com, and from SEO perspective, the tutorials moved have to be with 301 redirect to their new URLs.

I decided to write it as Linux bash 301 redirect checker script because it can be run on almost any Linux machine with curl installed and easily added to the Crontab. I believe the script is rather easy to be understood and used, even by not so experienced administrators.

What the redirect script is doing is to gather the URLs you want to check from an external file (in my case: /opt/scripts/301URLs.txt ), where every URL is placed on a separate line like this:

I chose this way as most of the get-all-urls-from-domain applications provide the listing like this, but certainly you can choose (and then modify a little bit my script) different format.
Then using simple bash loop it reads every line one by one, processing it with CURL extracting the headers (curl -I option).
Every header contains information similar to:

XHTML

1

2

3

4

5

6

HTTP/1.1301MovedPermanently

Date:Thu,18Aug201117:23:57GMT

Server:Apache/2.2.3(CentOS)

Location:http://www.freetutorialssubmit.com/ftp-voyager/109

Connection:close

Content-Type:text/html;charset=iso-8859-1

Once we have this information we can easily egrep for the 301 Moved Permanently part and if it exist, the 301 redirect checker script continues with the next URL. If the redirect is something different it writes down the URL into the /tmp/301report.txt file.

Finally the script checks whether the /tmp/301report.txt file exist – it will be created only if there is even one different from 301 Moved Permanently redirect – and will send it via e-mail to the address you have set in the EMAIL variable. If there are no such file, the script will end with no report.

Here is the whole script which you can freely use and modify as per your needs:

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

#!/bin/bash

#The subject of the mail message - can be anything.

#For example something that triggers your e-mail client filters.

SUBJECT="301 Redirect Compromised"

#Here you must change with your e-mail address.

EMAIL="mail-to-report-to@yourdomain.com"

#Temporary file I am using to store non 301 redirected URLs.

LINKS=/tmp/301report.txt

#Here I make sure this file does not exist before processing the URLs.

rm-f/tmp/301report.txt

#All URLs which should be checked are in /opt/scripts/301URLs.txt one per line.

cat/opt/scripts/301URLs.txt|whilereadline;do

L=`curl-I-s$line|egrep'^HTTP\/1\.1 301'`

sleep5

if[-z"$L"]

then

echo$line>>$LINKS

fi

done

#Check if there were non 301 redirects stored and e-mail them if any

if[-f/tmp/301report.txt]

then

mail-s"$SUBJECT""$EMAIL"<$links

fi

#EOF

To be honest, I am using a little bit modified 301 redirect checker script, which is also checking whether the redirected URL is the one I need, but it is more complicated and may confuse you if you have to modify it.
Also, more elegant solution will be if you decide to use array, but this redirect checker script fits my needs perfect.
Also I have put sleep command as if the number of the URLs is too big, it may lead to server abuse and blocking the IP you are checking the redirect from. If you do not need the script to wait 5 seconds for every URL, just remove that line.

Hope this scrip will cover your needs as well. Your comments and suggestions are welcome.

Shell

1

/opt/scripts/301URLs.txt

Tags

This tutorial will explain how you can forward a web page to other URL using PHP forwarding.
It will be useful when you do not want to use other way of forwarding, like .htaccess permanent redirect, META redirect, Java redirect, etc.
This type of forwarding can be used only on a server with running PHP on it.

Tags

This tutorial will show you how to redirect a web page to a new location using
META redirection. This is useful in cases when you want a simple redirect to a new page or website. Also it can be used if you cannot or do not want to use other way for redirection.

To do a META forwarding create one ‘hml’ or ‘html’ file and pout the following line into it:

Do not forget to change ‘http:////domain.tld/folder/index.html’ with the URL that you want ot forward to.

If you have already a page that you want to redirect the META tag must be put on its right place. This is in the ‘<head>’ section in the file.
The next is an example where is the right place in the HTML structure.