Tag Archives: linux

Post navigation

We needed to forward port 3307 to port 3306 to get around a new company wide firewall restriction blocking access to port 3306 (our MySQL server). It was a pain to find how to get port forwarding working in Ubuntu Server 12.04, which uses “ufw” as a front end to “iptables”. I couldn’t get it working without specifically forwarding to my IP, which I shouldn’t need to do (but at least it works).

This will forward port 3307 to 3306 so you can connect to your.ip.add.ress:3307 and have it automatically connect to a server (such as MySQL) on port 3306.

To do this you need “ufw” to be enabled, which you can check with “sudo ufw status”.

As part of setting up and testing routing rules in XenServer 6 I used the built in “lokkit” tool to temporarily turn off the firewall. Unfortunately, just opening the tool overwrote our custom “/etc/sysconfig/iptables” rules and cleared the file. This wasn’t a huge problem as we had a backup and just recreated it (you shouldn’t really be editing iptables manually anyway). On restarting iptables using “/etc/init.d/iptables restart” we received the error:

Loading additional iptables modules: ip_conntrack_netbios_n[FAILED]

This is very easy to fix and is due to a setting in “/etc/sysconfig/iptables-config” which was set by “lokkit” by default. The issue is that iptables is trying to load the “ip_conntrack_netbios_ns” kernel module, which doesn’t exist by default in XenServer (and other linux distributions).

Find the following line at the top of “/etc/sysconfig/iptables-config”:

IPTABLES_MODULES=”ip_conntrack_netbios_ns”

And set to:

IPTABLES_MODULES=””

A few people have said to also set “IPTABLES_MODULES_UNLOAD” to =”no”:

IPTABLES_MODULES_UNLOAD=”no”

But I found that “/etc/init.d/iptables restart” still failed so I left it as “yes”. You may be able to set to “no” so try this first.

This will stop the missing kernel module being loaded and allow iptables to start properly.

If you get any other errors about loading modules when restarting iptables, check “/etc/sysconfig/iptables-config” isn’t trying to load something in “IPTABLES_MODULES=” that you don’t have installed.

The steps are; install subversion, create the repository directories, set access control, set subversion to run at boot.

To install subversion in ubuntu just run:

sudo apt-get install subversion

Now create a directory to hold your subversion repositories, in my case I used “/home/svn”:

sudo mkdir /home/svn

Create a repository folder, for example “svnrepo1″, within this directory:

sudo mkdir /home/svn/svnrepo1

Now you can use the “svnadmin” program that comes as part of the subversion package to create a SVN repository within this folder:

sudo svnadmin create /home/svn/svnrepo1

The configuration file for the repository is created as “/home/svn/svnrepo1/conf/svnserve.conf” and contains the option to enable password protection as well as a lot of other useful settings. The important lines to uncomment to force password access are:

anon-access = none
auth-access = write

By setting “anon-access” to “none” you force people to enter passwords on connecting to the SVN. Now set up password protected access by uncommenting the following:

password-db = passwd

Settings “password-db” to “passwd” means the list of users and passwords in the “/home/svn/svnrepo1/conf/passwd” file will be used to check if someone has access. In a lot of cases it makes sense to keep this “passwd” file somewhere else so it can be used for all your repositories. In my case I set it to:

password-db = /home/svn/passwd

Just make sure to set the passwd file to be only readable by root:

sudo chmod 600 /home/svn/passwd

The “passwd” file is actually a very simple text file and looks something like:

[users]
harry = harrypassword
sally = sallypassword

Once the SVN server is configured and a repository set up as above you can run the SVN server using:

svnserve -d –foreground -r /home/svn

To make sure the SVN server starts at boot you need to set up init.d. Do this by creating and editing a file “/etc/init.d/svnserve” (I use “nano” to do my text editing on the command line):

sudo nano /etc/init.d/svnserve

Now paste in the contents of the odyniec.net init.d script. This script covers everything you need to start, stop and restart the “svnserve” program at boot so your SVN server can listen to all svn:// connections. There are alternatives to using this script, but this works and is simple to set up. Make sure you change the line with “DAEMON_ARGS” to point to the right place of “/home/svn”:

DAEMON_ARGS=”-d -r /home/svn”

Now tell Ubuntu to update its startup routine to include this new script:

sudo update-rc.d svnserve defaults

Reboot the server to make sure everything is working as expected.

You can now start, stop or restart the automatically booted SVN server using the following commands:

Connecting to your SVN server can be done using something like TortoiseSVN and the URL you use to connect to the “svnrepo1″ repository you just set up is:

svn://your.serv.er.ip/svnrepo1

There is a lot more you can do with the SVN configuration, such as adding group support etc, but this is the quickest way to set up a standard SVN server on Ubuntu to accept svn:// connections using “svnserve”.

As part of a larger daily backup cron job script I needed to quickly backup my MySQL databases to individual compressed “gzip” .GZ files. The command to do this is very easy, just run the command and pipe it to “gzip”:

mysqldump -u USERNAME -pPASSWORD DATABASENAME | gzip > OUTPUTFILE.gz

This requires you to actually put in the USERNAME and PASSWORD on the command line, which is obviously a bad idea due to logging of commands and other security reasons.

The MySQL recommended way of doing this is to instead use a separate file containing the login details. You use “mysqldump” with the argument “–defaults-extra-file” and specify the location of a configuration file such as “/root/mysqldetails.cnf”. It is a good idea to create this file and “chown” as root and “chmod” it to be “0400″ which will make it read-only by the “root” user.

The easiest way of doing this is to use dm_crypt‘s “cryptsetup” on your USB drive, create a keyfile then set the options in “/etc/fstab” and “/etc/crypttab”. By using a keyfile you can get the drive to automatically mount without having to type in your encryption password. I was doing this on a bare install of CentOS 6.3 but the steps should be similar on other distros with “cryptsetup” installed.

I needed to back up some important (and confidential) files to a USB portable drive that I wanted to encrypt with full disk encryption. You can do this in a variety of ways but the method here was the easiest I found. More information can be found at Brad’s Blog and HowtoForge.

Encrypting and mounting your USB drive

First you need to physically plug in your USB drive to the machine and then unmount it if it automatically mounts. I performed all the commands here using the root user. In my case, when I plugged in the USB drive it was found as “/dev/sdb” and automatically mounted by CentOS. To unmount:

umount /dev/sdb

Now the USB drive needs to be formatted using “cryptsetup” and the “luksFormat” command:

cryptsetup luksFormat /dev/sdb

The tool will give you a warning about overwriting data, which you need to confirm by typing an uppercase “YES”. You then type in and confirm your LUKS passphrase, which will be used to unlock the drive in future. This passphrase is also used later when creating the keyfile.

Now you can create a device mapper for the drive using “cryptsetup” and the “luksOpen” command. I called my mapper “secretvol” in this example so the drive will be mapped to “/dev/mapper/secretvol”. You will be prompted for the passphrase:

cryptsetup luksOpen /dev/sdb secretvol

Now before you can mount your newly mapped device you need to format the file system (I used ext3):

mkfs.ext3 /dev/mapper/secretvol

Now you can mount the USB drive. Make sure you have created the mount point (in my case “/mnt/encrypteddrive”) first then mount it with:

To unmount and lock the drive by closing the device mapper with the “luksClose” command:

umount /dev/mapper/secretvol
cryptsetup luksClose secretvol

Creating a keyfile to avoid entering your passphrase manually

A keyfile is good as it means you can unlock your USB drive without having to manually type the passphrase. To create a keyfile “/root/keyfile” for your device using “cryptsetup” and the “luksAddKey” command enter the following (you will need to enter your passphrase). The first command creates a random 4096 byte file, the second makes it read only to root and the third stores your passphrase in the keyfile using “luksAddKey”:

Now that you have a keyfile you can set up your linux install to automatically unlock and mount the USB drive by editing a couple of files.

Edit your “/etc/crypttab” file:

nano /etc/crypttab

Add the line below to add the “/dev/mapper/secretvol” device:

secretvol /dev/sdb /root/keyfile luks

NOTE: You can also use the UUID of your drive in “/etc/crypttab” to make sure that the right disk as detected by the kernel is used. In cases where you may be adding or removing disks this is really important as you may have “sdb” or “sdc” or “sdX” depending on what order the disks are detected by your linux install. To find the right UUID type:

ls -l /dev/disk/by-uuid

Which in my case told me that my UUID for “sdb” (my USB drive) was “6858274d-2370-4377-9426-d786c3e7a410″. The line in “/etc/crypttab” that you should use in this case to add “/dev/mapper/secretvol” is:

I’ve installed Oracle Database 11g Release 2 a few times on various Linux installs and apart from a few quirks it is a pretty similar process on most. The absolute bare bones default install, as described here, is easy to set up and doesn’t take that long. You can see more detail, including all the recommended steps if you follow the instructions in the Oracle install guide. I will describe installing 32bit Oracle Database 11g Release 2 on CentOS 6.3 32bit with the UI installed so we can use the Oracle installer directly. My computer’s name was “localhost.localdomain” as I was testing this in a development VirtualBox install.

First download Oracle 11g Release 2 from their website. For a linux install it comes as 2 zip files which you must first accept the license for before downloading. The exact version I downloaded was “Oracle Database 11g Release 2 (11.2.0.1.0) for Linux x86″.

Now you need to prepare your CentOS install by adding the required users and user groups for the install process. In my setup I am following oracle and running the following commands to add the “oinstall” and “dba” user groups:

groupadd oinstall
groupadd dba

Now add the “oracle” user, who we will be using to run the Oracle 11g install and give the user the correct group membership:

useradd -g oinstall -G dba oracle

Now create a directory and set the appropriate permissions where you are going to install Oracle. In my case I have installed it in the “oracle” user’s home directory under “/home/oracle/app”:

Now extract the Oracle ZIP files downloaded earlier into somewhere sensible. I chose “/home/oracle/database”. Navigate to the directory and run the install script as your new oracle user:

su oracle

cd /home/oracle/database
./runInstaller

NOTE: In my case, because this CentOS install was a VirtualBox virtual machine I needed to explicitly set the $DISPLAY variable to the local machine before the UI for the installer would run. This is done by running the following command and restarting my shell:

export DISPLAY=:0.0

Now the installer will start up. You can ignore entering your email in the first step “Configure Security Updates” and leave the default setting of “Create and configure a database” in the second step “Installation Option”.

For the “System Class” step of the install I just left it as the default “Desktop Class” and in the “Typical Installation” step I left everything as default apart from setting the Administrative password. The default settings puts the oracle base in “/home/oracle/app/oracle” with a global database name of “orcl.localdomain”. For the “Create Inventory” step I left the default folder of “/home/oracle/app/oraInventory” and the group name “oinstall”.

Now we get on to the interesting part of the install, which is the “Prerequisite Checks” stage. If you are running the install on a brand new copy of CentOS you will need to set a few system variables and install a set of prerequisites.

NOTE: You may not need to, but I needed to add more swap space to my CentOS install this time around in order to meet the prerequisites. Run the following commands as root to create a 2048mb swap file called “/swapfile” on your harddrive and set CentOS to use it for swap space:

Now set CentOS to always use this swap space at boot by editing your “/etc/fstab” file using the command:

nano /etc/fstab

And add the following line:

/swapfile swap swap defaults 0 0

So if you have passed the swap space test in the “Prerequisite Checks” in the Oracle install you can start to fix all those “Failed” messages. Click on the button “Fix & Check Again” and a window will pop up to tell you about the handy “runfixup.sh” script that will be placed in “/tmp/CVU_11.2.0.1.0_oracle/runfixup.sh”. So in your shell, navigate to the directory as root and run the script:

cd /tmp/CVU_11.2.0.1.0_oracle/
./runfixup.sh

The “runfixup.sh” script will fix all the system variables for you so you don’t need to set them manually. Now all that remains is to fix the dependencies, most of which can be installed using “yum” with the following command:

Now the only remaining prerequisite that causes a “Failed” message is “pdksh-5.2.14″ which has been removed from the CentOS repositories after CentOS 5 (see here). The replacement is “ksh” but if you install this package using “yum install ksh” you will get the same dependency check “Failed” in the Oracle install for “pdksh-5.2.14″ and “ksh” will conflict with “pdksh” if you then go to install it.

The solution is to install “pdksh” manually from RPM, which can be found at a variety of mirrors. I used the following command to install the “pdksh” package:

Now Oracle should pass all the prerequisite checks and you will see the “Summary” step of the install where you can click the “Finish” button. It may take a while but Oracle Database will install with all the required settings ready for you to use out of the box.

The final step is to execute the configuration scripts as root, which will pop up after you have unlocked any users you might need other than the defaults (you don’t need to though at this stage). The two scripts can be run as follows:

cd /home/oracle/app/oraInventory/
./orainstRoot.sh

cd /home/oracle/app/oracle/product/11.2.0/dbhome_1/
./root.sh

To test your install worked you can log in to the web based management interface for your computer “localhost.localdomain” with the user name “SYS” connecting as “SYSDBA” and using the password you set during the install of Oracle. Remember to open port 1158 on your firewall if you need to:

Apache Tomcat is actually easier than the standard Apache webserver to set up, which is great news if you are working with Java based web applications. All you need to do is download it and make sure it starts with whichever linux distribution you are using. Deploying applications in standard WAR format is really easy as well due to the simple web based management interface.

In my case I wanted Tomcat to start with Ubuntu and sit on the default port 8080 so I could have it running alongside my standard Apache webserver for PHP. We were developing a Spring application and used Maven to build and compile to a single deployable WAR file. You must have Java installed and set up for this to work. To check you have Java set up type:

java -version

This should tell you what version of java you have installed (hopefully Java 1.7). You also need to check that the “JAVA_HOME” variable is set by typing:

Now you will have a folder “apache-tomcat-7.0.32″ in your home directory. This needs placing somewhere sensible so copy it to “/usr/share/tomcat7″ using:

sudo mv apache-tomcat-7.0.32/ /usr/share/tomcat7

Now you can test your Tomcat install works with its default settings by starting it up. Note: before you do this you need to set the “JAVA_HOME” variable otherwise you will get errors (see my previous post).

To start up Tomcat navigate to “/usr/share/tomcat7″ and run “startup.sh”:

cd /usr/share/tomcat7

./startup.sh

With the default settings you should now be able to reach your Tomcat server home page by navigating to “http://your.ip.add.ress:8080″ where you should hopefully see the homepage and a nice message saying:

Now you will be able to log in to the manager GUI at “http://your.ip.add.ress:8080/manager/html” using the login details MANAGERUSER and password YOURPASSWORD. You can deploy applications and generally manage your Tomcat install from here.

The final thing to do is to set up Tomcat so that it starts every time your server starts. This is pretty easy as all you need to do in Ubuntu is edit the “/etc/init.d/tomcat7″ file:

Now to check everything is working on system startup reboot your machine using:

sudo reboot now

Navigate to “http://your.ip.add.ress:8080″ where the Tomcat home page should appear with no problems. Note: If you are having problems reaching your Tomcat home page make sure you have opened port 8080 on your server’s firewall.

It’s definitely worth reading some of the documentation on Tomcat, plenty of which is linked off your newly installed Tomcat home page. You should now have all you need to deploy your Java web applications as WAR files which is really easy using the manager GUI provided by Tomcat.

The default install of PHP on our CentOS 5.5 box was 5.1.6, which is very out of date (we are currently using PHP 5.3 elsewhere while we figure out how to get around some very serious problems with 5.4). Unfortunately, we needed to upgrade to PHP 5.2 and no further as 5.3 meant upgrading MySQL and potentially breaking compatibility with our web application.

It used to be that you could add the CentOS testing repositories and just update PHP but as PHP 5.2 is depreciated this option is no longer available. The solution is to use the Atomic repositories which can be added to your CentOS install by typing:

wget -q -O – http://www.atomicorp.com/installers/atomic | sh

This will add a new repository file “/etc/yum.repos.d/atomic.repo” which means we can use their packages as well as those from CentOS. Now we need to make sure that we don’t upgrade our PHP beyond 5.2 so we add a single line to “/etc/yum.conf” under the [main] section:

exclude=php-*5.3*

The exclusion means we will include packages from all repositories other than anything that matches “php-*5.3*” so PHP 5.3 won’t be installed as part of an upgrade.

Now just upgrade PHP and restart Apache:

yum update php

service httpd restart

You can check which PHP version you have using:

php -v

Now obviously you want to use a more recent version of PHP than 5.2 but in the rare case where you have to, the previous commands make things very easy.

We had a problem where a server wasn’t allowing us to upload any more files using our web application’s interface. This was due to an enormous “error.log.1.txt” in “/var/log/apache2/” caused by setting our log level to warnings rather than errors. Thanks to Josh at blindhog.net I could run a command and quickly find directories over 1GB in size:

Network address ranges are set slightly differently to standard wildcards. For example, to describe a range of IP addresses from 192.168.0.1 to 192.168.255.255 you use:

192.168.0.0/16

Where 16 describes the number of bits in the IP address that are used for comparison. In this case the 16 describes the first 2 bytes of the address: 192.168. You can read more about IP addressing at Rhyshaden’s Data Network Resource (and various other places).

To set your linux firewall up in webmin to use a range of IP addresses, just use the wildcard notation above. So in Webmin – Networking – Linux Firewall, when you are editing a rule in iptables you can put in 192.168.0.0/16 to describe a range of IPs (e.g. in the “source address or network” field to restrict access to a certain IP range). Manually setting these rules is more tricky but there are resources out there like Linux Home Networking and the Easy Firewall Generator to help. We just used Webmin as it makes this kind of work very easy indeed.

Categories

About Me

A manager and developer for research and business groups, including large scale organisations in the fields of science, engineering and medical research. Technically proficient in a wide range of technologies and languages, including online application and database development, as well as an interface designer based on established design patterns.

Gained a PhD in Electronic, Electrical and Computer Engineering from the University of Birmingham on the subject of "Multimodal Intent Recognition for Natural Human-Robotic Interaction".

Currently working as a Java Developer in Shoreditch, London. Interests include the application of modern development techniques to patient record systems, collaborative research methods, distributed systems for healthcare support, web based patient record and decision support systems, data sharing and multi-platform interoperability.

Subscribe to my Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.