Background

The largest disks used in PPC based TeraStations as supplied by Buffalo are 500GB giving a maximum of 2TB (4x500GB) on a system. It is now possible to get IDE disks up to 750Gb in size, and SATA disks up to 2.0TB in size. Larger disks may become available in the future. Many users would like to be able to use such disks to upgrade their TeraStations capacity.

The systems that are currently covered by this article include all the PPC based TeraStation Models:

Original TeraStation

TeraStation Home Server

TeraStation Pro v1

These all come with a Linux 2.4 Kernel. This version of Linux has a limitation where a single file system cannot exceed 2TB. As Buffalo have assumed that normal use is to use a RAID array treating all 4 disks as a single file system the largest disks that is supported by the standard Buffalo firmware is 500Gb. This article covers techniques for using disks larger than this (albeit with some restrictions).

The following ARM based TeraStation models are not covered:

Terastation Live

TeraStation Pro v2

These systems come with a Linux 2.6 kernel. This does not suffer from the limit of 2Tb in a single file system. Although there is no reason to suspect that the instructions described in this article would not work they should not be required as the standard Buffalo firmware should handle larger disks without issues.

SATA Disks on IDE Based Systems

It is possible to use disks larger than 750GB on an IDE based system via a SATA/IDE converter, although you have to make sure you get one small enough to fit into the available space. The ones tested were obtained via eBay, and consisted of a small PCB that was the same size as the back of a 3.5" drive with a SATA connector on one side and an IDE socket on the other as shown below:

The board plugged into the back of the drive and simply converted the connectors from SATA to IDE and there was just enough clearance inside the case to fit the power and IDE cables. For large disks a SATA drive plus a converter is normally cheaper than the equivalent size IDE drive, so this is attractive from a price point of view.

Approach

This wiki article discusses an approach that can be used if the use of RAID5 arrays is not critical to you. If RAID5 is crtical then an approach that supports this with large disks is discussed in the wiki article called TeraStation Larger Disks and RAID5.

The approach that has been used is to forgo the use of RAID arrays for the data areas, and instead use them as single independent disks. This means that the 2TB limit imposed by the 2.4 linux kernel applies at the individual drive level and not at the RAID array level. Using this approach means that the theoretical limit on a TeraStation is increased to 8TB (4 x 2TB).

Advantages

You can use drives larger than the 500GB maximum supported by the standard Buffalo firmware.

You can use mixed drive sizes, thus allowing for incremental upgrades in capacity of the system.

A single drive failure only loses any data associated with that drive.

Recovering the system back to a fully working state after a drive failure is simple.

The Buffalo provided firmware continues to be used as the basis of the system

The Buffalo browser based GUI can still be used to manage nearly all aspects of the system such as users and groups.

You can use linux soft-links to give you a single "view" of all 4 drives. This is means that the sytem will look a bit like it is running with a single drive, although in practise all 4 drives will have their free space managed independently.

The Buffalo supplied media servers continue to function and can deliver media from from all 4 drives.

The software upgrades available from the itimpi website can still be used with the system.

Buffalo firmware upgrades can still be used (although at this late date it is unlikely that they will provide new ones for these models as they have been superseded by the ARM based models).

Disadvantages

RAID arrays are not be used for the Data part of the hard disks (although one is still used for the System area). This means that you are not protected against a single drive failure. You therefore need to decide on a suitable back-up strategy to take account of this. If this matters to you then the approach described in this article cannot be used. This approach is ideal, however, in a case where backup of your is not critical, as might be the case if you are using the system purely as a media server.

You have to use a telnet enabled firmware release rather than the standard Buffalo supplied ones. Many might consider this to be an advantage rather than a disadvantage!

Manual steps are required to recover from a disk failure - you cannot use the Buffalo GUI to achieve this. However as these are the same steps as are required to set the system up in the first place this will probably not be an issue.

Telnet Enabled Firmware

TeraStations internally run the Linux operating system. Buffalo hide this from the average user providing the system "packaged" and with a browser based GUI to control and configure iy. Telnet enabled firmware allows users to log in using a telnet client to control and manipulate the system at the Linux command line level.

This allows users to do things like:

Configure the system at a more detailed level than allowed for by the Buffalo GUI.

Install new applications to extend the functionality of the Terastation.

In the event of any problems being encountered allow for a level of access that gives more chance of recovering user data without loss.

The changes described in this article require the use of a telnet enabled release. Hopefully the instructions provided are detailed enough that users can carry out the steps provided without needing much linux knowledge.

The standard software supplied with buffalo PPC based TeraStations does not provide for telnet access. Telnet enabled releases of firmware corresponding to virtually all Buffalo firmware releases can be found at itimpi's website. These are identical in functionality to the corresponding Buffalo firmware releases - the modification to add telent functionality being trivial. This means they will have exactly the same bugs (if any) as are found in the Buffalo releases.

The itimpi firmware releases are the ones that have been used while preparing this article. Firmware from other sources should work fine as long as it is telnet enabled.

To use telnet you need a telnet client. A rudimentary one is included with Windows which you can invoked by typing a command of the following form into the Windows Run box:

telnet TeraStation_address

A freeware one called PuTTY is recommended as a much better alternative. In addition to the standard telnet protocol PuTTY also supports the more secure SSH variant (although additional components need installing at the TeraStation end to support this).

TIP: If you already have a telnet enabled version of the firmware installed and you want to continue to use that version then you can run the Firmware updater in Debug mode and elect to not rewrite the linux kernel or boot image in flash, but merely update the hard disk. This is slightly safer as the flash chips have been known to fail.

Setting up for Larger Disks

Preparing the Disks

Most users will tend to run with their system either in RAID5 mode or SPANNING (jbod) mode. To use larger disks in the manner described in this article the first thing is to get your system in the state where there are no RAID arrays defined in the Buffalo Web GUI, and all 4 drives are being used independently. This will destroy any existing data so ensure that you have your data backed up first.
If you already have your system set up to use the four drives independently then this section can be skipped.

Note that you could do this AFTER you have carried out the step mentioned below on Perparing the Data Partitions - the order of these two steps is not critical.

Starting with a Running System

Most people will already have a running system, and in this case it is easier to get your system running in a mode where it can accept larger disks, but initially using the drives already installed. If this is the case and your TeraStation is already in a running state then:

Ensure that you have backed up your data, if not you will lose it.

Go into the Web GUI and select Disk Management. If this shows that you are already running with separate disks (Disk1 to Disk4) then you are already in a state where individual drives can be changed so there is nothing else to do.

If (as is normally the case) this shows that you are running using a RAID array then click on RAID Array 1 which will take you to a screen which has a button labelled Delete RAID Array. A Dialog block is displayed asking you if are Sure You Want to continue and warning you that doing so will delete all your data. Select OK. You will now be taken to a screen where you need to enter a displayed number to confirm the delete (on many firmware releases this screen will be in Japanese). All you have to do is enter this number and then press the left-hand button to Apply the change. While the Array is being deleted (which takes several minutes) all other access to the TeraStation will be prohibited. On completion you will be returned to the Disk Management screen, and this time RAID Array 1 will show as "Not Configured".

Most people run their teraStations with only RAID Array 1 configured. If you are running with RAID Array2 configured as well then repeat the above step for RAID Array 2.

Now select the Home option, and you will be returned to the home page of the Web GUI. You will see that all 4 drives are now listed with their space used individually itemised.

If you want to change any of the disks at this point then follow the instructions for Upgrading/Changing a Single Drive.

Starting with new Disks

It is perfectly feasible to start with a brand new set of disks, rather than with a running system. You are more likely to use this approach if you are considering upgrading all the disks at the same time. The alternative is to convert the system as described above, and then Upgrade a Single disk at a time as described later.

Initializing the Disks

If you are starting from scratch with a new set of hard disks then the first thing to do is to get the Buffalo software onto the system:

Install the 4 drives you want to use. If they are IDE drives make sure the jumpers on them are set to make them the Master/Single drive. If you are going to use a mix of drive sizes then it is recommended that they are installed in ascending size (i.e. Drive 1 is the smallest). This may not be critical but that is what has been tested.

Flash the TeraStation with the firmware release of your choice. This must be a telnet enabled variant of the firmware.

You will be given warnings about the partition structure being wrong and the disks about to be formatted. OK these to continue. You will see that the Disk I/O lights on the TeraStation all start flashing. You will almost certainly get a message from the firmware nupdater program about a timeout waiting for a response. Quit the firmware updater at this point. The disk I/O lights will continue flashing on the TeraStation. Wait until this finishes (which could take several hours).

Flash the TeraStation again - this time it should complete without error.

Go into the Web GUI and delete any shares that have been set up.

You will almost certainly find that array1 hs been setup to be a "SPANNING" array. Select this array and then the option to delete the array. You will now be given a screen in Japanese where you have to enter a displayed number for confirmation. Enter this number and then press the left-hand button. This will delete the array, and you will see your disks displayed as independent drives. Do not worry at this point if the capacities displayed are wrong.

Changing the Partition Sizes

The next thing is to get the partitions set up correctly to use all the available disk space. To do this:

Telnet into the system. Any telnet client will do, but the recommended one if you are using Windows is PuTTY (which is freeware). Login with a user with root privileges. If using the telent software from the itimpi web site this will be the "myroot" user - initially with a blank password (it is recommended that you immediately set a password using the 'passwd' command.

Try the 'df' command and you will see that it shows you all 4 disks but not using their full capacity.

Unmount the 4 file systems containing the data partitions using command of the form

umount /mnt/mountname

where mountname is the appropriate name from the output of the df command.

Repartition each of the four disks with the disk partitioning software using a command of the form

mfdisk -c /dev/xxx

where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model

You can get a menu of available commands in the mfdisk program by pressing 'm'. You may want at this point to select the 'p' option to show the current partitions that have been set up by the Buffalo firmware at the initialization stage.

Use the 'd' command to delete partions 4 and 3.

use the 'n' command to create new primary partitions 3 and 4. For partition 3 use the default for the start position, but for the end position type in a value one less than the value displayed as the default. For partition 4 use the defaults for start and end track. It is possible partition 4 is not needed, but the Buffalo firmware sets it up, and the above approach of giving it one track minimizes the wasted space

Use the 't' command on partitions 1, 3 and 4 to change the type to 'fd' and partition 2 to '82'. It is not clear if this is really required, but those are the partition types the Buffalo firmware sets so it is better to be safe than sorry.

As an example after making these changes on a 1.5TB Seagate drive and then using the 'p' command to print the current settings gives the following:

where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model

Repeat this for each of the four drives varying the last part of the device name as indicated earlier for each drive

Reboot the system

Go into the web GUI and you should now see all 4 drives listed with their correct capacity.

Setting up the Shares

It has been found that sometimes there is configuration information about shares left behind despite the fact that the drives have been changed. You therefore need to make sure that your share configuration is correct for the new mode of running.

There are some idiosyncaracies of the Buffalo software that you need to be aware of when setting up your shares:

The system does not seem to like shares of the same name set up on more than one drive.

Any folders that are at the root of the mounted file system will automatically be added as shares when the system next reboots, and will be given a description of TeraStation Folder.

On that basis following approach is recommended

Use the Web GUI to look at the shares and delete any existing shares that are no longer needed. Normally the steps above will have deleted any existing shares, so initially there are no shares listed.

Set up a share on each of the 4 disks. The suggestion is to simply use share names of the form share? where ? represents that drive number. If you do not need it, unselect the option for the Recycle Bin as this can take up a lot of space as files are deleted.

You can optionally also set up further shares at this point. The "Single Drive View" approach mentioned in the next step does not require this as everything is accessed via the shares on Disk 1.

Setting up a Single Drive view

At the client end it is more convenient to treat the TeraStation as a single drive rather that 4 separate drives. This can be achieved using the approach given below. It is recommended that you put the commands to create these links into a separate file (e.g. CreateLinks.sh) which is set to be executable so that you can re-set them up again later using a command such as

sh ./CreateLinks.sh

The best place to keep this file is probably in the home folder for the 'root' user (/root) as this is where you are initially located when you log in via telnet. As this is part of the System partition which is replicated on all 4 drives, it will remain available if you change one of the drives.

You will be able to also see this file at the base of the share. It can then be edited from Windows which most people will find easier than using the Linux level editors such as vi. Make sure you use an editor that can handle Unix style end-of-lines (e.g. TextPad). The standard Windows editors such as Notepad and Wordpad will not do.

You are likely to want to tune the commands mentioned here to suit your specific requirements.

A sample shell script incorporating these ideas is given that may be suitable with minimal modification for most users.

Drive Mapping

Telnet into the TeraStation using a user with root priveleges (e.g. myroot)

Issue the following command:

chmod 777 /mnt/disk?

This ensures all users will have full read/write access to each of the data partitions.

Create a folder on the disk1 share into which you intend to put disk1 content

Open up the share from your client (e.g. your PC) and you will see the _disk1, _disk2, _disk3, _disk4 folders which can be clicked on to take you into the drive specific content in a transparent manner.

Folder Mapping

You may want to use a further level of symbolic mapping at folders within the share. As an example say you have created a folder called TV on disk 1, and you want to have folders that at the share level (drive 1) look like this:

TV
Series1
Series2
Series3

but you want some of the series of to really be on drives other than drive 1. You could create a TV folder via the _disk2, etc links set up earlier and then by setting up the following symbolic links you would get the view that you wanted:

Note that if you want to use spaces or any other special character in any of the above commands they need to be escaped (e.g. entered as '\ '. Typing in all these 'ln'commandsd can be a bit laborious and error prone, so you may prefer to isntead get the machine to do a lot of the work as shown in the sample script below

TIP:
If your symbolic links do not show up in Samba then you have probably got one of the following wrong:

The symbolic link points to a none-existent folder.

The permissions on the file/folder do not allow Samba users to access the file

Copying/Moving Files

You may at times want to copy or move files between the different drives to balance out usage. If you do this, it is MUCH faster to do it via linux cp/mv commands rather than to try to do it via Windows shares.

The only downside to is that the files will end up owned by root. However the Sample script contains some chmod/chown/chgrp commands that are currently commented out that can be used for resetting the owner and access permissions. You may want to uncomment these by removing the leading '#'.

Sample Script

The following is a reasonably comprehensive sample script that is uses the ideas mentioned above. It is quite likely to be suitable almost unmodifed for many users. You should examine it to check that it meets your requirements.

It makes the following assumptions:

You have used the suggested name of CreateLinks.sh for the script file.

You have used the default share name of 'share?' on each drive where ? is equvaelnt to the drive number

You want all disk specific media files to appear under the _diskn style folders

You want all symbolic links to appear in the TV folder of drive 1, and this is the only content in this folder. This is the folder that is configured for the media server to use.

The 'TV' folder on Drive 1 and the relevant '_diskn' folders to hold disk specific content are created if they do not already exist.

Permissions are set on all files on the share that will allow anyone to access them.

This script is simply re-run any time you add new folders to the _disk? folders. Only top level folders need this - sub-folders do not.

This script could also be added as a start-up script if so desired so that it is automatically run every time the system is restarted.

Changing/Upgrading a Single Drive

It is quite likely that at some point you will need to change a drive. This could be because a drive has failed or because you are intending to replace one of the existing drives with a larger drive.

The steps are as follows:

If you are changing a drive for one of a different capacity then make sure that you have first backed up any data first. This could be to another drive on the same TeraStation as only the drive you are about to change is affected. Alternatively as the data will still be on the drive after you have removed it make sure that you have a way of attaching it to the TeraStation (e.g. a USB external box) after the new drive has been fitted so that you can copy the data onto the new drive.

Prepare the drive as for a new system making sure any jumpers are set correctly and there are no partitions already present on the drive.

Put the new drive into the TeraStation.

Start up the TeraStation. It should come up normally, but the TeraStation will be showing a flashing red light for the new drive at this point.

Telnet into the TeraStation using a user with root privileges.

Use the command

df

You will see that the disk you have just added is not shown as a mounted filesystem.

Use the mfdisk utility as described earlier to set up the partitions. This time you will also have to set up partions 1 and 2 as well as 3 and 4. Partitions 1 and 2 must be at least as large as these partitions on the other disk (you can safely use the mfdisk -c /dev/xxx style command to look at these as long as you do not write out any changes).

Format the data partition using a command of the form:

mkfs.xfs -f /dev/xxx3

where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model

Prepare Partition 2 on the new disk to be used as swap space using the command

mkswap /dev/xxx2

You now need to make sure that partion 1 of this disk is added back to the RAID0 array covering all 4 disks that is used to hold system files. You can look at the details of this array using the command

mdadm --detail /dev/md0

If the disk is not reported as faulty, removed then you may need to issue the following commands:

mdadm /dev/md0 -f /dev/xxx1
mdadm /dev/md0 -r /dev/xxx1

where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model.

Make sure that the system partition is part of the system partition RAID0 array (/dev/md0) by using a command of he form:

mdadm /dev/md0 -a /dev/xxx1

This command marks adds the new drive to the array and starts rebuilding its contents.. If you now look at the array status you will see it has 4 drives again, and that the new drive is marked as being "recovered". Wait for this to complete

Mount the new data partition on the drive, the flashing red light on the drive will go out and it should be ready to use. Alternatively if you reboot the system, the new drive will be automatically mounted as part of the booting process.

Copy your backed up data onto the new drive.

Flashing with new Firmware versions

The tests done show that it should be possible to flash a TeraStation setup as described in this article without any issues. However caution should be taken as this is a none-standard setup and this cannot be guaranteed to be true in all cases.

If you attempt a flash upgrade on a system that has been setup as described in this article and the firmware updater program gives any warnings about invalid partition structure or wants to format any of the disks you should abandon the firmware update as otherwise you will almost certainly lose data.