Pages

24 February 2013

Dell XPS 15 L502X Optimus Graphics under Ubuntu 12.04

Firstly I apologise for the quality of this article, its somewhere between the quality of the notes that I usually write for myself to keep track of instructions like this, and the quality that I publish so others can follow the logic. I'm a bit out of time at the moment, but thought these instructions could be of use to more people than just myself so I thought that I'd put in some effort to get them out...but only just so much effort.

Its been a while since I've played around trying to get the NVIDIA card in this laptop to play nicely under Linux and I thought I'd give it another spin. This laptop has Optimus which when it was released meant there were quite limited options in terms of getting NVIDIA GPU working. I played around with it a little bit back then, but the bumblebee project was really in its infancy, and getting it all installed and working was quite painful. Whilst I found things are still more complicated than they need to be obviously the bumblebee team have done a huge amount of work to get things working this well as they do now. Unfortunately the install process from Ubuntu on this laptop is still a little more difficult than it needs to be, so I've taken the time to list some notes on how I managed to get everything working.

Overview

To the best of my understanding the following diagram explains how the graphics hardware in this laptop is connected (apologies for the quality of the diagram, but hopefully its enough to convey the message).

This means that if you have just the Intel GPU working you can get access to both the laptop display and the display port. If you have just the NVIDIA GPU enabled you can get the HDMI output to work. When Optimus is enabled content is rendered on the GPU and copied back to the Intel GPU for display on the laptop panel or display port. It seems that the bumblebee team are keen to get HDMI working whilst using bumblebee, but they arn't there yet.

This currently leaves you with two options:

Intel - Bumblebee - Laptop display + Display Port: You can get the laptops internal display working and the display port working. You can even enable hardware rendering on the NVIDIA GPU using bumblebee to support Optimus for display on the either of these displays.

NVIDIA - HDMI: You can get the HDMI port working with all content rendered on the NVIDIA GPU.

Unfortunately presently you can't have both options working at the same time.

All of these instructions presume a fresh install of Ubuntu 12.04 (in my case 12.04.02), presumeably if you've done any other config there could be reminants of an old install that leave these instructions not working.

Intel - Bumblebee - Laptop display + Display Port

Installing Bumblebee to primarily use the Intel GPU and load the NVIDIA GPU selectively for GPU intensive applications.

The advantages of this method are that you can use both the internal laptop display and the display port simultaneously as well as the more powerful NVIDIA GPU when you need to and have bumblebee manage the power of the NVIDIA GPU so its not wasting as much power when it's idle.

Initially I essentially followed the bumblebee guide for intalling it under Linux which is fairly straight forward:
http://bumblebee-project.org/install.html#Ubuntu

This means that after reboot you're left in a state worse than before and you just be left with a black screen with no graphics driver loaded what so ever. Infact it leaves the system so screwed up that on boot you can't even ctrl+alt+f1 to switch to a tty.

...So where do you go from here...

Well if you're sensible you will reinstall the Intel drivers before you reboot and you'll happily be on your way.
$ sudo apt-get install xserver-xorg-video-intel

If you make the mistake that I did and reboot before you install the Intel driver you will be left with a black screen and you wont be able to log in what so ever. I found that while the system is still booting you can switch to one of the virtual consoles by pressing crtl+alt+f1, you must do this while the system is booting because once the system has reached the login state you can't swap back to the virtual console. Once you've got to the the virtual console login and just follow the instruction above. After this point you should be able to get x to start correctly:$ sudo service lightdm start(sometimes you need to reboot at this point to get x to start, I've even seen a couple of times where two restarts are required before everything gets going correctly)After you login you should be able to run bumblebee using:$ optirum glxspheres

Configuration files:

/etc/X11/xorg.conf/etc/bumblebee/*

NVIDIA - HDMI

If you don't care about the laptop display / display port / Optimus / Bumblebee / Power managment and just want to get the NVIDIA card running displaying on HDMI using the most up to date driver from NVIDIA.

Download the latest driver from NVIDIA currently 310.32:Install the kernel sources (the driver warns you about this if you forget and get part way through and need them): $ sudo apt-get install linux-headers-generic

The NVIDIA driver sets up everything you need, with one exception. Because there are two GPU's in the machine you need to tell the x configuration which card you want the driver to be applied to. If you reboot now without updating the xorg configuration the driver wont be loaded correctly. In this case you need to update the xorg.conf file to tell it which device to load the nvidia driver for.

03 August 2011

This tutorial/project uses OpenCV, OpenGL (+FreeGLUT) to build an augmented reality application from the ground up.

Lately I've done a bit of experimentation with computer vision, specifically I've been working with Augmented Reality from the ground up. I've made a tutorial program that might help others to get a few of the basic concepts down pat. This sort of work will no doubt seem like jibberish to those who haven't studied computer vision so I suggest you read up on that if you aren't familiar, but hopefully even those who don’t understand computer vision will be able to build the code and start experimenting even if they don't exactly know what’s going on under the hood. If your interested in working with augmented reality from a higher level perspective I suggest that you take a look at one of the higher augmented reality libraries like ARToolkit.

I'm currently only distributing the project as source only. You should be able to compile under both Windows and Linux and I've tested it with both Visual Studio Express 2010 and GCC and found that it builds and seems to run fine under both.

It is a learning code and has no real application. Please dont be to critical of the programing, it is intended as a computer vision example not a programming example.

Note: The demo requires OpenCV 2.2 or later (I've tested with both OpenCV 2.2 and 2.3 to date).

Linux
Install the following packages (instructions are for Fedora, I assume other distros will be similar, but slightly different).
opencv, opencv-devel, freeglut, freeglut-devel, cmake

Extract the package and create a new directory in the base folder (the folder that contains Source and Resources), move into the directory:# cd AugmentedReality# mkdir Build# cd Build

To build you need to run cmake to generate the makefile scripts and then make the program, afterward just run the program. # cmake ../Source# make# ./AugmentedReality

Windows
Download and install:

OpenCV 2.2 or later (I reccomend OpenCV 2.3 currently). See my notes elsewhereon this blog if you run into problems with OpenCV 2.2.

FreeGLUT

CMake (Optional)

You can now either use CMake to build the project solution files (following a method similar to the above Linux configuration) if you are familiar with CMake or create a Visual Studio Solution from scratch by following these instructions.

I will assume that you have installed OpenCV to C:\OpenCV2.3\, if not just substitute in the path you installed OpenCV to.

To get started you will need to print out the full chessboard image. I suggest you print out two copies on say an A4 sheet of paper. Attach one of them to something stiff like a piece of cardboard (you will get better calibration results with this). You want a nice reasonable sized white boarder around the chessboard image so that the detection algorithm will work correctly. You can either use the markers provided by printing out the photos in "Resources\Photos" at a photo shop, or create your own (explained later in the tutorial) using magazines or photos you already have.

The program has a number of modes that step you through the process. The mode can be changed by using the number keys.

0) No augmentation, just display the live image.

1) Find and display the corners of the chessboard.

2) Augment a planar image onto the chess board (planar).

3) Augment a planar image onto the chess board surface.

4) Show the calibration images in sequence.

5) Draw a 3D figgure on the chessboard.

6) Allow the user to select a marker image.

7) Show identified features in the marker and live image.

8) Show the mapped feature matches.

9) Show an augmentation on a natural image.

q or 'esc') Quits the program.

c) Reloads the calibration images from file.

h) Holds the next image in an internal image buffer.

a) Adds the currently held image to the list of calibration images.

s) Saves the list of calibration images.

b) Saves the held image as a marker image.

v) Toggles the view so that you can see the currently held image.

i) Toggles on/off subpixel accuracy.

m) Cycle through the list of avaliable natural markers.

d) Changes the displayed 3D model.

-, +) Changes the natural marker scale be a factor of 2.

The program will start in mode 0. This mode allows you to check that your webcam is connected, working and is selected by the software.

Firstly we can make sure that the webcam can detect the chessboard by pressing '1'. The chessboard will be drawn with markers showing the corners of the squares. If you hold the chessboard and camera completely still you will probably be able to notice that the corners will jiggle around a little. You can improve performance by turning on subpixel accuracy by pressing 'i'.

We can start with a simple augmentation by pressing '2' and then showing your webcam the full chessboard. The augmentation in this mode will only work if the entire chessboard is within the frame. If you find that you get no augmentation or the program crashes check that you have put the image files in the directory with the executable program. In this mode the augmentation is quite simply performed by finding the chessbord corners like in mode '1' and then painting the overlay image into the picture between the outside corners of the chessboard.

We can try a slightly more complex augmentation by going to mode '3' and then using our deformable chess board, carefully bend the chessboard and you should find that in this mode you can get the augmentation to bend with the chessboard to a limited extent. If you bend the chessboard to much you will find that the corners are no longer able to be detected and the augmentation will drop out. This augmentation operates in much the same way as the previous, however rather than only relying on the outside four corners of the chessboard all of the detected corners are used.

To progress further doing more complex augmentations we need to calibrate the camera, this involves capturing a number of images of the chessboard at different locations and orientations. By calibrating the camera we can find the intrinsic parameters of the camera, such as its focal length. For this step it is important to use a planar chessboard, this means you need the chessboard to be rigid as any deformation will cause you to capture poor calibration images, as there is in implicit assumption here that all the corners of the chessboard lie on a plane. From your previous experimentation you will now have an idea when the chessboard can be detected and when it can't. Try capturing an image by pressing 'h' to grab the next image from the camera and then 'v' to view the image that you just captured. What we want to do is capture a number of images of the chessboard at different orientations and scales so that we can calibrate the camera. So grab a few images using the above steps and add them to the calibration list by pressing 'a' to add the held image. If you capture an image that you don’t think will be any good simply don't add it to the list, once an image is in the list there is no direct way of removing it short of restarting the program, (or reloading all calibration image from file by pressing 'c', before you have saved the current list). To save the list for future use press 's', these images will now be automatically loaded when you restart the program so you don't need to perform this every time. If you are unsure whether an image will be good for calibration because you are unsure whether the chessboard will be detected you can view the image as previous by pressing 'v' and then changing the mode to '1', if the corners are found then you know the image will be ok. You will need to recalibrate the camera if you change some of the parameters of the camera like the zoom. You can view all images in the calibration list by going into mode '4', which simply cycles through the calibration images 1 at a time.

Now with your chessboard and going into mode '5' you will find that you should be able to get an augmentation with the teapot appearing on the chessboard. You can change the augmentation model by pressing 'd'. Have fun :)

Augmenting with the chess board is all well and good, but it is somewhat limiting. A better method is to use natural markers. To capture a marker enter mode '6', you can then click and drag to select an axis aligned rectangle marker image. I suggest when doing this first position the marker and then hold the image by pressing 'h' and then viewing the image by pressing 'v', from here the selection process is much easier because the marker no longer moves. When you have a marker image press 'b' to save it, using this method the marker is recognised as whatever occupies the full screen and should be a planar image and not deformable at all.

We can now examine the augmentation process by changing to mode '7', this shows where the features that we will use to match the marker image are located. You ideally want a good number of points to be found within your marker image. Cycle through the list of available markers by pressing 'm' a couple of times till you find your marker.

Mode '8' shows the matching correspondence between your captured marker and the live image. The system is performing well when the solid rectangle appears directly around the marker image. When this is happening the system is using RANSAC to find the best homography from your marker to the live image.

Finally you can augment with your natural marker by going to mode '9', the teapot should appear over the marker and stay still as you move the marker around. If you get poor results try different markers and try environments with more light (I have found that cheap webcams tend to blur the image in low light and this significantly reduces the accuracy of the system). You can change the scale of the model by pressing '+', '-' and you can change the model by pressing 'd'. Again pressing 'm' will cycle through the list of available markers. Have fun :).

The following video steps quickly through the entire process above to give you an idea of the results you should be getting.

Videos Demonstrating Augmented Reality

Here are a bunch of Augmented Reality videos that I have come across that are interesting and will hopefully get you interested in Augmented Reality. They also show you some of the potential of what is achievable.

28 June 2011

I recently had to install Fedora on a Dell XPS 15. Unfortunately while installation wasn't a problem getting it to boot for the first time was. I'd previously tried Fedora 14 and found much the same result and was hoping that the issue would just be gone in Fedora 15.

During boot I kept having problems with synchronisation with a number of different error messages like "scheduling while atomic", and "fatal exception in interrupt". This always inevitably lead to the same result Kernel Panic, although a couple of times it did tell me that it was trying by "fixing recursive fault but reboot is needed" although this never seemed to make a difference.

I had an idea that it was one of the kernel modules causing the problem, but I had no idea which, and so by trial and error I eventually determined that it was iwlagn. This fix is then reasonably simple just blacklist the module like so:

#vi /etc/blacklist.conf
+ blacklist iwlagn

Once black listed I found that Fedora booted successfully and I was able to log in.

There is one slight issue that I neglected to mention in the above is how do you manage to edit blacklist.conf, when you can't boot the system. I had a live Ubuntu CD around which I found worked and it was simply a matter of mounting the partition and editing the file.

24 June 2011

Why would you want to do this? I wanted support for Intel TBB which adds threading support to OpenCV and is not included in the standard build. Also I had problems with my web cam with the standard build and so also had to patch the code.

Download and install Intel TBB, in my case I installed it to: "C:/Development/Intel/TBB/Include/"

Download OpenCV 2.2 source code.

Apply the source code changes listed after this section. (Note these are not optimal, and some could be better specified with command link options, but I had trouble specifying a few command line options and this code was the first working version I came up with).

Open a command prompt with the Visual Studio paths and variables defined. It will be called something like "Visual Studio Command Prompt (2010)"

On Windows the source code can be configured using the following command line: # cmake -D CMAKE_BUILD_TYPE=RELEASE ..\
or# cmake -D CMAKE_BUILD_TYPE=DEBUG ..\Note: The way you need to specify command line arguments varys between versions of windows, see here for more details: http://opencv.willowgarage.com/wiki/InstallGuide#Build

Then build:# nmake

Link to the libraries from within your code, linking the *d.lib versions with the debug code and *.lib versions with release code.

Copy the *.dlls for both OpenCV and IntelTBB into the applications output directory.

Note I also found the following option worked for those who don’t link using nmake, but I haven’t tested it beyond configuring the build process:# cmake -D CMAKE_BUILD_TYPE=RELEASE -G"Visual Studio 10" ../

20 November 2010

I currently have Fedora 14 running on my ancient laptop primary PC and whilst the nouveau video driver is probably good I do find one major problem with it on this laptop. Unfortunately with the nouveau driver the fan on the graphics card runs constantly which is just noisy and annoying. I have used i8k tools (in conjunction with gkrellm) in the past to control this but I've had my share of problems with it causing the fan to cut on and off sporadically and so I just find the best solution is to use the proprietary NVidia driver. Unfortunately getting them on Fedora isn't as easy as it should be, but it isn't to hard if you know what your doing.

I use vi to edit config files but if your not familiar with vi just substitute it with nano and you should be on your way.

Installing The Driver
Download the latest drivers from NVidia. The current version for my card is NVIDIA-Linux-x86-173.14.28-pkg1.run (but this should work fine for any version of the drivers you need). Make sure you know the full path where you are saving them, for this example I will use:

/home/username/Downloads/NVIDIA-Linux-x86-173.14.28-pkg1.run

Log in as root because you'll need it for the rest of these instruction:

# su

Blacklist the nouveau driver by adding the following lines to the end on the blacklist config file (I'm not sure this step is necessary, if someone tries without it and it works let me know...I've just been doing it ever since I worked out how to install the drivers a few Fedoras ago and I'm not sure it's still relevant):

I have found that in Fedora 14 changing to runlevel 3 can create problems. We should just be able to:

# init 3

and have the system kill the x server, but I have found that it can lock-up and if it does make it the xserver seems to still be running at some level and the NVidia driver complains during install. So the easiest way to get around this is to start the system in runlevel 3 (temporarily) and then swap back once we have the driver installed. To do this change the default startup runlevel in inittab from 5 to 3:

# vi /etc/inittab

- id:5:initdefault:

+ id:3:initdefault:

Reboot the PC:

# reboot

When you reboot you should have a console login. Login and then su as root again.

19 September 2010

It seems somewhat silly that the WD My Book World 2ND Edition (MBWE II)supports USB drives that have NTFS and FAT filesystems but doesn't support the Linux EXT3 considering it runs Linux.

These instructions will let you mount and use USB drives with EXT3 partitions and I expect drives with other supported filesystems.

Firstly be careful! As with every tutorial I post only do this if you know what you are doing I take no responsibility you do it all at your own risk.

I actually expect that this will work for other filesystems that the NAS knows about but I don't have any drives with these formats to test it with, if you do try please post your results for others.

Manual Mount
Firstly you will need to SSH into the drive (you may need to turn it on using the admin web interface). There are lots of other sites that will tell you how to do this if you aren't familiar with this process.

If you aren't familiar using SSH you don't want to type the # it just indicates it is a command to enter, and everything is case sensitive. If you needed this help then you really need to be careful.

If you just want to mount the EXT3 drive as a one off it's easiest to do it manually, I've been doing this for some time now.

To mount the drive we need to find the device file associated with the drive, we can do this by running fdisk.

# fdisk -l

...

Disk /dev/sdb: 1024 MB, 1024966656 bytes

32 heads, 62 sectors/track, 1009 cylinders

Units = cylinders of 1984 * 512 = 1015808 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 1 159 157697 b Win95 FAT32

/dev/sdb2 * 160 1009 843200 83 Linux

The first set of entries (/dev/sda*) which I have omitted here are the WD's internal drive (if you have a dual drive version I'm assuming you will have a second drive listed here as well).

You should have a set of entries for your EXT3 drive. The above example is for a 1GB drive with two partitions, the first is a ~160MB fat drive and the second is a ~840MB EXT3 partition. So what I'm interested in is /dev/sdb2.

Next we create a folder to mount to, this can be anything but I've found the following nice because you can log into the copy manager on the web interface of the drive and use it to copy files between the drives (if you don't care about the copy manager /DataVolume/usb is probably a better place to make it).

# mkdir /shares/usb1-1share1

Then we mount the drive (replacing /dev/sdb2 with whatever device file fdisk showed your EXT3 partition to be listed as):

# mount -n -t ext3 /dev/sdb2 /shares/usb1-1share1

for other filesystems try:

# mount -n /dev/sdb2 /shares/usb1-1share1

You can then copy files to the mounted drive either using the web interface copy manager or from the console using either of the following commands:

# cp /DataVolume/sharename/file /shares/usb1-1share1/

# cp -r /DataVolume/sharename/folder /shares/usb1-1share1/

Note however that this method does not share the mounted filesystem though the standard mechanisms like CIFS. While it is possible to do this with this method it is just easier to automount the drive so if you want this read on.

I am currently not a member of all of these communities, and haven't bothered to list this on all these old threads, but if you have come here from one of these communities or another post about this it might be helpful if you would link this post.

This code patch is for Firmware Version 01.02.04 (currently the most recent firmware version) and should work with (Firmware Version 01.01.16).

First things first lets backup the file incase we make a mistake:# cp /sbin/usb_auto_share_add.php /sbin/usb_auto_share_add.php.bkp

Should you need it you can restore the file using:# cp /sbin/usb_auto_share_add.php.bkp /sbin/usb_auto_share_add.php

The only text editor on the NAS is currently vi, I like to recommend nano or pico to inexperienced users but unfortunately they aren't installed on this system. So I have developed an alternative for those who aren't familiar with vi.

Copy the file to a share you have, open the file over the network, edit it and then copy the file back. Here's how to do that, if your happy with vi just skip on a bit.

Find the share you want to copy to:# ls /DataVolume/

Copy the file to the share using (substituting sharename with the case sesitive name you found using ls):

# cp /sbin/usb_auto_share_add.php /DataVolume/sharename/

Now open the file over the network using the method you normally would, make the edits outlined below and then save the file.

Once you've made the changes copy the file back to the folder using

# cp /DataVolume/sharename/usb_auto_share_add.php /sbin/

Here are the changes to make to the file. If your going to post this elsewhere a little credit that you found it here first might be nice :)

Open the file in vi:

# vi /sbin/usb_auto_share_add.php

Sorry for the layout of the following, I suggest that you copy it into something nicer before you read it (like notepad with Word Wrap turned off).

Now you should be able to automount EXT3 drives and get them to show up as a shares like other filesystems on USB drives (according to your USB permsssions, see below):

It would have been nicer to get the filesystem type from mount but I haven't got around to that yet.
Now I didn't spend to long on this patch, and I havn't tested it with other file systems, but I believe it will also work with others supported by the NAS. I believe mount will try the filesystems listed in order in the following file:

# cat /etc/filesystems

You may be able to add more file systems to the list and even build and compile modules for file systems that aren't supported yet but that is well beond the scope of this article. I might add another post in future if I have the need, ext4 anyone???

I have identified a number of potential issues using EXT3, some which I have experienced and some that I hypothesised will create problems, if you find any more let me know and I'll list them here.

To access the share using normal methods you need to set the USB share permssions apropriately using the admin web interface (Advanced Mode->Users->USB Share Permissions).

Because EXT3 is a native Linux file system supporting permssions, the permissions of the files may cause problems. This will particularly be a problem if you have created the files (or permssions) on another Linux system and then connected the drive to the NAS or are doing the reverse. You should be able to fix this using commands like:# chmod 775 /shares/usb1-1share1/# chown root /shares/usb1-1share1/# chgrp jewab /shares/usb1-1share1/

I will try to get in contact with the developers and see if they will include this patch into the next firmware update. If I get any worthwhile feedback I'll let you know.

Update: I've tried to contact the developers (listed in the file) and the email address appears to be dead. If you know of a firmware developer that works on the MBWE II or want to contact support to have this included in future firmware updates please feel free I'm sure it will help out many others.