Help for those seeking to reduce black borders around the picture on PS3 Linux.

Veteran

This post is for those of you that if you use the FULL SCREEN version of the 480p-to-1080p resolutions modes you find portion of the picture "exceeding" the bounds of the monitor and if you do not use those modes you have large black borders around the picture.

If you follow PS3 Linux quite closely you might have found the fix already, but some of you might have not or did not care too much about getting their hands dirty with a custom kernel or re-compiling their kernel.

WARNING: Panajev should not be blamed if you have to re-install Linux on your PS3. I can tell you this guide has worked for me (and I do not see why it should give you problems with your system), but your mileage might always vary.

If you decide to compile the kernel pulled from Geoff Levand's tree (CELL/PS3 focused tree), you can follow this great guide here carefully:

If you run Fedora (I am running Fedora 7), you might want to pay attention to the end of the tutorial in which you copy back vmlinux to the /boot directory and configure the kboot.conf file. This is really horrible nitpicking, but keeping things consistent with the standard Fedora 7 naming convention should help you to do this process again in the future)

Fedora users will have to do something slightly different (for starters they have the /etc/yaboot.conf file to worry about and not the kboot.conf file, while the author seems to be using YDL).

Example of what I am saying:

cp vmlinux /boot/vmlinux-2.6.22-rc7-g9b6d7171

should be (on your Fedora installation)

cp vmlinux /boot/vmlinuz-2.6.22-rc7-g9b6d7171

(just check the /boot directory for the naming of all kernel images)

and the thing about the yaboot.conf file (the videomode I set assumes you want to use 720p with the FULL_SCREEN flag set to 0, to prevent overscan problems):

That is what you need to add in the yaboot.conf file: put it before the other kernels to make sure it boots first.

Like it is mentioned in the guide, the 2.6.23-rc3-g2733ff47-dirty string you see here is the name resulting after the make modules_install procedure has finished when "I executed it on my system" and on your system at the time you follow the kernel customization guide you might receive a different value which you will have to use in a consistent manner.

At the end of this post I have placed my yaboot.conf file hoping it might be of some help to those who have some doubts (please do ask questions, if I know the answer I will reply asap and if I do not know the answer I will tell you so too ).

Well, for everybody playing with either the custom kernel sources or the fedora PPC kernel (which supports the PS3), the file you want to look at to fix the "black borders around the screen" problem is the following:

/usr/src/linux/drivers/video/ps3fb.c (assuming you set the linux symlink appropriately as the guide specified)

so open it with vi, gedit, or your editor of choice and look for PS3FB_RES_FULL.

the line I edited, compared to the original version, is only the 1280x720 line in that quote. The other values are all default values (the conservative set of values for 1080p is {1920, 1080, 64, 32, 0}, but this is not the only thing you have to change in this file so pay attention to the rest of this post).

More conservative values are 38 and 18 for the margins and the final values depend on your TV (on mine 36 and 18 worked better).

The values 36 and 18 refer to the black borders margin_x and margin_y used on the sides of the screen (to prevent relevant portion of the image to be displayed out of the sides of the screen).

The values of margin_x and margin_y are to be intended as follows:

On the left and right sides of the picture to be displayed on the screen there is a border which is margin_x pixels wide (so in total the border is margin_x * 2 wide horizontally).

On the top and bottom sides of the picture to be displayed on the screen there is a border which is margin_y pixels wide (so in total the border is margin_y * 2 wide vertically).

The other section you must edit is the following one (in the same file):

Again here I only edited the 720p line (I use ps3fb mode 3) compared to the original file (do not go touch the "full resolution" versions of those resolution further down in the file though).

The values I edited are these two: 1208, 684 ... are they special ? Yes.

You can think of those values in the following way: 1280 - (margin_x*2), 720 - (margin_y*2)

Of course you might say: "hey, to find the best values I need to recompile the kernel tons of times!!!!!!!!! It's slow!!!!!!!". Do not panic, there is a short-cut:

Go back in /usr/src/linux/ (inside this directory or inside /usr/src/ps3-linux to keep the custom kernel updated run the command git pull once in a while and then follow the same procedure outlined in the guide to compile the kernel [with the minor modifications for Fedora users as I have posted in this thread, well if you use Fedora as your Linux distribution that is):

go inside the drivers/video/ directory and delete the ps3fb.o file.

Now edit the ps3fb.c file again and repeat the procedure you used to compile the kernel outlined in the guide I linked to in this thread from here:

Part II - Continued
make make modules_install

Click to expand...

forward (do not repeat the earlier steps just to optimize the values of the black borders or else it will take much longer to compile the kernel although it can mae you feel safe hehe ).

This is the content of the "fixed" boot-game-os script which you'll find in the /sbin/ directory: you have to be root to edit it or have root privileges.

The good news for Fedora users is that with the custom kernel (the one from Geoff Levand's tree) you can now shutdown+halt and reboot Linux without pressing the power button on the PS3 for 5+ seconds (in some cases, very very rare cases, you might still have to resort to the "less automatic" method).

you do this to shutdown and turn the power off (you must be root to execute it or have root privileges):

Code:

shutdown -h now

you do this to reboot the system (it reboots Linux and you must be root to execute it or have root privileges):

Code:

shutdown -r now

man shutdown should explain you what this command does.

Edit: if you have problems (stalling after the HAL daemon starts up), try turning the PS3 on with your monitor turned on as well (I say this because many of you might be working remotely, even at home, with ssh+vnc+samba+Visual Studio remote Makefile projects).

Veteran

Recently the ps3fb.c file changed quite a bit and at first I thought the current black borders fix could not be performed.

On the CBE-OSS-DEV mailing list I learned that it is not the case and that there was an even better way to perform what we tried to do in this thread thanks to a kind Sony Software Architect:

Code:

The first good news is that you no longer have to care about ps3fb_res[].
You can still modify ps3fb_modedb[] to achieve what you want.
> > Will there be a different/better way to achieve our goal of minimal and
> > not over conservative black borders ?
The second good news is that now you can also use fbset to tune the width of
the borders. Examples:
- If you want to decrease the left margin of the 720p mode by 5 pixels:
| # fbset
|
| mode "1124x644-60"
| # D: 74.178 MHz, H: 44.957 kHz, V: 59.942 Hz
| geometry 1124 644 1124 644 32
^^^^
old xres
| timings 13481 298 148 57 44 80 5
^^^
old left margin
| bcast true
| accel true
| rgba 8/16,8/8,8/0,8/24
| endmode
|
| # fbset -xres 1129 -left 293
- If you want to move the visible part of the screen, you can use e.g.
`fbset -move left'
Even better, ps3fb now does automatic rounding (uprounding) of
parameters if they don't fit.
If you add an empty mode to /etc/fb.modes, e.g.
| mode "null"
| geometry 1 1 1 1 0
| timings 0 0 0 0 0 0 0
| endmode
you can even let ps3fb round it up to whatever resolution you want, like
| # fbset null -xres 800 -yres 600
With kind regards,
Geert Uytterhoeven
Software Architect

So,

(I use ps3videomode -v 3 which is 720p non full-screen @60 Hz)

1.) first step:

add the following into /etc/fb.modes

Code:

mode "ps3_fb"
geometry 1 1 1 1 0
timings 0 0 0 0 0 0 0
endmode

2.) second step:

add the following (all into one line with a new line after it) into /etc/rc.local

Well, actually before doing that, read through the following if you are a bit unsure about what to do:

the xres, yres (these first two values define the resolution you want to set with fbset) and the related left, right, upper, and lower values (these last four define the black borders around the picture) are very good values for my set, but they might not be the best values for your set.

Take notice of the output of fbset first (you have to run it as root):

Values of the timings line (minus the last two and the first one... you should not need to play with them... left margin = left black border):

left_margin right_margin upper_margin lower_margin

The difference between the new xres and yres values (the new values you want to specify) and their respective original values has to be taken into account when setting the margins: in my case for example I have increased xres by 84 pixels and yres by 40 so I had to reduce by 42 pixels the values of the left and right margins and by 20 pixels the values of the upper and lower margins.

Finally, a nice kernel hacks independent way of reducing the black borders problem down to something manageable .

I use that command quite nicely enough on an updated Fedora 8 installation... I do not know the source of the command error (the fact that fbset is having issues is the reason you still have black borders IMHO, the fbset command is not working properly for you).

Newcomer

Hi guys I am also getting the "ioctl FBIOPUT_VSCREENINFO: Invalid argument" message when I try to change geometry and borders, even though I carefully did the maths so the numbers add up properly.

I am running the default PsUbuntu kernel (Gutsy 7.10, 2.6.22-14-cell), and have a 28" Viewsonic monitor whose native res is 1920x1200. When I try the ps3videomode in "full screen", it's clipping several lines of text - possibly because this monitor is known to have slightly < 28" viewable area.

Do I need to be running the custom updated ps3 kernel mentioned above for this fbset method to work? Thanks.

ModeratorLegend

I've a vaguely related question. My TV is 1680x1050 native, supporting 16:9 resolutions. If I run Linux on PS3 will I be stuck with 720p upscaled? That'll be no good for running a computer. Or 1080p downscaled? The PS3 browser is pretty blurry, and if Linux will be like that then there'll be no point in me trying it.

Veteran

I've a vaguely related question. My TV is 1680x1050 native, supporting 16:9 resolutions. If I run Linux on PS3 will I be stuck with 720p upscaled? That'll be no good for running a computer. Or 1080p downscaled? The PS3 browser is pretty blurry, and if Linux will be like that then there'll be no point in me trying it.

Click to expand...

I do not remember if that resolution is part of one of the documented display modes for ps3fb, but you should be able to use 1080p with borders set appropriately to give you an as close to 1:1 picture as possible.

About Us

Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!