The Gallery theme is a beast to start out with. I happened to have pro installed, which is a tiny bit different, but should be similar.

First the RSS in the upper right is coded into the gallery_functions.php
<div id="page-menu" class="buffer">
<ul id="page-nav" class="sf-menu">
<li class="rss">">RSS Feed
<?php
wp_list_pages($pagequery);
?>

</div>

Dropping off that whole li line with the class rss is what you are after. As far as the menu order goes, I believe that is a menu setting in the actual wordpress panel, gallery should come with 2 optional menus, you can set it up there.

To modify the black bar with the categories, you will find that information in your styles CSS, you can see the background is applied to the #access id.