I don't understand why you don't want to use libertineotf (which is meant to be used with xelatex), but if it works with this package and not if you call the font yourself this probably means that you have more than one libertine version – libertineotf calls fonts by file names. Add \XeTeXtracingfonts=1 to your document and check the font pathes in the log file.

Btw: Don't load xunicode before fontspec. This will break the working of the package. In a modern system you don't need to load it at all, fontspec will do it.