GdalRasterLayer + VectorLayer hassle

I'm trying to display a GdalRasterLayer with a VectorLayer containing some Points (representing points of interest) in a map.
But the RasterLayer disappears when I add a VectorLayer. Even if the VectorLayer doesn't contain anything.

After searching a while I found this commitmessage (ID 193) from august the 8th:

Currently, the GdalRasterLayer has multiple issues when trying to use rasters in a map. It does not support rotation, multiple rasters, rasters with vectors, and panning or zooming past the boundries of the raster.

Modifications:

I modified the GdalRasterLayer, along with GeoTransforms.cs (to handle rotation), to fix these problems.... snipped the rest of the modifications...

Did I understand correctly, that this is not supported/implemented currently, since it seems to me that the modifcations are not really adressing the issue? I applied the patch anyway, but no effect on my issue.
I stepped through the code and ended up in GerPreview() where some logic says that the RasterLayer is outside the Maps boundaries and does not need to be displayed. The boundaries seem to change when I add a GeometryLayer, even if its empty. Or am I just looking
in the wrong direction?

But probably Im just using it wrong, since I discovered SharpMap only a couple of days ago.

Thanks for your answer.
So it should be possible to display both layers at the same time, right?
Actually I already have tried ZoomToExtents, just left it out to shorten the code in the post, but unfortunately this had absolutely no effect.
What do you mean by refresh? I see no such Method in Map or Layer.
But re-rendering the map/layers is triggered everytime I zoom or pad anyway, so I think this would have no effect too :(

There is a refresh method on the MapImage (or MapBox) control. What are the coordinates of your image? It could be that your image is so far away from your points that the map is zoomed too far out to see the image. Does it display correctly if you only
load the image? Only load the points?

I can confirm that if you are using the patch you quoted, you can display rasters and vectors together.