Mapserver/Displaying OSM Shapefiles

The Shapefile page tells you how to convert to OSM to shapefile. Once you have that you can set it up in Mapserver. Mapserver is a WMS server, and a bunch of other things.

Here's a mapfile which may help get you started. It's pretty heavy, but it uses most of the styles defined in the osmarender setup, and is built against the resulting shapefile as described in Converting OSM to GML. Of course, once you have it, you'll want a browser for it, which is where things like OpenLayers come in.