Created attachment 5257[details]
reproduces described bug when run under mono
When using a custom formatter for outputting JSON-encoded data using WCF, an exception is thrown when a web-call with json-output is made:
Exception Cannot cast from source type to destination type. at System.ServiceModel.Channels.WebMessageEncoder.WriteMessage (System.ServiceModel.Channels.Message message, System.IO.Stream stream) [0x0010f] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel.Web/System.ServiceModel.Channels/WebMessageEncoder.cs:190
at System.ServiceModel.Channels.Http.HttpRequestContext.InternalReply (System.ServiceModel.Channels.Message msg, TimeSpan timeout) [0x00034] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Channels.Http/HttpRequestContext.cs:139
at System.ServiceModel.Channels.Http.HttpRequestContext.Reply (System.ServiceModel.Channels.Message msg, TimeSpan timeout) [0x00000] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Channels.Http/HttpRequestContext.cs:101
at System.ServiceModel.Dispatcher.MessageProcessingContext.Reply (Boolean useTimeout) [0x00026] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/MessageProcessingContext.cs:96
at System.ServiceModel.Dispatcher.OperationInvokerHandler.Reply (System.ServiceModel.Dispatcher.MessageProcessingContext mrc, Boolean useTimeout) [0x0001d] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/OperationInvokerHandler.cs:69
at System.ServiceModel.Dispatcher.OperationInvokerHandler.ProcessRequest (System.ServiceModel.Dispatcher.MessageProcessingContext mrc) [0x00044] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/OperationInvokerHandler.cs:29
at System.ServiceModel.Dispatcher.BaseRequestProcessorHandler.ProcessRequestChain (System.ServiceModel.Dispatcher.MessageProcessingContext mrc) [0x00000] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/BaseRequestProcessorHandler.cs:15
at System.ServiceModel.Dispatcher.BaseRequestProcessorHandler.ProcessRequestChain (System.ServiceModel.Dispatcher.MessageProcessingContext mrc) [0x00017] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/BaseRequestProcessorHandler.cs:16
at System.ServiceModel.Dispatcher.HandlersChain.ProcessRequestChain (System.ServiceModel.Dispatcher.MessageProcessingContext mrc) [0x0000b] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/BaseRequestProcessor.cs:72
at System.ServiceModel.Dispatcher.BaseRequestProcessor.ProcessRequest (System.ServiceModel.Dispatcher.MessageProcessingContext mrc) [0x00018] in /home/unknown/mono-3.2.3/mcs/class/System.ServiceModel/System.ServiceModel.Dispatcher/BaseRequestProcessor.cs:26
This exact same binary works fine under Windows with .NET framework 4.0. I've observed the above exception with the following mono versions:
- Mono JIT compiler version 2.10.8.1 (Debian 2.10.8.1-8)
- Mono JIT compiler version 3.0.6 (Debian 3.0.6+dfsg-1~exp1~pre1)
- Mono JIT compiler version 3.2.3 (tarball Mon Oct 28 20:45:22 CET 2013)
A reasonably minimal example to reproduce this bug can be found here:
http://code.msdn.microsoft.com/Supporting-different-data-b0351c9a
(a compiled exe has been attached to this bug-report)
The accompanying blog-post for this code is here:
http://blogs.msdn.com/b/carlosfigueira/archive/2011/05/03/wcf-extensibility-message-formatters.aspx
And finally, another user encountered this bug (in an identical use case) and posted about it here:
http://forums.xamarin.com/discussion/6320/call-to-a-rest-service-with-json-with-a-content-type-format-raw-results-in-invalidcastexception
Please let me know if I should test this against trunk, or provide any more details.

So, after some debugging I came up with a fairly dirty solution to this issue. I'm attaching a diff for the modified WebMessageEncoder.cs to this post. I tested this patch with the 3.2.3 version and the debian 2.10.8 version as well.
It is far from a good fix, as I barely understand where the RawMessage is supposed to be created; I am merely interpreting a SimpleMessage in the WebMessageEncoder in the case the message is not a RawMessage. Aside from that, the patch does work in my particular use-case, and should not break any existing behaviour (no tests failed). I doubt it's optimal in terms of performance though.
I hope someone can find a more elegant solution to this issue, but in the meanwhile this will have to do.