I’ve been using Web Services for many years prior to .Net, but with .Net I continue to have some serious mind blocks in terms of how to best pass data between the client and server, specifically when passing data in the form of objects/entities.

The problem I have is that in my business scenarios I usually have business objects both on the client and on the server. I download data from the server, then stick it into client side objects that often times are identical to the server side objects. The problem is .Net does not make this sort of thing easy with its proxy based approach.

The problem is that .Net creates proxy copies of my business objects. These objects are only container shells of the full objects. They get created even if there is nothing omitted from the original object as is the case in a typical entity object. Instead I end up with a weak shell of an object that servers for nothing more than transport.

Once the data arrives on the other end I can’t use the data as is. I have to now move this data into my real business objects that actually do the work on the data. Note that this is true even if I create pure data container classes that don’t include methods.

I keep asking this question and I keep getting blank stares from people – am I really out of my mind here? This seems like a very common scenario. Most people then continue to tell me to pass DataSets across the wire, which works a little better, but even there similar issues exist. If I pass a dataset to the client app I still have to go through all sorts of rigamorole to reattach the data I downloaded to my real life business objects by merging or otherwise kludging the data logic to force the DataSet into the right state on the client.

I’m writing about this because I’m curious for some input. Is everybody really using mostly DataSets to pass data over the wire instead of objects? I mean this completely goes counter the whole OO paradigm of dealing with data in the first place. The Proxy approach works well in true disconnected to the server scenarios, but it's not a good match for side by side applications which in my work tends to be the more common case. Side by side meaning two separate sets of data on the client and server with data being shared between the two as opposed to a central database that is always updated on the server.

What I would like to see at the least is an easier mechanism to map types for the proxies generated. I know I can do this by writing custom code, but it would be nice if we could specify the classes that are used for proxies conditionally.

Another area that would be useful would be tools to facilitate object copying. I’ve built my own routines to do this and true it isn’t rocket science, but this happens frequently enough that it sure would be nice to have a generic way to clone a serializable object easily. So when I get back my AuthorProxyEntity I can then assign it easily to my live Authors.AuthorEntity instance.

I do this now with a cheesy Reflection based CopyObjectData() method, which basically goes through the target object and copies any properties it can match from the source object. I have the same sort of thing for a DataRow object which guarantees that the data retrieved is in the proper update state.

So, am I off my nut? What are other people doing to pass data around and avoid having to write and re-write object parsing code each time?

The Voices of Reason

What I tend to do is pass objects as XML string around. Even when using .Net on both the server and client side this gives a better performance than passing DataSet objects. Why this is I don’t know but this is true both with Web Services and Remoting.

With my own objects I tend to put these in a separate, shared, DLL and include a ToXml and FromXml function to facilitate the conversion. Yes this means I have to do some additional stuff each time I pass an object around but at least I get a real object on the client instead of some generated proxy object. An additional benefit I only discovered later was that not all DataSets can be serialized. Adding RowError information is enough to trip the normal serialization done by web services.

Maurice, this is interesting especially about the XML String vs. Serialized DataSet. I have to test to verify that but I find that hard to believe. As you probably know I've done lots of object persistance stuff in Fox which uses wwXML and the ability to genereically take XML structures and assign them to various sources including Objects. This is not difficult to do at all especially if you have an object structure to start with, but I kind of had gotten used to the idea of not having to write this kind of low level XML parsing any longer with .Net. The more I look at some of the stuff that I'm working with at the moment the more I find that using the old approach has still much merit left in it and this auto-Proxy approach with complex object is nothing but a major PITA that can't be debugged if it goes wrong. For example I have two methods in my Web Service currently that receive the same object parameter. Signature is otherwise identical. One works the other doesn't (fails with an internal serialization problem). Very frustrating as I see no way to get past that error.
If I pass XML manaully I can at least fix this stuff or work around it.

Rick: If you can use an ObjectSet, it implements IXmlSerializable which has methods to RealXml and WriteXml. In my mind, this would be the best way to (de)serialize an object graph of business objects for distribution. Note that this implementation is broken in the PDC bits. I haven't tried the 2005 version yet.

David, it's interesting that you bring up ObjectSpaces, because I was working with entity objects yesterday for some of this stuff (it's for an article actually). I didn't use OS, but I used some of the same constructs. One problem I see with ObjectSets and Entities is that they are serialized as objects without schema (I think - haven't tried this). If you share this with none .Net applications they're not likely to be able to utilize the XML generated.

Rick, if you use the serialization build into the ObjectSet you get schema in the xmlns:osd="http://schemas.microsoft.com/data/2002/09/20/persistenceschema namespace. Hmmmmm. The OSD mapping file makes use of this schema too. The actual serialized content looks like a diffgram. If you use a custom collection of entities you would need to write your own serialization logic but I wouldn't recommend writing too much of your own if Microsoft is working on it. One of the interesting aspects of serializing a bunch of entities is what if the same entity gets referenced twice (two orders point to the same product)? You don't want to just copy the properties of the object because you don't want two product objects on the other side. You would have to serialize using ID/IDREF or use some kind of an identity map when deserializing. The only reference that does something like this that I can find is XMLObjectWriter http://www.123aspx.com/rotor/rotorsrc.aspx?rot=41985Ultimately, you may want the same entity to be serialized in several formats depending on the application (security, etc). Why not create a declarative mapping file to map objects to xml? I think this is what the OSD mapping file is for (Object Schema Definition)

Dave, ObjectSpaces is an interesting technology for sure, but it's too far off to really take all that seriously at the moment. Heck even in the current Whidbey refresh the mapper isn't working and most of the OS implementation is barely documented. Have you tried serializing with Paul's ORM Mapper? What does that get you?

The Wilson ORM marks the ObjectSet with the <Serializable> attribute. I haven't looked at exactly what this gives you but I have a feeling this is going to be totally inadequate for what most people really want. I was going to work on implementing a serialization strategy for the Wilson ObjectSet but I haven't gotten around to it. Maybe you want to attempt something like this?

Dave, Serializable will serialize Ok as long as all items in the chain are marked as serializable, but you'll get no schema (except what get written into the WSDL in its convoluted form). This may work for receiving, but is going to be tough for sending. Same situation that we were stuck in with VFP...

> Is everybody really using mostly DataSets
> to pass data over the wire instead of
> objects?
This is what I've seen most .NET developers doing.

> I mean this completely goes counter the
> whole OO paradigm of dealing with data in
> the first place.
Yup.

> The Proxy approach works well in true
> disconnected to the server scenarios, but
> it's not a good match for side by side
> applications which in my work tends to be
> the more common case.
Have you tried with Remoting? I thought Remoting does "deep serialization" and you wouldn't run into these issues -- but I really haven't tried it.

Rick is right in saying that there should be option like remoting to pass custom objects to and from Webservices and Strictly speaking it is not supported directly as like remoting and interfaces.

Let say i have a custom dll and i am adding the reference to the same in both client app and webservice everythign is fine here.BVut when i am assign the object of the same from the proxy to our object in client it didnot allow as it didnot consider the object same from the same class when it come thru proxy.

Here i had 2 options use the proxy object's object only by using some thing like
localhost.websrvnm.clscustom proxy=new localhost.websrvnm.clscustom
and
localhost.clscustom obj =new localhost.clscustom
then obj=proxy.method (which return the class type)

However the same is fine with interface approach in remoting.The question is why it is not so in webservice while i made a search i found this answer which is much closer taht is
since the contact for webservice is wsdl which didnot understand the class we are providing the mismatch occurs.

Forget to add one line that if you remove the class definition from the proxy generated you can make the same work taht client and server now talk to each other but every time a webreference is get updated you need to modify the proxy again.
any comments are welcome
sndshreeman@rediffmail.com

Technically the classes are not the same. .NET generates the proxy in a new namespace even if it has the same name, so the class signature is different and can't be replaced. The real issue is if the class is autocreated VS will generate a proxy, where we would really like to forego the proxy and have it load up a REAL class. This wouldn't be quite like remoting but go a long way toward making life easier when retrieving data from a Web Service.
Alternately some functionality that can copy object data would be very welcome to at least make the job of pulling the data out of a proxy into the real object.

This is an interesting BLOG. I had an issue this week where I had a collection of objects that I needed to move from a WebService to a front-end and vice-versa. I was then going to bind the objects in the collection to a form. Web Services dumbs down objects to field values instead of Properties, and therefore I could not bind my objects. Furthermore, the objects coming back from web services are completely different than what you start with.

My solution was to serialize the objects to and from the Web Service. So I always pass out and in a Binary String. I then de-Serialize the object on the other side and Voila -- I have my original object with all its properties in tact.

Passing DataSets to me is quite insecure as it gives your entire data Schema to hackers. Binary Serialization makes the web Service act more like a business layer as opposed to an inter-operable middle-tier. So if your intention is to work with other languages, then this is NOT the option for you. XMLSerilization may be better.

Hope that explanation helps. If anyone needs a code example, just holler back.

Hi Osie, yes I've considered this as well. This makes it possible to get the data across efficiently but it also defeats the process of a Web Service somewhat because now you end up having to add some client logic to the server.

Couple of things in this: You are passing strings? Are you sure? I think you mean passing byte arrays? Additionally realize that binary serialization does not serialize DataSets as binary, but still as XML! The reason it is not in the clear is because it's base64 encoded in your case which is not really difficult to undo! So, this is no security at all - if you need security make sure you run over HTTPS.

I've run into this problem on numerous projects. I was looking for a better way when I came across your blog. Here is what I did in the past...

Serialize the proxy classes to XML using the XmlSerializer, then Deserialize to business objects using the XML. Since the proxy object is XmlSerialized from the business object on the server, the Xml schema should be the same.

It's kind of sad to be forced to do this, but it works. I am still looking for a better approach.

I do agree with casie,rick and the trick all we follow is somehow use serilization one end and deserialize at the other.I discussed the same with ms people and what explanation i got is this a architectural design decision that what webserviec meant for is passing the data in xml format from one endpoint to other and it is not recommended for type sharing etc.,
After i went thru few more blogs and articles i agree with them .
The problem is still there and will be there for more time i guess as we are thinking in oops way that is passing and sharing data and we are right since we are on n tier distributed arc but they are asking for SOA and aspect orientation.They are right in a way if we see the purpose of webservice but still we need some more improvement specifically on serialization issues ..
hope that serialization and validation will be incorporated in webservice soon.

Hi Shreeeman, I've had that argument before but I don't really buy into it. While Web Services are a message mechanism rather than an Ooop mechanism, Microsoft has made it explicit policy to make it an Ooop mechanism. In stock .NET 1.1 you can't even pass messages in any other way than to use objects (with WSE 2.0 you can create messages in code more easily). If they stuck true to the SOA model than there would be at least a way to easily create SOAP messages via code with SoapParts.

This is really a use case issue, IMHO. 90% of the time you will want to serialize/deserialize a type. So why should this not be supported in some way (and I'm not saying that this is should be the default)? I could see the argument if the way the implementation proxy would be completely immune to OOP version issues, but the proxy is created from the real type underneath it and so the main reason to stay with a SOA type message - version independence - isn't even served either.

Ulimately I think the real solution to this problem would be better support for copying of objects in the framework. Being able to take the message and more generically assign it to an object would serve both practicality and the 'purists' who want to preserve the message as a pure message mechanism. I have my own code to do this, but it feels like a hack - at least if the framework supported this it would be an 'official' hack <g>...

hi Rick
Agree with you that "as it is" serialization should occurs and better mechanism should be provided from MS as they have now converted the WS more then "simple" message 'xml' transfer that is if we are not passing the and geting data then it is not worth for us.
however still webservice can't be a pure oop in the sense that type sharing is a probelm and further the gotcha is not a single vendor is creating the client and server.

also my point is that they should ship some standard on the same.
But i didn't get ur point on "In stock .NET 1.1 you can't even pass messages in any other way than to use objects "
we could pass dataset and xml compliance variables say string..as well as objects from service to client which will be accessible thorough the proxy though with limitation and with lost originality from the parent class... and though i m not favour of typed dataset...the point however is that how can we put them in an SOA model and AOP.

Very much agree with you on the similar copy topic and furtrher on why should we create serialization/deserialization always or why should we modify the generated proxy r create our custom one using soapclient protocol.

do update me as well whenever u found something update on same.
thanks
shreeman
sndshreeman@rediffmail.com

There's no way in stock 1.1 to build Soap Messages by hand. You have to add a Web reference and use that. There's no way at runtime to decide - "I want to use this service adn call it now" (short of generating code at runtime).

I also think it makes no sense to say that because we're using objects to insinuate that that automatically won't fit with SOA. Microsoft has always used the object as the mechanism to translate and if proper serialization is added as a (optional) feature it won't affect the messages on the wire. The raw Web Service logic isn't affected, only how that data is generated and optionally consumed from our own code. Since most Web Services are internal and likely controlled by the same entity even this makes a lot of sense.

Again, I think the better way to solve this for Microsoft maybe just to provide better tools to copy objects or provide serialization in a way that doesn't have to map 1->1. This is basically how my routines work - they allow some flexibility, so if there are properties that exist on the proxy but not on the imported object or vice versa these properties are ignored. This provides the version flexibility that you would otherwise have to code anyway...

I am in process of solving the same problem. At present, my solution is to manually modify the VS.NET generated client-side proxy code (Reference.cs). I have the same assembly on both client and webservice. This assembly has a class named DictionaryPair, which is returned by a web method. I added a reference to my assembly to the generated proxy code, left the [System.Xml.Serialization.XmlIncludeAttribute(typeof(DictionaryPair))] as is, but removed the DictionaryPair class definition from the proxy. So far everything has compiled and my initial tests indicate things are working.

Of course the down side to this is that if (when) I regenerate the proxy due to additions to the web service (I've really just started coding it), I will have to manually reapply my changes. Perhaps I'll try to write a VS.NET macro for this...

I find this thread very interesting. I have faced this issue in my own code. What I think the problem is, is that these are *webservices* not object transport mechanisms. Obviously, one has to send data to the service, and receive data back, and granted Msoft could do more here.

However, if you require duplicate copies of objects being used by webservice *and* your client, something is wrong. The logic to operate on those entities should reside with the webservice server, not on the client. If the client can operate on the objects themselves, then why are you using webservices?

I think perhaps if one examines the functionality of the objects in question it will become clear that some things clearly belong as a webservice and other things clearly can (or should) be handled on the client.

People are passing around Datasets because this makes a good parameter passing mechanism in the case of complex sets of data between a service and a client.

There is a tendency to see everything as a nail when one has a hammer, and perhaps that is the case here with webservices.

While I agree to some extent I think it's silly to say - this class belongs to the Web Service and therefore you have to manage how to get data out of it. While that certainly could and probably should be the default the 90% plus scenario will be that you'll load this data into an object that has the exact same structure! So now you're writing manual process code to property mapping, right? Over and over again for every object that passes data over the wire? No way... this process should be automated.

It doesn't even matter whether this object runs in .NET, Java or Visual FoxPro. Those other tools will have to have their own mechanism, but in .NET regardless of whether you're a SOA purist or not - you get back an object. An object that on its own is nothing more than a data container that you can't do anything with other than copy it.

What I propose is that there should be an easy way to copy objects. That way the infrastructure can stay as it is, and those that need it can easily and effectively copy objects from one structure into another. There could be additional options for skipping members that don't match on both ends without failing. As I said above I do this now manually but it'd be much nicer if this was part of the framework.

Looking at my Web Services I see that well over 90% are one to one matches of object to what goes over the wire. I don't think I'm unique in this - this is the 90% use case.

Your issue is valid, I can not speak for most but the best solution to date I have decided on - is to write a code generator that reads my database and creates transport objects - where by they seralize properly and move the data from one end to the other leaving me an object on the other end. I want to believe that datasets are the right way to handle this delima but the fact that Java does not happily work together or even recognize the format (the famous :any dilema) means that datasets say to the world you only care about .NET (which is fine but seems like it only serves to be a bandaid) I am not a huge fan of typed-datasets because I like my model to float from version to version leaving the client operable.

How do I send an object as parameter to a webservice method and return an object from that method to the client ?

Explaination:
I have a class named UserInfo which represents a table named UserInfo. It contains user loginid and password as properties which are the fields of the table UserInfo. Now i have to pass this information as an object to a method called CheckLogin in webservices and return an object which contains the login success status to the client depending on which i will display a relevent message on the login page.The problem lies with serialization/deserialization.
Hope this info is sufient to understnd the problem..

Kindly get back to me as soon as possible..Its very urgent..Please Help!!

I thought I was the only one who could see a gap in the .Net Web Services. I've been working on a large application where we use Web Services for scaliability to load balance. The problem is that the client can use the same set of User defined objects which get passed from the web service (serialized) to the client and back again, but I lose all of the methods in the objects and just get the Properties.

From reading this...it sound like I can either serialize and then deserialize on the other end OR edit the Reference.cs to just pass through that type rather than a new Web Service Instance of it.

As another note, if you create more than one .asmx and pass the same object to the client from two different .asmx, they also have different namespaces and can't be cast to one generic object on the client side. So if you grab for instance a Person object from one .asmx and then pass it to another, you need to do what Rick was saying about Copying to the other object type. This causes all sorts of problems with when you add new properties remembering to add them to tyour Copy functions! Again, you could get round this by editing the namepaces in the Reference.cs.

Does anyone know of a way of editing how the Reference.cs is created?

I use NullableTypes and when you pass these through in an object, the 'using NullableTypes' doesn't get added and we currently use a text replace script to enter these in. But there must be an easier way surely! Maybe a template or something?

Rick, I agree totally with all of your points. There needs to be something in the framework to de-proxy a received soap 'object' into a local object type.

In order to solve this problem I have been considering writing the same 'cheesy Reflection based CopyObjectData() method'. In my situation the server has a copy of all of the classes that are on the client (a PPC in my case) so there will be guarenteed a 100% match between members between the client and the server. This has the effect of moving the problem half a step sideways since I need to first copy the server object into the server implementation of the client object, send the client object through the pipe and then do a CopyObjectData at the client in to the final client implementation of the client object.

The idea of moving datasets through SOAP does not sit well with me. It may be good for simple solutions or where there is only a couple of SOAP calls but I am working on large enterprise projects in which everything is as strongly typed and as OO as possible and there is a lot of communication over SOAP. This philosophy applies to interprocess communications and as a result we run smack into the problem beign discussed.

For any posterity that come across this thread looking for answers, it is very easy to fix this problem in visual studio2005 (It may work in 2003, I havent checked).

First go to your buisness objects, and make sure they are marked [Serializable()]. You might consider giving them an xml namespace (all the same ns) as well as making sure all your webservices have the same namespace. I'm not sure if the namespace bit is required, but it's what I did.

Next go to the generated Reference.cs file (you may need to hit the button to show all files)and add in the namespace(s) of your buisness objects (the dll(s) have to be referenced in the project, obviously). Then scroll down in the Reference.cs file and start deleting the auto-generated stubs for your buisness objects (just the object class definitions, dont delete anything else).

Thats it; compile away. If you have a second webservice do the same in it. Now you can pass objects from one webservice to the next, etc etc, just like you thought you were going to be able to do in the first place.

Note that your buisness objects may behave irreguarly if you attempt to call database functions when no database is availible, so a little caution or foresight in design is needed. Also note that every time you update your web reference in visual studio it will regenerate the Reference.cs file, and you will need to redo your changes. (Or do as I did and write your own little program to generate the Reference.cs file to use your buisness objects and save yourself the hassle)

So I spent a bit of time this evening mucking around with WCF Serialization. The efficiency, complexity and usability of Serialization is a key component in using any technology that needs to communicate data over service/application boundaries and so this is usually one of the first things I take a look at. WCF is pretty impressive in this regards for its flexibility and implementation.

I am trying to solve similar issues - sharing data objects between the the web service and the client. Does WCF address this? Does WCF get rid of the problem where you have an object on the server, an object in the proxy and then you have to copy it to the client object?

why don't you just create your own proxy rather than use the proxies that are automatically generated. Inherit from System.Web.Services.Protocols.SoapHttpClientProtocol, call the invoke method and cast the returned object array to whatever object you want. You can copy all the necessary code from the proxies that are automatically generated but basically it's the following:

[System.Web.Services.Protocols.SoapDocumentMethodAttribute("http://MyWebService/GetPerson")] public BusinessObjects.Person GetPerson() { //Note - weird aspect of invoke is it only works if the method name it is called from // is the same as the method it is calling. i.e. if this method was called something // other than GetPerson the Invoke would fail. object[] results = this.Invoke("GetPerson", new object[0]); return ((BusinessObjects.Person)(results[0])); }

Use schema importer extensions, Jelle Druyts has worked out an example showing how to use this (not without minor bugs though). Still doesn't solve the problem of serializing a non-tree object graph, still looking into this, might switch to normal Serialization in stead of Xml serialization (I don't care about the WSDL basically).

Returning and using objects from a web service with VS2005 has become far much simpler. Instead of coding serialization routines manually its now a matter of simply adding references to your project. Object binding can also be done using smart tags. This as demonstrated <a href = " http://www.xhydra.com/dotnet/returning-objects-from-a-net-web-service.html" target = "_new" >here </a> and <a href = "http://www.xhydra.com/dotnet/binding-objects-to-client-side-controls.html" target = "_new">here</a>

Vin - that's not really the point. The point is that you have data and need to get into your actual business objects somehow. Databinding to data from Web Services directly is rarely what's called for in a business application - you're binding to data that needs to be saved back to a database eventually with validation and other rule checking.

Rick, I know this is an old discussion, but interesting nonetheless. What route did you end up taking? Custom mapper/object copier? I've found myself in a similar situation, and I think in the end I may move my business layer out in front of my web services layer. With this approach, I'll just use data transfer objects between the services and biz objects. Although the copy approach would work just as well, I don't like the idea of having to reference an external biz object library from both the web service and the client end--it doesn't seem correct that the client would have to have any sort of "outside" knowledge of the objects it retrieves from the service. I believe the client should be able to reference the service itself for all of it's object definitions.

@Russell - I suppose it depends what level the client works at. For example, I have several applications that are both full applications on both the client and server (ie. an ASP.NET app, and also a full Desktop app that only uses the server to retrieve updates). In that scenerio business objects run both on the 'client' and the server.

So if you're building a completely thin client then yeah it makes sense to not have a heavy business object on the client, but otherwise - it makes sense. I suspect there are a LOTS of applications that actually work in such an environment. In a pure thin client environment message objects as is are probably sufficient. Still even there it can be useful to have more full features business functionality like access to validation rules and other business operation.

FWIW, WCF address this scenario a lot better by allowing you to define a [DataContract] explicitly.

and you can share those message types on both ends of the connection which solves this problem to a large degree.

Ultimately though I think that what would be really nice if the framework had some generic mechanism for copying objects based on structure so that you can more easily map one object to another explicitly.

I know this blog entry is moldy-oldy but the problem still exists: how do you move a strongly typed class through a web service without namespace conflicts, serialization, or resorting to datasets or flattening your class into a string.

The posting by Metro Sauper May 19, 2006, pointed to MSDN, and following that link which is now at the location below, I think I've found a solution, at least for a simple class that has nothing but value types and get/sets for all properties.http://msdn.microsoft.com/en-us/library/aa480505.aspx

Some manual voodoo is required but it only took a minute. The biggest issue I see is trying to remember this pattern for all projects. If it turns out to be inefficient or otherwise ineffective, I'll post back here, but I encourage everyone to just try out the technique described and let's see where it goes.

To me, Web Services are a retrograde step from COM+ objects for this very reason.

With vbscript and COM+ I could create an instance of an object on a completely different computer with: -

Server.CreateObject("SomeDistributedObject")

and an instance exists with all the properties and methods.

with Web Services, not only do you need a copy of an assembly on both the server and client (versioning issues), you need to add an intermediate stage of populating a new instance of an object from the returned data, or deserialize an object passed.

Either way, there is no longer a central point where the class exists and can be referenced from anywhere. instead, it's a clumsy system of redistributing an assembly, to all dependents, whenever a change has been made.

hi, everybody, i have a simple question to ask ! i'm consuming a Java Web Service (On Netbeans 6.8) with a .NET client (On VS2008), and i used Serialization to pass objects ! and now i need to pass the XML file, i thought about Converting it to a String ! and pass it as a String to the WS, is this a good idea (i have a feeling that what i'm doing is stupid !)if it is, can somebody give me a better way ! Merciiii.

@BC - it depends on what the Web Service expects in terms of the actual 'type'. Is expecting a string, or an XML Entity? If it's the former what you're doing should be fine. If it's the latter you'll want to send an XML Document or XML Node to the service. When you import the the WSDL it should provide you with the proper type definition for the method you're calling or property you need to set on the service messages.

The two objects are essentially the same (as far as serialization is concerned), but with different namespaces, so why not just serialize the Web Service object, replace the namespace with the 'real' object's serialized namespaces and de-serialize it as the 'real' object. That could be a generic utility method, something like below: -