Entity Framework is world’s most difficult ORM for n-tier application. See how I have produced a 100% unit testable fully n-tier compliant data access layer following the repository pattern using Entity Framework.

Introduction

Entity Framework was
supposed to solve the problem of Linq to SQL, which requires endless hacks to
make it work in n-tier world. Not only did Entity Framework solve none of the
L2S problems, but also it made it even more difficult to use and hack it for
n-tier scenarios. It’s somehow half way between a fully disconnected ORM and a
fully connected ORM like Linq to SQL. Some useful features of Linq to SQL are
gone – like automatic deferred loading. If you try to do simple select with join, insert, update, delete in a
disconnected architecture, you will realize not only you need to make
fundamental changes from the top layer to the very bottom layer, but also
endless hacks in basic CRUD operations. I will show you in this article how I have added custom CRUD functions on top of EF’s ObjectContext to make it
finally work well in a fully disconnected N-tier web application (my open source
Web 2.0 AJAX portal – Dropthings) and how I have
produced a 100% unit testable fully n-tier compliant data access
layer following the repository pattern.

In .NET 4.0, most of the problems are solved, but not all. So, you should
read this article even if you are coding in .NET 4.0. Moreover, there’s enough
insight here to help you troubleshoot EF related problems.

You might think “Why bother using EF when Linq to SQL is doing good enough
for me.” Linq to SQL is not going to get any innovation from Microsoft anymore.
Entity Framework is the future of persistence layer in .NET framework. All the
innovations are happening in EF world only, which is frustrating. There’s a big jump on
EF 4.0. So, you should plan to migrate your L2S projects to EF soon.

Extending the generated ObjectContext

First you have to extend the ObjectContext that gets generated when you add
an ADO.NET Entity Data Model. I usually create a new class and then inherit from
the generated class. Something like this:

The original ObjectContext is called DropthingsDataContext.
Throughout the code you will use this extended XXXX2 class than the original
one.

Next step is to set the MergeOption to NoTracking for each and
every entity. If you don’t do this, the entities are attached to the
ObjectContext’s ObjectStateManager by default, which tracks
changes to the entity so that it can save the changes when SaveChanges is
called. But that also means your entities are tied to something on the data
access layer and such tie is strong tie. As long as the entities are alive, the
tie exists. So, you need to detach the entities before you pass the entities
through the tiers. However, when you call the Detach(entity) function, it
not only detaches the entity, but also the entire graph. That’s good and
expected. But what’s unexpected is, all the referenced entities and collections
are set to NULL. So, if you have to pass an entity to other tier and you have to
carry along all the referenced entities, there’s no way to do it in EF. That’s
why I have done this:

I am setting the MergeOption to NoTracking for all the entities in the
constructor. This makes all the entities to be by default “detached”, but still
keeps the references to child/parent entities/collections intact. They aren’t
put in the ObjectStateManager.

Next I create a static dictionary which contains the Attach and AddTo calls
for each and every entity.

This is a much simpler syntax and especially here you don’t need to know the
internals of doing join and which keys in two tables participates in the join.
It’s completely abstracted on the ObjectContext design. It is good!

But getting entities from a many-to-many join was not so trivial. For
example, here’s a query in EF, which selects a related entity based on many to
many mapping.

from widgetInstance in dc.WidgetInstance
where widgetInstance.Id == widgetInstanceId
select widgetInstance.WidgetZone.Column.FirstOrDefault().Page

The entity model is like this:

Notice the FirstOrDefault() call inside the select clause. That’s the
think does the many-to-many query on the Column table and maps the Page
entity.

Similarly, say you want to select a far ancestor’s property from a great
great great great child. For example, in this diagram, you have the ID of a
WidgetInstance at the bottom left. Now you want to select only the
LoweredUserName of the ancestor entity User. You can see from the diagram that
there’s a parent object WidgetZone, then that’s related to Page via a
many-to-many mapping using Column, and then the Page has a parent object User.
And you want to select only the LoweredUserName of User.

For this the query in EF is like this:

from widgetInstance in dc.WidgetInstance
where widgetInstance.Id == widgetInstanceId
select widgetInstance.WidgetZone.Column.FirstOrDefault().Page.aspnet_Users.LoweredUserName

If you wanted to do this with Linq to SQL, you would have ended up with a
query that’s the size of a small blog post.

Inserting a disconnected entity

Here’s the generic version of Insert<> which can insert any
disconnected entity.

It confirms the entity is inserted, an auto identify is generated and
assigned to the entity as expected and also the entity is returned as detached,
which you can now pass through the tiers.

Inserting disconnected child entities

Inserting a disconnected child entity is so insanely different than other
popular ORM libraries, including Linq to SQL, that if you have a repository
layer, get ready for some massive refactoring. The common principle of inserting
a child entity is – first you have to attach the parent entity in the context,
then you will have to set the mapping between the parent and the child (you
can’t have the mapping already!), and then you will have to call SaveChanges.
Here’s the code:

You must have spotted the horror and pain of inserting child object by now.
Say from some upper layer, you got an entity with parent entity already assigned
to it. Before you can insert it, you will have to first set the parent entity to
null and then attach it. If you don’t do it, then Insert silently fails. No
exception, you just get nothing happening. You will wonder yourself - the poor
EF Program Manager must have been severely underpaid.

If you search around, you will find various alternatives to this. Some tried
the context.SomeSet.AddObject(…) approach. That works fine for one and only one
insert per context. But you cannot insert another entity of same type, having
the same parent entity using that approach. I have tried 4 different approaches,
this is the only one that works in all scenarios – both connected and
disconnected, having parent entities as real and stub, inserting one or more
within same context or in different contexts. I have literally written over
thousand lines of test codes to test all possible ways of inserting child
entities to prove this works. EF SDETs in Microsoft, here I come.

Inserting disconnected many-to-many entities

Just like child entities you can insert many-to-many mapping entities. You
can treat them as if they have two or more parents. For example, if you look at
this diagram:

Here the Column is a many-to-many map Entity. So, it has two foreign keys –
one to WidgetZone and the other to Page. In EF world, you can think Column
having two parents – WidgetZone and Page. Thus the Insert<> is similar to
inserting child entity.

Again there are several ways to do this kind of Insert available on the web
and I have tried many of them. Most of them fails when you try to insert
multiple child entities using the same context. This one is proven to work. I
got hundreds lines to test code to back my claim.

Updating a disconnected entity

Update is not straightforward as Insert. First you will have to Attach the
disconnected entity and all referenced entities, keeping in mind they might
already exist in ObjectStateManager and thus trying to attach the entity will
result in the dreaded:

An object with the same key already exists in the ObjectStateManager.
The ObjectStateManager cannot track multiple objects with the same key

The common solution found on the internet for updating a disconnected entity
is like this, that works in most of the common scenarios, except one
not-so-common scenario. The common Update function you see is like this:

First it attaches the entity in the context. Then he SetEntryModified
function will go through all non-key properties of the entity and mark it as
modified, so that the context sees the object as modified and does an update
when SaveChanges is called. SetEntryModified is like this:

This works fine when you load an entity in one context and then try to update
it in another context. For example, the following test code shows loading an
entity in one context, then disposing that context and trying to update in
another newly created context.

This works fine as expected. Even if you have loaded the entity with all
referenced entities, and then passed through the tiers and then got the entity
back for update without the original referenced entities, it works as well.

This is a typical n-tier scenario where you have one entity, without any of
the referenced entity. Now you are about to update it and you only have the
foreign keys at hand. So, you need to create stubs for the referenced entities
from the foreign keys so that you don’t need to hit database to read the entity
again along with all referenced entities. The above test passes for this
scenario as well, but the one that fails is close to this scenario. I will get
to that. Now see another scenario.

In a high volume n-tier app, you cache frequently used entities. Entities are
loaded from cache and then changed and updated in database. Something like
this:

Here I am simulating a caching scenario where the cache is an out-of-process
cache or a distributed cache. I am loading an entity along with its referenced
entities, just to be sure and then updating some properties. After the update,
the changed entity is serialized. Then in another context, the changed entity is
created from the serialized stream. This ensures the was no tie with the
original context. Moreover, it proves it works in distributed caches where the
objects aren’t stored in memory, but always serialized-deserialized. The test
proves in such caching scenario, update works fine.

Now the moment you have been waiting for. This particular scenario required
me to change the Update method completely and choose a completely different
approach for updating entities.

Say you have to change one of the foreign keys. The way to do this in EF is
to change the referenced entity to a new stub. Something like this:

If you try this, you get no exception, but the changed foreign key does not
update. EF cannot see the changes made in the referenced entity. I thought I
will have to change the WidgetReference as well to reflect the change
made in the Widget property. Something like this:

With this code, and doing the WidgetReference thing, the test passes.
Remember, with this update code and not changing the WidgetReference
would result in an exception:

failed: System.Data.UpdateException : A relationship is being added or deleted from an AssociationSet 'FK_WidgetInstance_Widget'. With cardinality constraints, a corresponding 'WidgetInstance' must also be added or deleted.
at System.Data.Mapping.Update.Internal.UpdateTranslator.RelationshipConstraintValidator.ValidateConstraints()
at System.Data.Mapping.Update.Internal.UpdateTranslator.ProduceCommands()
at System.Data.Mapping.Update.Internal.UpdateTranslator.Update(IEntityStateManager stateManager, IEntityAdapter adapter)
at System.Data.EntityClient.EntityAdapter.Update(IEntityStateManager entityCache)
at System.Data.Objects.ObjectContext.SaveChanges(Boolean acceptChangesDuringSave)
DropthingsEntities2.cs(278,0): at EntityFrameworkTest.DropthingsEntities2.Update[TEntity](TEntity entity)
Program.cs(646,0): at EntityFrameworkTest.Program.Changing_referenced_entity_should_work_just_like_updating_regular_entity()

So, you have to use this new Update<> method, as well as do the
WidgetReference trick.

Deleting connected and disconnected entities

Even a simple delete does not work as you expect in EF. You have to handle
several scenarios to make it work properly in common use cases. First is the
disconnected delete. Say you loaded an entity from one context and then you want
to delete it from another context. Something like this:

100% Unit testable ObjectContext

Making DataContext in Linq to SQL unit testable was hard. You had to extend
the generated DataContext and add a lot of stuff. EF is not easier in anyway.
Here’s how I made a fully unit testable ObjectContext, which contains all the
code I have shown above and some more – the complete class.

You can take this class as it is. The only thing you need to change is inside
the constructor set all the entities to MergeOption = NoTracking. Then you need
to create the AttachTo and AddTo mapping for every entity you have in your
ObjectContext. This is the only hard part.

Here’s an example of unit testing a repository which is using the
extended ObjectContext:

As you can see I can stub out all Query calls using Moq. Similarly I can stub out Insert,
Update, Delete as well and thus produce unit test for the repositories.

Conclusion

Entity Framework is hard to use in a disconnected environment. Microsoft did
not make it any easier than Linq to SQL. Just like Linq to SQL, it requires a
lot of hack to make it work properly in an environment where database operations
happen on a mix of connected and disconnected entities. Since you have to do
hack on Linq to SQL anyways, I recommend going towards Entity Framework at the
expense of these new hacks and fundamentally different way of dealing with
entities, since most of the innovations are happing on the EF area nowadays.

Comments and Discussions

I was stuck with this problem for several hours until i found your solution. My model is full of independent associations and to make it worse, i was using disconnected entities. Thank you very much.
I made some minor modifications to make it run on EF4, need to create a new entitykey instance for it to update the store.

Please don't get the wrong idea from my comment, I think this is an excellent article, but some of the things you mention in your opening paragraph are misleading to say the least.

I think the key here is that by following an established pattern that is suited to your ORM you can do some very good things in terms of SoC, Test Coverage and loose coupling. It's always possible to create a huge mess if you go about it the wrong way, regardless of how good the ORM is.

Your Article is really helpful to understand the Entity Framework with Tier Architecture but When i tried to download the source code.. I am getting three files which i imported in my class library project where i have MyDatabase.edmx file is..

I tried to change the inheritance from DropthingsEntities to MyDatabase. but i am getting an error at aspnet_Applications & all those properties which you have defined in constructor.

Can you/ anyone guide me how can i resolve that and able to user this class project with available code ?

First of all EF solves a huge number of issues and is simple! You're introduction doesn't indicate that. For a new person this article is scary! It pushes people away from EF! So after reading the introduction, I didn't want to go any further mostly because you have no subheaders in the article listing each of the problems and the solutions you found! You know something like PROBLEM 1 Can't do this with EF using N-Tier. Then a discussion on why and the solutino would be in order.

I say this because I was looking for articles to point my developers to to learn EF! This popped up and I am NOT going to point my devs to this one for obvious reasons!

when i was loading your "Download EntityFrameworkTest.zip " using vs2010, it was mistake of some entity, base on the entity "DropthingsEntities " is NULL. please give me this class file "DropthingsEntities", and send email "cloudcountry@163.com".

when i was loading your "Download EntityFrameworkTest.zip " using vs2010, it was mistake of some entity, base on the entity "DropthingsEntities " is NULL. please give me this class file "DropthingsEntities", and send email "cloudcountry@163.com".

Hi, this post shows really great approach, but when I started using the Dropthings "databasehelper" the old one I was a bit Annoyed of writing Attached functions for each entity. That's why I tied the following:
private static T Clone<T>(T objsrc, T objdest)
{
PropertyInfo[] propertyInfos = (typeof(T).GetProperties());

foreach (PropertyInfo pInfo in propertyInfos)
{
bool isAssosiation = false;
foreach (object a in pInfo.GetCustomAttributes(true))
{
// Returns if the property is in Assosiation
if (a.GetType() == typeof(System.Data.Linq.Mapping.AssociationAttribute))
{
isAssosiation = true;
break;
}
}
// Set the Properties source value
if (isAssosiation == false && pInfo.Name != "CreateDate")
{
pInfo.SetValue(objdest, pInfo.GetValue(objsrc, null), null);
}
}
return objdest;
}
Here I get a new instance of the entity while Update() and copy the updated values of the original (Disconnected) entity to the new one.
Though its hitting the DB but , I thing this was usable. Please Suggest what do u think about this approach.

Hi! this is an excellent article. I have the issue you described when trying to update references.
However, I have a question about your solution: the TryGetObjectByKey method hits the DB to retrieve the original values of the entity, doesn't it?
Again, thanks for your article it was really helpful. I will try to adapt some of your hacks to the EFPocoAdapter

You have confirm all my work since last year. I knew EF will be the future persistance layer so when i started an important proyect for my company and i decided for it ( with all the bugs and issues ) . I did all you describe in this post and some other things with EF in vs 2008 sp1 for a n tier enterprise application for Chemical Laboratories.

Thank you very much again!

By the way...what do you think about Telerik ORM? here is a Comparative Table:link

We use all telerik tools but i need to know your opinion about this one

In my MVC application I have two linked One-2-Many relationship in this way:

Entity 1 --- * ASet 1 --- * BSet

I use Lazy Loading and have loaded ASet data. I want to load BSet data for a giving ASet item on demand and add these BSet items to my disconnected model in order to update all my model when user save changes in my root entity.

I have not had time to study the latest version of Dropthings yet (I always learn a lot from studying your code), and since I have decided to use EF4 for my next project, this article comes in very handy for me.

There's a saying going like "In the kingdom of the blind the one-eyed is the king.".