Building a Simple Virtual DOM from Scratch

I gave a live-coding talk last week at the Manchester Web Meetup #4. I built a virtual DOM from scratch in less than an hour during the talk. It was the most technically complicated talk that I have ever given by far.

The video of my talk is uploaded here. This post is basically a typed-out version of my talk and aim to clarify extra bits that I haven't had time to mention in the talk. I would recommend watching the video before reading this. It would make things a little bit easier to follow.

Background: What is Virtual DOM?

The Document Object Model (DOM) is a programming interface for HTML documents.

For example, when you do this:

const$app=document.getElementById('app');

You will get the DOM for <div id="app"></div> on the page. This DOM will have some programming interface for you to control it. For example:

$app.innerHTML='Hello world';

To make a plain object to represent $app, we can write something like this:

constvApp={tagName:'div',attrs:{id:'app',},};

Didn't mention in the talk

There is no strict rule of how the virtual DOM should look like. You can call it tagLabel instead of tagName, or props instead of attrs. As soon as it represents the DOM, it is a "virtual DOM".

Virtual DOM will not have any of those programming interface. This is what makes them lightweight comparing to actual DOMs.

However, keep in mind that since DOMs are the fundamental elements of the browser, most browsers must have done some serious optimisation to them. So actual DOMs might not be as slow as many people claim.

Didn't mention in the talk

Object literals (e.g. { a: 3 }) automatically inherit from Object. This means that the object created by object literals will have methods defined in the Object.prototype like hasOwnProperty, toString, etc.

We could make our virtual DOM a little bit "purer" by using Object.create(null). This will create a truly plain object that doesn't inherit from Object but null instead.

render (vNode)

Rendering virtual elements

Now we got a function that generates virtual DOM for us. Next we need a way to translate our virtual DOM to real DOM. Let's define render (vNode) which will take in a virtual node and return the corresponding DOM.

src/vdom/render.js

constrender=(vNode)=>{// create the element// e.g. <div></div>const$el=document.createElement(vNode.tagName);// add all attributs as specified in vNode.attrs// e.g. <div id="app"></div>for(const[k,v]ofObject.entries(vNode.attrs)){$el.setAttribute(k,v);}// append all children as specified in vNode.children// e.g. <div id="app"><img></div>for(constchildofvNode.children){$el.appendChild(render(child));}return$el;};exportdefaultrender;

The above code should be quite self-explanatory. I am more than happy to explain more tho if there is any request for it.

ElementNode and TextNode

In real DOM, there are 8 types of nodes. In this article, we will only look at two types:

ElementNode, such as <div> and <img>

TextNode, plain texts

Our virtual element structure, { tagName, attrs, children }, only represents the ElementNode in the DOM. So we need some representation for the TextNode as well. We will simply use String to represent TextNode.

Extending render to support TextNode

As I mentioned, we are considering two types of nodes. The current render (vNode) only only renders ElementNode. So let's extend render so that it supports rendering of TextNode too.

We will first rename our existing function renderElem as it is what it does. I will also add object destructuring to make the code looks nicer.

src/vdom/render.js

constrenderElem=({tagName,attrs,children})=>{// create the element// e.g. <div></div>const$el=document.createElement(tagName);// add all attributs as specified in vNode.attrs// e.g. <div id="app"></div>for(const[k,v]ofObject.entries(attrs)){$el.setAttribute(k,v);}// append all children as specified in vNode.children// e.g. <div id="app"><img></div>for(constchildofchildren){$el.appendChild(render(child));}return$el;};exportdefaultrender;

Let's redefine render (vNode). We just need to check if vNode is a String. If it is then we can use document.createTextNode(string) to render the textNode. Otherwise, just call renderElem(vNode).

src/vdom/render.js

constrenderElem=({tagName,attrs,children})=>{// create the element// e.g. <div></div>const$el=document.createElement(tagName);// add all attributs as specified in vNode.attrs// e.g. <div id="app"></div>for(const[k,v]ofObject.entries(attrs)){$el.setAttribute(k,v);}// append all children as specified in vNode.children// e.g. <div id="app"><img></div>for(constchildofchildren){$el.appendChild(render(child));}return$el;};constrender=(vNode)=>{if(typeofvNode==='string'){returndocument.createTextNode(vNode);}// we assume everything else to be a virtual elementreturnrenderElem(vNode);};exportdefaultrender;

Now our render (vNode) function is capable of rendering two types of virtual nodes:

Let's make our app more interesting

Now let's make our app more interesting. We will wrap our vApp in a function called createVApp. It will then take in a count which then the vApp will use it.

src/main.js

importcreateElementfrom'./vdom/createElement';importrenderfrom'./vdom/render';importmountfrom'./vdom/mount';constcreateVApp=count=>createElement('div',{attrs:{id:'app',dataCount:count,// we use the count here},children:['The current count is: ',String(count),// and herecreateElement('img',{attrs:{src:'https://media.giphy.com/media/cuPm4p4pClZVC/giphy.gif',},}),],});letcount=0;constvApp=createVApp(count);const$app=render(vApp);mount($app,document.getElementById('app'));

Then, we will setInterval to increment the count every second and create, render and mount our app again on the page.

src/main.js

importcreateElementfrom'./vdom/createElement';importrenderfrom'./vdom/render';importmountfrom'./vdom/mount';constcreateVApp=count=>createElement('div',{attrs:{id:'app',dataCount:count,// we use the count here},children:['The current count is: ',String(count),// and herecreateElement('img',{attrs:{src:'https://media.giphy.com/media/cuPm4p4pClZVC/giphy.gif',},}),],});letcount=0;constvApp=createVApp(count);const$app=render(vApp);let$rootEl=mount($app,document.getElementById('app'));setInterval(()=>{count++;$rootEl=mount(render(createVApp(count)),$rootEl);},1000);

Note that I used $rootEl to keep track of the root element. So that mount knows where to mount our new app.

If we go back to the browser now, we should see the count increment every second by 1 and works perfectly!

We now gain the power to declaratively create our application. The application is rendered predictably and is very very easy to reason about. If you know how things are done in the JQuery way, you will appreciate how much cleaner this approach is.

However, there are a couple of problems with re-rendering the whole application every second:

Real DOM are much heavier than virtual DOM. Rendering the whole application to real DOM can be expensive.

Elements will lose their states. For example, <input> will lose their focus whenever the application re-mount to the page. See live demo here.

We will solve these problems in the next section.

diff (oldVTree, newVTree)

Imagine we have a function diff (oldVTree, newVTree) which calculate the differences between the two virtual trees; return a patch function that takes in the real DOM of oldVTree and perform appropriate operations to the real DOM to make the real DOM looks like the newVTree.

If we have that diff function, then we could just re-write our interval to become:

src/main.js

importcreateElementfrom'./vdom/createElement';importrenderfrom'./vdom/render';importmountfrom'./vdom/mount';importdifffrom'./vdom/diff';constcreateVApp=count=>createElement('div',{attrs:{id:'app',dataCount:count,// we use the count here},children:['The current count is: ',String(count),// and herecreateElement('img',{attrs:{src:'https://media.giphy.com/media/cuPm4p4pClZVC/giphy.gif',},}),],});letcount=0;letvApp=createVApp(count);const$app=render(vApp);let$rootEl=mount($app,document.getElementById('app'));setInterval(()=>{count++;constvNewApp=createVApp(count)constpatch=diff(vApp,vNewApp);// we might replace the whole $rootEl,// so we want the patch will return the new $rootEl$rootEl=patch($rootEl);vApp=vNewApp;},1000);

So let's try to implement this diff (oldVTree, newVTree). Let's start with some easy cases:

newVTree is undefined

we can simply remove the $node passing into the patch then!

They are both TextNode (string)

If they are the same string, then do nothing.

If they are not, replace $node with render(newVTree).

One of the tree is TextNode, the other one is ElementNode

In that case they are obviously not the same thing, then we will replace $node with render(newVTree).

oldVTree.tagName !== newVTree.tagName

we assume that in this case, the old and new trees are totally different.

instead of trying to find the differences between two trees, we will just replace the $node with render(newVTree).

importrenderfrom'./render';constdiff=(oldVTree,newVTree)=>{// let's assume oldVTree is not undefined!if(newVTree===undefined){return$node=>{$node.remove();// the patch should return the new root node.// since there is none in this case,// we will just return undefined.returnundefined;}}if(typeofoldVTree==='string'||typeofnewVTree==='string'){if(oldVTree!==newVTree){// could be 2 cases:// 1. both trees are string and they have different values// 2. one of the trees is text node and// the other one is elem node// Either case, we will just render(newVTree)!return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}else{// this means that both trees are string// and they have the same valuesreturn$node=>$node;}}if(oldVTree.tagName!==newVTree.tagName){// we assume that they are totally different and // will not attempt to find the differences.// simply render the newVTree and mount it.return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}// (A)};exportdefaultdiff;

If the code reaches (A), it implies the following:

oldVTree and newVTree are both virtual elements.

They have the same tagName.

They might have different attrs and children.

We will implement two functions to deal with the attributes and children separately, namely diffAttrs (oldAttrs, newAttrs) and diffChildren (oldVChildren, newVChildren), which will return a patch separately. As we know at this point we are not going to replace $node, we can safely return $node after applying both patches.

src/vdom/diff.js

importrenderfrom'./render';constdiffAttrs=(oldAttrs,newAttrs)=>{return$node=>{return$node;};};constdiffChildren=(oldVChildren,newVChildren)=>{return$node=>{return$node;};};constdiff=(oldVTree,newVTree)=>{// let's assume oldVTree is not undefined!if(newVTree===undefined){return$node=>{$node.remove();// the patch should return the new root node.// since there is none in this case,// we will just return undefined.returnundefined;}}if(typeofoldVTree==='string'||typeofnewVTree==='string'){if(oldVTree!==newVTree){// could be 2 cases:// 1. both trees are string and they have different values// 2. one of the trees is text node and// the other one is elem node// Either case, we will just render(newVTree)!return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}else{// this means that both trees are string// and they have the same valuesreturn$node=>$node;}}if(oldVTree.tagName!==newVTree.tagName){// we assume that they are totally different and // will not attempt to find the differences.// simply render the newVTree and mount it.return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}constpatchAttrs=diffAttrs(oldVTree.attrs,newVTree.attrs);constpatchChildren=diffChildren(oldVTree.children,newVTree.children);return$node=>{patchAttrs($node);patchChildren($node);return$node;};};exportdefaultdiff;

diffAttrs (oldAttrs, newAttrs)

Let's first focus on the diffAttrs. It is actually pretty easy. We know that we are going to set everything in newAttrs. After setting them, we just need to go through all the keys in oldAttrs and make sure they all exist in newAttrs too. If not, remove them.

Notice how we create a wrapper patch and loop through the patches to apply them.

diffChildren (oldVChildren, newVChildren)

Children would be a little bit more complicated. We can consider three cases:

oldVChildren.length === newVChildren.length

we can do diff(oldVChildren[i], newVChildren[i])
where i goes from 0 to oldVChildren.length.

oldVChildren.length > newVChildren.length

we can also do diff(oldVChildren[i], newVChildren[i])
where i goes from 0 to oldVChildren.length.

newVChildren[j] will be undefined for j >= newVChildren.length

But this is fine, because our diff can handle diff(vNode, undefined)!

oldVChildren.length < newVChildren.length

we can also do diff(oldVChildren[i], newVChildren[i])
where i goes from 0 to oldVChildren.length.

this loop will create patches for each already existing children

we just need to create the remaining additional children i.e. newVChildren.slice(oldVChildren.length).

To conclude, we loop through oldVChildren regardless and we will call diff(oldVChildren[i], newVChildren[i]).

Then we will render the additional children (if any), and append them to the $node.

constdiffChildren=(oldVChildren,newVChildren)=>{constchildPatches=[];oldVChildren.forEach((oldVChild,i)=>{childPatches.push(diff(oldVChild,newVChildren[i]));});constadditionalPatches=[];for(constadditionalVChildofnewVChildren.slice(oldVChildren.length)){additionalPatches.push($node=>{$node.appendChild(render(newVChildren));return$node;});}return$parent=>{// since childPatches are expecting the $child, not $parent,// we cannot just loop through them and call patch($parent)$parent.childNodes.forEach(($child,i)=>{childPatches[i]($child);});for(constpatchofadditionalPatches){patch($parent);}return$parent;};};

I think it is a little bit more elegant if we use the zip function.

importrenderfrom'./render';constzip=(xs,ys)=>{constzipped=[];for(leti=0;i<Math.min(xs.length,ys.length);i++){zipped.push([xs[i],ys[i]]);}returnzipped;};constdiffChildren=(oldVChildren,newVChildren)=>{constchildPatches=[];oldVChildren.forEach((oldVChild,i)=>{childPatches.push(diff(oldVChild,newVChildren[i]));});constadditionalPatches=[];for(constadditionalVChildofnewVChildren.slice(oldVChildren.length)){additionalPatches.push($node=>{$node.appendChild(render(additionalVChild));return$node;});}return$parent=>{// since childPatches are expecting the $child, not $parent,// we cannot just loop through them and call patch($parent)for(const[patch,$child]ofzip(childPatches,$parent.childNodes)){patch($child);}for(constpatchofadditionalPatches){patch($parent);}return$parent;};};

Finalised diff.js

src/vdom/diff.js

importrenderfrom'./render';constzip=(xs,ys)=>{constzipped=[];for(leti=0;i<Math.min(xs.length,ys.length);i++){zipped.push([xs[i],ys[i]]);}returnzipped;};constdiffAttrs=(oldAttrs,newAttrs)=>{constpatches=[];// setting newAttrsfor(const[k,v]ofObject.entries(newAttrs)){patches.push($node=>{$node.setAttribute(k,v);return$node;});}// removing attrsfor(constkinoldAttrs){if(!(kinnewAttrs)){patches.push($node=>{$node.removeAttribute(k);return$node;});}}return$node=>{for(constpatchofpatches){patch($node);}return$node;};};constdiffChildren=(oldVChildren,newVChildren)=>{constchildPatches=[];oldVChildren.forEach((oldVChild,i)=>{childPatches.push(diff(oldVChild,newVChildren[i]));});constadditionalPatches=[];for(constadditionalVChildofnewVChildren.slice(oldVChildren.length)){additionalPatches.push($node=>{$node.appendChild(render(additionalVChild));return$node;});}return$parent=>{// since childPatches are expecting the $child, not $parent,// we cannot just loop through them and call patch($parent)for(const[patch,$child]ofzip(childPatches,$parent.childNodes)){patch($child);}for(constpatchofadditionalPatches){patch($parent);}return$parent;};};constdiff=(oldVTree,newVTree)=>{// let's assume oldVTree is not undefined!if(newVTree===undefined){return$node=>{$node.remove();// the patch should return the new root node.// since there is none in this case,// we will just return undefined.returnundefined;}}if(typeofoldVTree==='string'||typeofnewVTree==='string'){if(oldVTree!==newVTree){// could be 2 cases:// 1. both trees are string and they have different values// 2. one of the trees is text node and// the other one is elem node// Either case, we will just render(newVTree)!return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}else{// this means that both trees are string// and they have the same valuesreturn$node=>$node;}}if(oldVTree.tagName!==newVTree.tagName){// we assume that they are totally different and // will not attempt to find the differences.// simply render the newVTree and mount it.return$node=>{const$newNode=render(newVTree);$node.replaceWith($newNode);return$newNode;};}constpatchAttrs=diffAttrs(oldVTree.attrs,newVTree.attrs);constpatchChildren=diffChildren(oldVTree.children,newVTree.children);return$node=>{patchAttrs($node);patchChildren($node);return$node;};};exportdefaultdiff;

Make our app more complicated

Our current app doesn't really make full use of the power of our virtual DOM. To show how powerful our Virtual DOM is, let's make our app more complicated:

src/main.js

importcreateElementfrom'./vdom/createElement';importrenderfrom'./vdom/render';importmountfrom'./vdom/mount';importdifffrom'./vdom/diff';constcreateVApp=count=>createElement('div',{attrs:{id:'app',dataCount:count,// we use the count here},children:['The current count is: ',String(count),// and here...Array.from({length:count},()=>createElement('img',{attrs:{src:'https://media.giphy.com/media/cuPm4p4pClZVC/giphy.gif',},})),],});letvApp=createVApp(0);const$app=render(vApp);let$rootEl=mount($app,document.getElementById('app'));setInterval(()=>{constn=Math.floor(Math.random()*10);constvNewApp=createVApp(n);constpatch=diff(vApp,vNewApp);// we might replace the whole $rootEl,// so we want the patch will return the new $rootEl$rootEl=patch($rootEl);vApp=vNewApp;},1000);

Our app now will generate a random number n between 0 and 9 and display n cat photos on the page. If you go into the dev tools, you will see how we are "intelligently" inserting and removing <img> depending on n.

Thank you

If you read all the way up to here, I would like to thank you for taking the time to read the whole thing. It is a very very long read! Please leave a comment if you actually read the whole thing. Love you!

This is the kind of article that makes me feel so Lucky to be alive at this moment and be part of this code side of the world which is filled with extremely passionate and knowledgeable people who really make efforts to share their knowledge with others.

It's because of engineers like you that it's actually approachable to not only learn to code, but learn to make amazing things.

Forgive me for saying so much, I just happen to really appreciate this, even if I'm not thinking of building a Vdom right now.. Knowing how things work behind the wheel feels very good.

Jason, I'm out of words to express my admiration for this tutorial! You've taken a modern concept that is on fire and considered out of reach of mere mortals, and made it accessible. I used to think that the virtual DOM was some kind of evil magic I'd never understand, but now I can confidently say that I do. That said, this was a really heavy read and tripped me up several times, but I'm really glad I pushed through. I do have a question (or two) bothering me; can I post it as a separate comment here?

Cool! So, I see the use of recursion in the code (diff() calls diffChildren(), which calls diff()), and once I started looking for the worst case, my head started to spin. Now, the logic reaches the point you marked (A) when the tag names are same, which makes me wonder what if we're stuck with a structure that's all same-name tags all the way? Let's say something like this:

So the first time we call diff(), it will call diffChildren(), which will call diff(), which will call diffChildren() down the chain, and so on. At this point, my brain circuits get fried and I can't figure out where this is going. 😛

Sure. You have stepped into the common trap of thinking recursion recursively. Your brain will stack-overflow first before the computer does haha.

The reason why you are so confused is because you are lacking the "leap of faith" for recursion. You try to figure out what is happening, then you look into the function; it calls itself, and u look into the function again... Then you are lost.

All you need is faith!

The first thing is to define what diff and diffChildren do. I made it very clear for diff.

Imagine we have a function diff (oldVTree, newVTree) which calculate the differences between the two virtual trees; return a patch($tree) function that takes in the real DOM of oldVTree and perform appropriate operations to the real DOM to make the real DOM looks like newVTree.

So the idea is, you know diff will somehow call itself again at some point. And when this happens, all you need is the "leap of faith"; believe that diff will work as you wish! Don't look into diff again to try to figure things out! Just think about what diff should do and return. By our definition, it will return a patch! So just assume the recursive calls to diff will work and read on.

Teaching recursion using this example is a bit hard, have a look at this article which I explained very clearly how you could obtain faith in recursion.

The leap of faith takes some practice to get use to it. If you want some exercise, I am happy to give you some challenge and guide you through them. Feel free to DM me on twitter: @ycmjason

Oops, I just realised there is one more, which is when there is no children in the node. But I didn't explicitly deal with that case as it will be automatically dealt with in the for loop in diffChildren

That's because the (I forget the name of the linked example . . . flatten()?) the second example is just a single function calling itself, whereas in the vDOM case, we have diff calling diffChildren calling diff, and so on, so it's harder to reason about.

This means that all the return statement before the last return can be considered as base case.

I see that, and what I did this time was write out a two-depth example and trace it by hand. My head still went for a spin, but I kind of see how this is supposed to work. I think I'll leave it at that for now. 😇

Now, on to a really important question: did you write this code simply based on the "leap of faith"? Is this how algorithms like merge sort and quick were written? On leaps of faith? I'm having a really hard time believing that! Is leap of faith good enough for serious professional/interview problems? 🤔

All said and done, this exercise has reminded me how badly I need to do a course in algorithms, which I'm going to do over the next few days. So, for that, too, thanks a lot! 😇😇

Yes. Totally based on the leap of faith. It always work! It's the very important thing you need when dealing with recursion.

Is this how algorithms like merge sort and quick were written?

Well, merge sort and quick sort if written in a recursive way, can be reasoned about using the "leap of faith" for sure. Whether or not the original Author has the leap of faith there is noway to find out. 😂😂

Is leap of faith good enough for serious professional/interview problems?

Leap of faith will definitely work in professional and interview problems. It's just a mindset you should have when writing recursive solutions, not really a method. Once you do more recursion, you will become confident enough to hold that faith all time.

Couldn’t agree more with this comment. I looked into Reacts vdom a while back to better understand and I just got lost and lost. This post was first time I’d seen someone breakdown concisely and in a way that doesn’t overwhelm the reader with “frivolous” complexities

Hey, really great write up. I missed the talk at Manchester Web Meetup, but wanted to see this talk, when I first learned React a few years I looked at how the Virtual DOM was put together and did a similar createElement/mount function pair, but I think the real power on the Virtual DOM, especially when used in React is the diffing mechanism. I didnt attempt that at the time, but I think what you have put above is great because I think it helps show why/how frameworks like React update themselves in reaction to changes and you do it with understandable and clean code, awesome :D

I'm the same, I think there is a lot to be gained from clean code, not just for yourself, but for other developers that do (or will) work with it too :) Doing it in example/tutorial code even more so as it improves the value and quality of the article, like you have here :)

Inside the function that we put into additionalPatches we should render not the newVChildren but the additionalVChild.
This code appears in the first example in the explanation of diffChildren function.