DaST Concept: A simpler, smarter, and much more powerful alternative to Forms and MVC.

Pure HTML templates and uniform codebehind is the future of web development. The article presents ASP.NET DaST Rendering Engine and its underlying DaST concept that brings web apps architecture to a new level getting rid of all problems and complexity related to standard ASP.NET Forms and MVC.

Abstract

DaST pattern and ASP.NET DaST Rendering Engine is a further development of the ASP.NET
Scopes Framework and its underlying server page technology based on fully templated
data scope trees which I proposed in my
first article in November 2010. From now on the pattern will be called DaST
which stands for Data Scope Trees. After working with DaST for a while, my opinion
is that this approach opens unmatched web development opportunities and has all
chances to completely replace standard ASP.NET Forms and MVC patterns in the near
future.

This article consists of 4 parts. First of all, I'll present a new VideoLibrary
DEMO application which demonstrates full power of DaST pattern on a real web site.
This application will serve as a good how-to sample and a DaST technology demonstrator
for all further releases of ASP.NET DaST framework. In the second part I'll give
a brief overview of the DaST concept itself and will decompose a VideoLibrary page
to a scope tree, so that those who see this technology for the first time could
get an idea about the new DaST approach and its advantages. In the third section I'll
take a closer look at the demo application back-end architecture and implementation
and will highlight all important syntax and design changes that took place since
1st Alpha release in November. This section is targeted only for those, who read
my previous article.
And in the final section I'll share some framework development plans for the near
future.

One more thing :) As many of you criticized my previous article for its length,
I'll keep this one short, lean and mean. But if you come across this framework for
the first time and you wish to try it in your projects, current article will not
be enough and you'll have to read my more detailed and deep
Scopes Framework article from November.

Currently, all DaST releated news and events are posted on the
DaST development site. Check this site
for framework updates and important changes. The latest version of ASP.NET DaST
Rendering Engine as well as the VideoLibrary DEMO source code are located in
the download section of the site.

!!!UPDATE (May 11, 2011): ASP.NET DaST Rendering Engine project has officially become open-source starting from May 9th!
We're looking for open-source developers, so if you feel you can contribute the project, you're welcome to contact the DaST team. DaST
open-source information site is here. DaST project console home on SourceForge is here. Public DaST discussion forums are here (anynimous
access is enabled)

VideoLibrary DEMO - Technology Demonstrator

To demonstrate advantages of DaST web development pattern over the standard Forms and MVC approaches,
I created a sample application, VideoLibrary, which will serve us as an example, tutorial, and technology
demonstrator in the same time. VideoLibrary DEMO is a prototype of a video portal built on ASP.NET DaST
framework. Having complex and dynamic UI, this sample application includes most of typical functional elements
used to build rich Web 2.0 sites and demonstrates how these functionalities could be achieved using DaST
Rendering Engine. Here are some nice and very popular Web 2.0 features that are implemented in VideoLibrary:

The following are some application screenshots (Fig. 1 and Fig. 2) and brief description
of how the application operates on a user level. Although the application does not
do much meaningful things by itself and serves mostly as demonstrator, it can be
used as a prototype for a similar applications in a real world. Note that Fig. 1
and Fig. 2 show the application in 2 different jQuery UI themes.

Fig. 1: VideoLibrary DEMO initial look

The application displays paged selection of a number of video packshots (see Fig. 1).
Of course, there are no real videos -- we just use black rectangle with colored
text on it. Color of that text defines the category of the video: red category,
green category etc. Each packshot has "+" button in the top-right corner. Clicking
on it, the video is added to the playlist on the left side and button turns to "-".
Clicking "-" removes video from the playlist. On mouse hover, the packshot displays
"i" button which invokes the details dialog window.

Fig. 2: VideoLibrary DEMO details dialog

The dialog window (see Fig. 2) displays some more details and paged list of video
packshots in the same category (i.e. with the same color). Packshots in this list
work the same way as in the playlist and the selection grid. The dialog window is
tabbed. You can switch to another tab to add video reviews and ratings. Note that
all changes you make are saved within your current session only. You can also play
with application look-and-feel by selecting your favorite jQuery UI theme in the
top-left corner.

All application data comes from data layer simulated by XML file and LINQ. Since
this application is a DEMO, I've made it so that all changes you make (add reviews,
ratings, playlist, etc) are persisted within the current session only and are
discarded when the session ends. On Fig. 3 you can see a part of XML file used as
a data source. It's structure is trivial and you can get an idea how VideoLibrary
data is organized.

Fig. 3: Back-end data XML

Although, beautiful and attractive UI elements is an important part of the application,
all the advantages of DaST web development pattern become obvious only when we look
at the application back-end code. Being clear, transparent and uniform, the DaST
architecture of the VideoLibrary back-end results in unmatched front-end flexibility
and allows almost unlimited complexity of the web application UI.

If you wish to run the live test of VideoLibrary DEMO application immediately, you
can do this on the DaST Development Site
where I'll always deploy the latest version of this application. The entire VideoLibrary
DEMO source code is available for download as attachment to this article. I suggest
that you get the source code, browse and get familiar with it, as for the rest
of this article I'll use this sample code for all my explanations.

DaST Concept Overview

Do you think it is possible to create an ultimate server page rendering engine that
outperforms standard Forms and MVC in easiness, architecture, flexibility, performance,
presentation separation, and all other important web development attributes? As
the answer to this question I'd like to present the ASP.NET DaST Rendering Engine
based on a new DaST web development pattern. The name of the pattern stands for
Data Scope Trees and you'll understand what this means later. Even if you're an
experienced web application programmer or architect, and not familiar with this
pattern yet, I promise that the rest of this article will change your idea about
the server-page based web application design and development.

The entire DaST concept departs from a very simple thought that every web site in the
world is nothing, but a bunch of data produced by server-side logic and presented
to the user in the client browser. All you see on a web site is just some data wrapped
into HTML text! This means that the entire page rendering process can be viewed
as two separate steps: 1) generation of some data values and 2) presentation of
these data values in the form of HTML to the client.

To accomplish data generation on the first step we need some kind of codebehind
class. Speaking abstractly, the output of such class would be a set of string values.
In DaST framework this class is called a controller (like in MVC). The second rendering
step is more interesting. After all string values are generated by the controller,
we need to mix them with some markup and give the HTML output to the client. Sounds
simple, but how to accomplish this technically? Every experienced developer knows
how standard Forms or MVC platforms render their pages by going though the complicated
and resource consuming page lifecycle, rendering multiple individual server controls,
applying master pages, calling data binding implementations for repeating controls,
processing events, etc. etc. This process is quite complex and we will try to approach
the rendering task from another side. In order to render our prepared data into
HTML, we will just take a complete and valid HTML template and insert the prepared
string values into this template at the right spots. The spots where data values
get inserted are marked with special placeholders which are replaced on rendering
stage by real data values. And this is it! We just need a pure HTML template and
nothing else -- no server controls, no page lifecycle, no data binding -- we just
dumped all that complexity at once!

Look at our demo application on Fig. 1. Now imagine if we had a list of prepared
data values (like all those names, descriptions, page numbers, etc) and a complete
HTML template with placeholders at the locations where values are to be inserted.
What would our rendering process be? Well, the entire rendering would be reduced
to a trivial search-and-replace operation for substituting placeholders by the real
values. And this is basically all the DaST does to render your pages!

Data Scope Trees

Of course, I abstract away some technical details by just saying that a bunch of
data gets applied to the template. In reality, this process has to be more granular,
because we also need some way to manage our data values so that they get applied
at the right location only. Having placeholders is not enough, because we simply
don't want to produce all data values at once and, moreover, we may not be able to.
It's much better to, first, produce values for one area of the page, then for another,
and so on. Plus we need some way to provide data for repeated content that can be
nested in another repeater at any level. And what about Ajax and partial page updates?
Thinking about all these requirements I finally came to the idea of the data scopes
and data scope trees which became the core of the DaST pattern (hence, the pattern
name) and the framework based on it.

Data scope is simply a group of cohesive data values. Cohesion criteria is chosen
by the developer. Data scope can be applied to the specific area of an HTML template,
meaning that placeholders in that area are replaced by the corresponding data values
from data scope. As a result, we get the final HTML output for this specific piece
of the template. Extending this technique for the entire page, we can have multiple
data scopes applied to all of the areas of HTML template to render the
template
completely and output the page to the client. Data scopes can also be nested to
form hierarchical structure called data scope tree. And finally, I assert the
every
page of any complexity can be viewed as an HTML template with a data scope tree
applied to it. Voila! :)

So, first thing that a web developer must accomplish is to decompose the application
UI to a set of randomly nested data scopes. From my own experience, choosing the
right data scope nesting structure requires some DaST specific skills that come
after several exercises. Let's do this for a part of our demo application UI shown
on Fig. 1. The resulted data scopes are shown on Fig. 4.

Fig. 4: VideoLibrary partial UI decomposition into data scopes

First of all, there is always a rootscope (NULLscope) that serve as a parent of
all other scopes. Then we have PageSizeRepeater and SiteThemeRepeater to output
items for page size and for jQuery theme drop-down lists (see Fig. 1). Next we have
several other data scopes on the left side to form playlist UI. PlaylistHeader scope
shows current pager info. PlaylistPager scope wraps a playlist item pager.
PlaylistRepeater
is needed to display multiple video packshots represented by a nested VideoItem scope.
Each VideoItem scope has 2 more nested scopes: AddButton and RemoveButton. The idea
is to show only AddButton scope when item is not in the playlist, and only
RemoveButton
scope otherwise. Next, on the right side of the page we have a video item selection
grid. Structure for this grid is absolutely the same as for playlist except that
data scopes on the same level have different names. Now, if we depict our nested
data scopes on Fig. 4 as tree, we will get a data scope tree identifying our page.
A complete data scope tree for the VideoLibrary application is shown on Fig. 5.
Upper part of the tree corresponds to UI on Fig. 1 which we have just decomposed.
The bottom subtree coming from DetailsPopup scope corresponds to UI on Fig. 2.
Data scopes with attached controllers have red
backgrounds and this will be explained later.

Fig. 5: VideoLibrary complete data scope tree

Data Scopes in Template

Ok, after we logically decomposed the UI into data scopes, next question
is how to point the specific data scope at the specific area in the template.
The solution here is straightforward. Since our data scope tree is built
so that it represents the logical structure of the document, we just use the same
structure in the HTML template by wrapping corresponding areas into valid
DIV container elements representing boundaries within which data scopes are
are applied. This means, that these containers in the HTML template have
the same hierarchical structure defined by the data scope tree. Each data
scopeis identified by its name e.g. "PlaylistPager" or "VideoItem". The same
name should be specified in the scope attribute of the corresponding
HTML container element. So, the system does not care how complex your template
is is and what type of markup it has -- it only cares about nested scope DIVs i.e.
the data scope tree inside this template. For example, if we wanted to create
a valid template for data scope tree on Fig. 5, a part of this template could look
like the markup on Listing 6.

NOTE that using only DIVs for scope containers is a limitation of the current version
of DaST Rendering Engine and will go away in one of the next versions of the framework.

Controller Overview and Data Binding

To generate real data for template the developer has to implement a controller class.
The process of generating data for the corresponding template I call data binding.
For every data scope in the tree and the corresponding scope DIV in the
template
the controller contains a data binding function called binding handler. The output
of a binding handler is basically a set of values for a current data scope.

In the simplest case, there is a single controller responsible for generating
data for all data scopes in the scope tree. In the normal case, there are
multiple controllers and each one of them is responsible only for a part of a
scope tree. When a controller is attached to a data scope, it becomes
responsible for generating data for this scope and all child scopes in the
subtree unless those child scopes have their own controllers attached.

Look at Fig. 5 again -- data scopes on the red background are the ones that have
controllers attached. There is always at least one controller attached to a NULL
scope called a root controller. Other controllers are child controllers. Just
like user controls in Forms or partial views in MVC, child controllers in DaST
should be used whenever it is appropriate to factor out a certain common piece
functionality. One of the examples in our demo application is
PagerController
attached to PlaylistPager, VideoItemPager, AlikeVideosPager, and
CommentsPager
data scopes (see Fig. 5).

Rendering Process

Now I will redefine the DaST rendering process in terms of controllers and
binding
handlers. To render the entire page, the framework starts from generating data for
the NULL data scope in the tree by invoking the corresponding binding handler in
the root controller and recursively repeats this operation for all data scopes in
the tree invoking appropriate handlers on responsible controllers.
There is always
a specific and consistent order in which data scopes are processed. In terms of
tree walking algorithms, the framework uses post-order walk with top-to-bottom in-order
traversal. Recalling graphs from high school, it's actually called right-to-left
in order traversal, but because it is more appropriate to depict scope tree horizontally
(like I did on Fig. 5), I call it top-to-bottom. Such traversal order is chosen,
because the actual scope DIV tags in HTML template are encountered exactly in this
order.

So, all the developer needs to do to render a page using DaST Rendering Engine is
to 1) create an HTML template similar to the one on Listing 6 and then 2) implement
a controller for it (or multiple controllers for multiple
templates). Listing 7 shows a high-level outline
for the root controller (VideoLibraryController.cs) of our demo application. This
root controller is the one attached to the NULL scope on Fig. 5.
Binding handlers
producing values for corresponding scopes are on lines 87-193. Names of these handler
functions are chosen by the developer. On line 10 you see the
SetTemplate() function
- that's how we tell the controller which template to use.
SetModel() function
on line 15 is needed to assign binding handlers or child controllers to
data scopes
in the tree. Finally, lines 45-73 are action handlers invoked in response to user
actions (think of these as a counterpart of events).

Actions and AJAX

A huge advantage of data scope trees is that they allow for the most clear
definition of partial page updates and the simplest usage of AJAX capabilities
in web applications comparing to all other existing frameworks. The underlying
idea of DaST Ajax is as simple as everything else in DaST -- every scope in data
scope tree can be individually refreshed at any time during the async postback!
When data scope is refreshed, all its child data scopes in the scope tree are
refreshed too. In terms of data binding this means that the system re-invokes
binding handlers corresponding to the refreshed scopes to regenerate data
values, update parts of the template, and output them back to the client.
Data
scope tree traversal order is kept on the postback, maintaining the consistent
order of binding handler invocation.

Next, async postbacks do not occur randomly, but in response to some events in the
browser i.g. button clicks, timer ticks, etc. DaST Rendering Engine provides a simple
and lightweight mechanism allowing to raise events on the client side, and handle
those events on the postback inside the controller classes. In DaST a counterpart
of event is called action. Each action has a name and a string argument. This is
not a limitation, because string argument can contain just any possible JSON-serializable
object. When action occurs, async postback comes to the controller and appropriate
action handler is executed (see lines 45-73 on Listing 7). Action handler is the place
where we can do some processing and instruct the specific data scopes to refresh.

DaST Pattern Résumé

OK, at this point you should have a clear top-level picture of what the DaST
pattern and DaST Rendering Engine actually are and how they differ from other
patterns and frameworks available to the developers today.
Below I will summarize the most obvious benefits of using DaST pattern in your
web applications.

Maximum possible level of separation between presentation and back-end code.

As the result of the above, outstanding application back-end testability and easiness
of TDD.

Usage of W3C compliant pure HTML templates to control every single
byte of the UI markup.

As the result of the above, unmatched UI flexibility, maintainability, and simplicity
in the same time. Instead of delving into markup of server controls, you work with
trivial HTML!

Ability to change your page layout and look-and-feel entirely in 10 minutes by
playing with HTML markup in the template. Obviously, this DOES NOT require
any site recompilation!

Minimum framework learning period. No need to read 400 page books to start creating
highly dynamic and complex sites -- all you need to know is how to decompose your
UI into i>scope trees and how to implement action and binding handlers
inside the controller classes.

All back-end design becomes simple, clear, and uniform. No matter how complex
your application UI is, no matter how big your data scope tree is, in the
back-end it will always be the same -- just a controller with a list of
action and binding handlers!

No page lifecycle. No server controls. No more wondering how to make the control
render this way or that way -- you HTML template and binding handlers
control your rendering process entirely from the beginning to the end.

Native support of AJAX partial update without single line of code or additional
markup. Just compare to update panels in Forms :)))

Binding handlers are re-invoked on the postback ONLY for the refreshed
part of the page meaning that your code is executed ONLY when it needs to! Again,
compare to Forms, where a page goes through the entire lifecycle executing tons
of unnecessary code.

DaST does not deny any of standard Forms features, but rather compliments them.
Our favorite session management, application cache, authentication classes, etc.
are still available within action and binding handlers inside the
controllers. Standard Forms based pages can coexist with DaST based pages
is one application allowing smooth step-by-step transition from Forms to DaST.

Summarizing all this, and based on my own experience, application development
time decreases by up to 90% (not a joke).

Main purpose of the current article is to give an overview of important changes
in the new version of the framework. If you see this concept for the first time
and want to learn in details how to create HTML templates and implement
controllers, you'll have to read
this article which is much more deep and detailed. Or you can wait until documentation
and tutorial becomes available on DaST dev site at
http://www.rgubarenko.net, but I can't promise any dates.

Important Framework Changes. VideoLibrary Back-End.

In this section I'll quickly go through all the important syntax and design changes
since the previous version of the framework. I'll not delve into much back-end details,
because the entire mechanism has already been described in
my previous article in November, I'll only highlight the differences for
those who read that article and who is already familiar with the Scopes Framework.
So, the following sub-sections is just a list of differences between 2 versions.

1) Binary and namespace

Changes happened to the framework binary and namespace are the following:

Name of Name of the framework DLL is changed to AspNetDaST.dll (located in demo site Bin folder).

All public classes used by the developers are located in AspNetDaST.Web namespace.

2) ScopesManagerControl is gone

In Scopes Framework I used a temporary solution that we had to add a
ScopesManagerControl
to an ASPX page to make it work with the framework. Now this is gone. In the DaST
Rendering Engine we inherit pages from DaSTPage class from
AspNetDaST.Web namespace
and implement only one ProvideController() method. You dont have to add any markup
to the ASPX page -- DaSTPage parent class takes care of it. Listing 8 shows the listing
of VideoLibrary.aspx.cs codebehind class. Everything should be clear: we instanciate
the root controller on line 14 and enable partial updates on line 15. Note that currently,
EnablePartialUpdates must always be set to
TRUE. This is due to the fact that I
did not decide yet if we need to support full postback or not. In Forms the reason
to make async postbacks optional was that UpdatePanel controls add significant complexity
and performance overhead. In DaST, partial refreshes are native and I don't see any
reason to make them optional.

3) Data binding changes

As you may notice on Listing 7, arguments in binding handlers are eliminated. Handler
argument was used to add placeholder replacements for the current scope. From now
on, placeholder replacements are added directly to the scope instance which you
can access using CurrentPath and
ControlPath tree navigators. The meaning of these
rendered scope tree navigators has also slightly changed:

ControlPath used inside action or binding handler always points the
scope to which
current controller is attached.

CurrentPath used inside binding handler points to the currently bound
scope. CurrentPath
used inside action handler points to the scope specified in client-side JavaScript
Action(..) call as target scope.

Just look at Listing 9 that shows one of binding handlers from DetailsPopupController.cs
class. On lines 202-203 instead of calling Replace(..) on the binder object passed
as a parameter like it was before, we call Replace(..) directly on the
scope instance,
using CurrentPath that points to the current ReviewsInfo scope.

Listing 9: Some of binding handlers in DetailsPopupController controller.

At the first look it may seem that data binding syntax became more complex, but
actually this design gives us a huge programming benefit, because now we are able
to databind scopes from inside the binding handlers of other scopes. This is great,
because if a scope is simple and contains just a couple of values, why would I flood
my controller class creating a separate binding handler for it? This new feature
is used on lines 207-208 of DetailsPopupController class (see Listing 9) where we use
navigator to point at another ReviewsInfo2 scope and add same placeholder
replacements to it.

4) Working with scope instance

In the new version of DaST Rendering Engine all framework calls became more uniform.
Now all methods and properties related to manipulation with the specific scope are
accessible through the scope instance object. Data binding functions (some of them
I already talked about in the previous section) is one group of such methods. Another
group of methods provide operations on scope context parameters. Diagram on Fig. 10
shows public interface of RenderedScopeInstance class.

Fig. 10: Public interface of RenderedScopeInstance class.

Below is a brief explanation of all class members. Most of them should be well
familiar to you from Scopes Framework. Even though the names are changed, the
meaning is still the same. Members marked "(NEW)" are the ones that were added in the
current version of the framework.

5) Scope content repeating

In DaST framework content of each scope is output one time by default meaning that
the resulting HTML fragment is just whatever this scope container has in the
template.
In order to repeat scope content, we should call Repeat(..) function on the target
scope. We also have a RestartRepeater(..) method (see Fig. 10) that resets repeating
count to 0. If this method is called and is not followed by calls to
Repeat(..),
the scope will not output any content. Listing 11 shows how
PlaylistRepeater scope
outputs its list of video packshots. First, on lines 168-170 we retrieve pager values
from PlaylistPager scope to which PagerController is attached. Then we get list
of video items from simulated data layer. Then, on line 173, we have a call to
RestartRepeater(..)
and, finally, on lines 174-179 we simply loop through the list of objects calling
Repeat(..) on each run and passing the video item object to
VideoItem scope with
VideoItemController attached to it.

One more new feature in this version of the framework is that we can save generic
objects as scope parameters. On line 178 in listing on Listing 11 we set the entire
video item object as a parameter for the VideoItem scope. The system accepts any
object as scope parameter as long as this object is serializable to JSON i.e. can
be saved to a simple string.

6) Scope visibility

In the previous version of the framework a data scope could be made
invisible meaning that output of this scope was an empty string. In the current
version each data scope has more visibility options and this is set using
RenderType property of the scope instance (see Fig. 10). This
property is of enumeration type and its possible values are summarized below:

Normal - scope is rendered normally.

Empty - scope is rendered as a container only i.e. the output of such
scope is
an empty DIV container. Binding handlers for this scope and all scopes in a subtree
are NOT called. Parameters for this scope ARE persisted. Parameters for
scopes in a subtree
are NOT persisted.

None - same as Empty, except that it is rendered as an empty string and its
parameters are NOT persisted.

Note that when scope is refreshed, its RenderType is automaticaly reset to
Normal.
Also, it's important to understand that if visibility is set to
None, you will not
be able to refresh this scope, simply because there is no HTML container that would
be able to accept the new content. Finally, you have to plan which scopes in the
scope tree should be used to save parameters, because when you set scope visiblity
to Empty or None, all its child scope parameters are discarded.

Let's look at the example how delayed loading effect is made on details dialog.
Listing 12 shows the most important back-end functions participating in the delayed
loading. On the client side, when you click "i" button on any video packshot, jQuery
Dialog plugin is used to popup the dialog box. Initially dialog box has only
animated loader
icon and no other content, because in ROOT_DataBind() method we set
DetailsContent
scope render type to Empty. This means that none of
binding handlers for child scopes
of DetailsContent scope are invoked on the initial load. When dialog opens, "LoadContent"
action is raised and handled on line 51. Inside this action handler we refresh the
DetailsContent scope and set its "VideoID" param to the id passed as action argument
from client side. Next, since DetailsContent scope is refreshed, the system re-invokes
its binding handler as well as binding handlers of all its child scopes
in traversal
order. So, DetailsContent_DataBind() on line 170 gets invoked. "VideoID" populated
in action handler is retrieved and used inside binding handler to retrieve actual
video item and use its values to add placeholder replacements. So, after this
binding
handler is executed, the system carries updated ouput to the client and DetailsContent
scope is updated with real video item info i.e. our delayed loading works exaclty
as it should. Notice lines 174 and 175 where we hide CommentsContent and
AddCommentScreen
to apply the same delayed loading technique to them, but this time within the open
dialog box.

7) Method handlers

Sometimes ability to set scope parameters is not enough and we wish to call an actual
method on the controller to execute certain activities. This feature was added to
a new version of the framework. Listing 13 shows how UpdatePagerValues(..) method is
registered for PagerController class. In the future I may come up with something
more intelligent, but in the current version, a valid method has to take single
object parameter and return object result. On line 110 we have the
UpdatePagerValues(..)
private method. It's not a problem that there is only one parameter allowed, because
I can always pass a JSON object graph and get separate values from it. On line 25 the
method handler is registered so that it becomes callable from other controllers.

Next, Listing 14 shows how the method is invoked from an action handler of
VideoLibraryController.
On line 77 we, first, point to the PlaylistPager scope which has a
PagerController
attached and call Method(..) function (see Fig. 10) to invoke previously registered
method by name. Notice how I pass the generic object as method parameter.

The first question that rises here is why do we use all this method registering
technique and dont just expose a public interface on the controller and call functions
directly? Well, this should never be done in DaST, because, recall, that there is
always a single instance of the controller per data scope which is reused for all
repeated scope instances. So, calling controller methods through registration mechanism we
simply allow the system to maintain the right context on the target controller instance.

8) Direct template manipulation

This is one of my favourite features in the new DaST framework. So far we modified
scope content only by replacing placeholders with real values. If we needed to hide
or show some areas depending on certain condition, we could enclose these areas
into actual scopes and use RenderType property for hiding and showing these
scopes. But
the problem is that we might need multiple conditional areas and creating separate
scopes for each of them would become messy in the template and would flood the back-end
controller with tons of unwanted trivial code for hiding, showing, and refreshing
these data scopes. And the solution here is to allow direct scope markup manipulation
which beautifully complements already existing placeholder replacements. First manipulation
that I've added is ShowConditionArea(..) function (see Fig. 10) to show or hide a part of the
scope markup.

For example, ReviewsInfo scope (see Fig. 5) is responsible for the text displayed
in the title of the second tab of the details dialog (see Fig. 2). When there are
comments for the current video, the text should say "XX user review(s)".
If there are no comments, the text will be "0 user reviews(s)". But what
if I wish to display more user friendly message instead, something like "No user reviews
yet"? Now the answer is simple - use the conditional area! Listing 15
shows part of the template with ReviewsInfo scope. To create a conditional area,
markup must be enclosed between <!--showfrom:condition_name--> and
<!--showstop:condition_name-->.
This special format is recognized by the system during rendering and processed accordingly.
On Listing 15 we have 2 conditional areas with conditions comm-empty:yes and
comm-empty:no
meaning that comments are empty and not empty respectively.

After template is ready, we just need to call ShowConditionArea(..) on the
ReviewsInfo
scope to show or hide areas depending on the condition. Look how this is done at
lines 200-201 in the code listing on Listing 9. On line 200 we instruct the rendering
engine to show comm-empty:yes area only when
commCount <= 0 i.e. when there
are no comments. Now imagine how flexible your UI can be having this small tool.
You can have multiple conditional areas, nest them into each other for conditional
AND, or put them beside each other for conditional OR! Just like placeholder replacements,
this manipulation can be applied to repeated content. Also remember, that DaST applies
template transformations in the same order they were called in the controller class.
I.e. if you first add some placeholder replacements and then apply conditional areas,
the order of transformations will be kept during rendering. Using this amazing tool
we must keep in mind that conditional areas, obviously, cannot have nested data
scopes. In the next version of the framework I plan to add more useful direct template
transformations.

9) DaSTUtils class

I've added DaSTUtils class containing a set of utility functions simplifying some
common programming tasks. This utility was used in listing on Listing 13 to retrieve
values from generic object passed as a parameter to UpdatePagerValues(..) function.
DaSTUtils class diagram is shown on Fig. 16.

Fig. 16: DaSTUtils class.

Brief summary of the functions is below:

GetValue - retrieves public property with specified name from the object.

HasValue - checks if the object has a public property with specified name.

ParseJSON - parses JSON string and returns object graph.

Further Development Plans

Although, current version of the DaST Rendering Engine is still Beta, it's
pretty stable and you can start playing with it and using it in your projects (obvioulsly,
no warranty -- see copyright). I can't promise anything about official documentation
-- most likely we will have to wait for several months. Overall architecture of
the platform is complete and I dont expect any major design changes; however, I'll
be refactoring certain parts of the framework and adding more features to it. Most
significant DaST developments that will take place in the near future are summarized
below.

Clean exceptions, troubleshooting utils, etc.

First thing needed is clean and uniform exceptions handling with meaningful messages
to help finding a problem. I'll create a new exception type thrown for all developer
faults. We also need some way to view model scope tree information at any time.
This is very useful for troubleshooting and testing. By design, the scope tree data
structure is not exposed to outside and this is not going to change, but I'll add
something like ToXml(..) method or similar so that the developer will
always be able to see how his changes impact the internal data structures.

More direct template manipulation features.

In addition to placeholder replacements and conditional areas I plan to add the
ability to repeat parts of the resulting fragment. This means that right inside
the data scope we will be able to create repeaters and mix them with conditional
areas. This feature is extremely important and useful, because we might not want
to create separate data scopes for repeaters and grids. I'm sure the developers
will really appreciate outstanding flexibility allowed by direct template manipulation
in DaST applications.

Transfer scope DIV attributes to initial scope parameters.

I was planning to do this for the current version, but simply forgot. It happens
:) Like standard Forms allow parametrizing controls by specifying their properties
right inside the ASPX file, DaST framework will allow parametrizing data scopes.
Inside the template you'll specify scope DIV tag with a set of attributes. All atributes
starting from underscore "_" will become scope parameters and will be available
to you through the scope instance. Underscore is needed so that scope params do
not overlap with standard HTML attributes.

Allow various scope containers.

I have to check everything first, but it seems to me that there is no problem
to allow scope containers in the templates to be not just DIV tags, but all suitable
container tags i.e. those that have innerHtml property. This is needed,
because, some apps have to have strict W3C compliance meaning that, for example,
DIVs cannot be put inside SPANs which creates certain limitations for placing DIV
scopes in the templates. Allowing data scopes to use various containers solve this
problem completely.

Moving from ASP.NET Ajax to jQuery Ajax implementation.

This, I think, is the most significant change that will involve major development.
Current partial update implementation in DaST Rendering Engine is totally based
on the standard ASP.NET Ajax Library. The implementation itself is very simple,
partial update engine is quite stable, and I was able to merge DaST partial updates
into the standard AJAX in a very elegant way, which sounds pretty good. The only
problem is that Microsoft's client AJAX Library is huge and 90% of its code becomes
useless if your application uses DaST framework. So, my opinion is that in a good
production version such overhead is unacceptable and must be eliminated. This basically
means that in the future the entire AJAX library will have to be rewritten to suite
the needs of DaST developers. The solution that seems attractive to me is to use
jQuery Ajax implementaion, but I'm not sure that this is the best solution. I might
also end up with clear JavaScript implementation without any 3rd party libraries.

And this is it. I invite everyone to visit DaST dev
site frequently and watch for news and framework updates. We're still setting
up an open-source project environment and I'll post the news about it whenever anything
is ready. I apologize for the delays -- I was really busy during last several months
with new interesting projects on my primary job. But since all architectural work
is complete, from now on I'll be updating the framework very frequenly adding more
features to it. The official release of the DaST Rendering Engine will be in a couple
of months, but the current Beta version is fully functional and stable to be used
in your projects.

Please read my discussion with Pete Sutcliffe where I describe a very typical situation when doing client coding is not just undesirable, but actually kills application flexibility and maintainability. This is number one.

Second, if you wish, you can always do client coding in DaST. I mean you just always have an option to do everything on server with very little coding and get an application with maximum flexibility as a result. And in MVC you DO NOT HAVE SUCH OPTION, unless you do some complex and time consuming coding.

I actually did read your conversation with Pete before voting on the article, and while your enthusiasm is great I don't think your approach has any merit.

I'm not really here to convince you otherwise or to engage in any lengthy discussion, but since you ask I will summarize why I think your solution is awful:

1. Modern ajax-enabled web pages emphasize speed, which means reducing the time it takes to update the page. The latest trend here is to return JSON from the server and use client-side templates to rendering the html markup (e.g. using jquery). This consumes MUCH less bandwidth than any html page, let alone one that is also bogged down by encoded view state, and it reduces the load on the server too.

2. Although it would be nice to have a set of completely untouched html+css files, it just isn't a very realistic scenario. You typically need to add element attributes (ids and classes, alt tags for images, form postback urls, etc.), which means you are modifying the designers work anyway. It becomes even worse if you want to create reusable blocks, like a box framing some content, as the designer has no concept of producing "just the box" - typically what you get to work with are full pages, which then need to be broken down into smaller parts for rendering in order to allow reuse and repositioning.

3. Also, any competent web developer will be able to make smaller adjustments on the fly without going back to the designer - if that isn't the case then educating or firing them is probably a better solution than a new web development framework.

4. Finally, there are just too many other code smells here - string-based replacement logic, huge dictionaries of unstructured variables, etc. It will be a mess to maintain and impossible to test.

You are of course entitled to have a different opinion, but I would strongly advise you to spend a couple of months working with MVC and jQuery first.

1. Modern ajax-enabled web pages emphasize speed, which means reducing the time it takes to update the page. The latest trend here is to return JSON from the server and use client-side templates to rendering the html markup (e.g. using jquery). This consumes MUCH less bandwidth than any html page, let alone one that is also bogged down by encoded view state, and it reduces the load on the server too.

OK, trend does not mean anything -- trends are changing From what you wrote is simply means that you totally sacrifice future application flexibility for the sake of client side performance. Actually, it still a question how much performance you gain on the client side, but it is already obvious that if you code contains such heavy script load, it becomes hard to maintain and modify by the support team. Only skilled developer can do modifications knowing exactly what's going on the server-side

2. Although it would be nice to have a set of completely untouched html+css files, it just isn't a very realistic scenario. You typically need to add element attributes (ids and classes, alt tags for images, form postback urls, etc.), which means you are modifying the designers work anyway.

No, you're generalizing too much. Take VideoLibrary as an example. I can totally change it's look and feel in 5 minutes playing with only HTML in the templates. And not only I, everyone having average HTML skill can do this. Think about Wordpress hypothetically based on DaST ... You'd be able to customize every single piece of your page not worrying to break something. Again, i think the main understanding problem here is that we're talking about the different types of applications: those maintained by the dev. team vs. those maintained by support team.

3. Also, any competent web developer will be able to make smaller adjustments on the fly without going back to the designer - if that isn't the case then educating or firing them is probably a better solution than a new web development framework.

Hmm, educating HTML designer to be able to make "small" back-end code adjustments? Well, during my career I can not recall such case, to be honest

too many other code smells here - string-based replacement logic, huge dictionaries of unstructured variables, etc. It will be a mess to maintain and impossible to test.

Well, this one is very strong and unsupported statement... Why string replacement smells? It's much easier than assigning value to a Label (for example) that is rendered after. Moreover you have a full control over every single byte of your HTML template, no output is produced behind the scene. Also, why dictionaries are unstructured? In fact they are structured and this structure is defined very well by your scope tree where each scope can have a set of associated parameters. Regarding testability ... From my discussion with Pete below, it should be obvious that DaST application is at least as testable as MVC2. I'm not sure about MVC3 yet, but saying that it's not testable means that you did not read the article.

You are of course entitled to have a different opinion, but I would strongly advise you to spend a couple of months working with MVC and jQuery first.

The DaST concept is not something that I came up with randomly -- it's a result of web development observation and problems that my team had with WinForms and MVC apps during at least 5-7 years.

OK, trend does not mean anything -- trends are changing From what you wrote is simply means that you totally sacrifice future application flexibility for the sake of client side performance. Actually, it still a question how much performance you gain on the client side, but it is already obvious that if you code contains such heavy script load, it becomes hard to maintain and modify by the support team. Only skilled developer can do modifications knowing exactly what's going on the server-side

You are missing the point. Sending json across the wire and rendering html on the client saves both bandwidth and server resources, both of which translate into reduced cost. This is important if you have any kind of significant traffic. It also provides a much better experience for users.

rgubarenko wrote:

No, you're generalizing too much. Take VideoLibrary as an example. I can totally change it's look and feel in 5 minutes playing with only HTML in the templates. And not only I, everyone having average HTML skill can do this. Think about Wordpress hypothetically based on DaST ... You'd be able to customize every single piece of your page not worrying to break something. Again, i think the main understanding problem here is that we're talking about the different types of applications: those maintained by the dev. team vs. those maintained by support team.

Again, you are missing the point. I can apply the same fixes you can to the original files produced by the designer, although I'd also have to subsequently merge those changes into my views (a fairly simple task if you have a decent diff tool).

And you completely missed my point about composability - being able to reuse smaller chunks of html as components on multiple pages. In order to do this you literally have to break the page into multiple separate pieces. But maybe you prefer markup duplication over DRY?

Your obsession with support is also misplaced. Support people should not "maintain the solution". They should determine the cause of the problem and assign the issue to someone in charge of that particular problem area (e.g. "spec is wrong" -> product manager, "bug" -> developer, "disk full" -> operations).

You keep repeating that you can change the html as you like, but what is the true value of that? For instance, you cannot change the input values to the view and so are extremely limited in what you can do. For instance, you cannot (without modifying the code) change some label text to be html encoded, only display the year of a DateTime, render a TimeSpan < 1m as "moments ago" or take any other view decisions like that. You can only change the static markup, but I see little value in that in real-life scenarios.

rgubarenko wrote:

3. Also, any competent web developer will be able to make smaller adjustments on the fly without going back to the designer - if that isn't the case then educating or firing them is probably a better solution than a new web development framework.

Hmm, educating HTML designer to be able to make "small" back-end code adjustments? Well, during my career I can not recall such case, to be honest

That's not what I wrote. (Web developer = programmer) != designer.

rgubarenko wrote:

Why string replacement smells? It's much easier than assigning value to a Label (for example) that is rendered after. Moreover you have a full control over every single byte of your HTML template, no output is produced behind the scene. Also, why dictionaries are unstructured?

I don't even know where to start with this. I think it's sad that you prefer some convoluted/string-based DataBind replacement over the niceness of being able to just do @Model.UserName in the view (used to be <%= Model.UserName %> in earlier MVC versions).

Unstructured means that it is a flat list of key/value pairs. Structured means you have nesting, e.g. like having a UserModel which contains a list of AddressModel instances. Strings are also bad because you get no refactoring support, and once you have 100s of values on a page who in their right mind can keep track of all of them? How do you detect that a dictionary key was changed in code but not in the view? The compiler doesn't help you anymore.

rgubarenko wrote:

he DaST concept is not something that I came up with randomly -- it's a result of web development observation and problems that my team had with WinForms and MVC apps during at least 5-7 years.

Web development is hard - you need to know html, css, javascript, C# and a multitude of other stuff. Every process of producing web applications has to make some compromises, but I do not think yours makes (any of) the right ones.

You are missing the point. Sending json across the wire and rendering html on the client saves both bandwidth and server resources, both of which translate into reduced cost.

I'm not getting your point. Sending JSON saves bandwidth, so what? How bigger the bandwidth of DaST application would be comparing to MVC? Do you have the actual numbers or you *think so*? Even if it were 20% (which it definitely would not with proper design), I'd say that in most of the cases this traffic waste is acceptable, if I can get a simpler, more stable, and flexible application architecture. Same way you state that JSON saves bandwidth, I can state that client script load makes your application hard to maintain.

mertner wrote:

It also provides a much better experience for users.

How? Sorry, but it seems to me that you're making these statements based on some stereotypes. In DaST partial content is returned and panel is updated. In MVC partial content is returned, picked up by the script and panel is updated. So what is the difference here from the end user point of view? In 1 millisecond faster response? The user will not notice any difference, BUT when the developer comes to update some pages, he'll have to delve into your scripts instead of moving around simple HTML elements.

mertner wrote:

Again, you are missing the point. I can apply the same fixes you can to the original files produced by the designer, although I'd also have to subsequently merge those changes into my views (a fairly simple task if you have a decent diff tool).

Well, your point is completely wrong in this case. Look what you're proposing: give views to designers to do styling, then YOU take these views, sit with a merger and finalize your views So question to you: how many people are involved in the process? How many resources? Merging HTML mixed with control statements is 5 minute work for you? Definitely not - couple of hours for some serious templates at least. And this process will always require significant portion of your involvement. In case of DaST, unless scope nesting structure is changed, you dont have to worry about most of HTML layout changes.

mertner wrote:

<layer>And you completely missed my point about composability - being able to reuse smaller chunks of html as components on multiple pages. In order to do this you literally have to break the page into multiple separate pieces. But maybe you prefer markup duplication over DRY?

I'm not getting it again. Factoring out smaller peaces of the bigger page is absolutely the same in DaST (child controllers and tempaltes), MVC (partial views), and WebForms (controls). You're not limited in breaking your page into pieces. If you wish, you can do all in one controller, otherwise, you can create a number of hierarchically structured child controllers. Same as in MVC or Forms ... Did I misunderstand your question?

mertner wrote:

Your obsession with support is also misplaced. Support people should not "maintain the solution". They should determine the cause of the problem and assign the issue to someone in charge of that particular problem area (e.g. "spec is wrong" -> product manager, "bug" -> developer, "disk full" -> operations).

Again, this is theory -- reality is always different. This may be true for huge companies with thousands of employees, but not for smaller firms. The ultimate purpose is to get your work done, to roll out a product and to get paid. Client does not really care if your senior R&Ds will have to do HTML styling -- they just need a working product. And the idea of DaST in this case is to expose a most trivial presentation controlling mechanism, the pure HTML templates, allowing for quick and simple layout modification. Taking into account, that 95% of change requests are simple layout modifications, I have good reason to claim that HTML templates in DaST have many advantages over other framework approaches.

mertner wrote:

You keep repeating that you can change the html as you like, but what is the true value of that? For instance, you cannot change the input values to the view and so are extremely limited in what you can do. For instance, you cannot (without modifying the code) change some label text to be html encoded, only display the year of a DateTime, render a TimeSpan < 1m as "moments ago" or take any other view decisions like that. You can only change the static markup, but I see little value in that in real-life scenarios.

True value is actually huge. I was working for a big media entertainment company running about 40 different sites for Bell, Walmart, etc. with complex and dynamic UI. And, as I mentioned above, 95% of all change requests are simple layout. Let's put it simple -- changing HTML template in DaST you can make your site look completely different in no time. For some companies this is just "nice to have" ability, but for others, this is the key in their business (take numerous CMS engines, for example). But, of course, I accept that there are certain applications that do not benefit from having HTML templates.

mertner wrote:

I don't even know where to start with this. I think it's sad that you prefer some convoluted/string-based DataBind replacement over the niceness of being able to just do @Model.UserName in the view (used to be <%= Model.UserName %> in earlier MVC versions).

This sounds like your personal preference, which is totally fine. I also like strongly typed properties, but unfortunately they do not coexist with the level of flexibility and template simplicity which I target.

mertner wrote:

Unstructured means that it is a flat list of key/value pairs. Structured means you have nesting, e.g. like having a UserModel which contains a list of AddressModel instances.

OK, I see, but you can easily save any generic objects of any structure to parameters is DaST. IMHO no big benefit that I can see, but if you really like having *layered* data objects, DaST does not limit their usage.

mertner wrote:

Strings are also bad because you get no refactoring support, and once you have 100s of values on a page who in their right mind can keep track of all of them?

Yes, real refactoring is missing in this case, but I used search-and-replace which did the work. Regarding 100s of values ... well, same way you'd have to keep track of them in MVC, right? You'd have to remember which properties are available in your model and which can be used in your view. Note that you have intellisence in VS, but HTML designers use DreamWever

mertner wrote:

Every process of producing web applications has to make some compromises, but I do not think yours makes (any of) the right ones.

You have your opinion and I respect it as long as it is well supported. One way to solve our dispute is to see DaST in action which I hope will happen soon.

I've made my case, and you seem to be stuck with your current thinking. And when I make a point you either ignore or discard it. I really don't care as it's not my company money you are pouring down the drain with this effort. But I sincerely wish you wouldn't publish articles claiming to have invented the ultimate web framework, when all you really have is no clue.

As such, I'll withdraw from this debate and wish you good luck. Feel free to gloat if you ever conquer the world.

This article as one before article has some interesting part... BUT:
What when you paste "videoID" or "Videoid" or "video_id" or "video-id" to this scope dictionary data? Will it work? I think not. I don't want how i named my own variables in my own code I have IDE for that. I am to lazy. Also i don't see much diffrence between
<!--conditional-comment-->
and
if(..){}else{} in MVC.

But from other hand I see advantages so I look forward to final release.

For data scope you have:
1) parameters that system persists from request to request (using JSON and storing them currently in ViewState)
2) placeholders that you "bind" to real data using Replace(..) (for current request only)

For parameters, if you store it with name "video-id", then you retrieve it, obviously, with the same name.
Placeholders do not have any specific format. I use "{name}", you can use "--name--" or whatever. As I said, it's just a simple search-n-replace operation, and only thing that you have to think of choosing the format, not to overlap with the rest of markup in the template. Definitely, instead of Replace("{VideoID}", strID) you can have Replace("{v-I-d-E-o-I-d}", strID) or whatever you wish.

For parameters, if you store it with name "video-id", then you retrieve it, obviously, with the same name.Placeholders do not have any specific format. I use "{name}", you can use "--name--" or whatever. As I said, it's just a simple search-n-replace operation, and only thing that you have to think of choosing the format, not to overlap with the rest of markup in the template. Definitely, instead of Replace("{VideoID}", strID) you can have Replace("{v-I-d-E-o-I-d}", strID) or whatever you wish.

This is EXACTLY why i don't like in your code. It's enough if you have 1, 2 pages to maintain. But what you if you have 20? 40? 100? You expect everyone to remember ALL of that strings-placeholders. Especially if you doesn't remember what is meaning or purpose of some variable? I don't know how you do it, when you pass wrong string to ViewState, but if this only return null... It will be huge problem. Possible that nobody would even notice that error.
And what with refactoring? Will you search all files manually to replace them with something else? Or you place trust to automation in visual studio?
I know that I can create struct with const strings to contain that string, and I usually do that with session and ViewState and other containers... but it's not elegant, and it's just another layer of abstraction.
Maybe I am like to complain, but I like Model with it's intelisense... there are no place to make an error.

Magic strings? No, it is most common mistake to make by me.

I wish you have done something like custom classes encapsulated in scope class, which inherit all of custom class variables, like:

masterScope.someVar = 1;
masterScope.nestedScope.otherVar = 2;

I don't know if it's achievable but it certainly would be nice

Another problem to me are that conditional comments. What would happen if someone remove one? Or change name? Application would throw error? Certainly not in compilation.
I too prefer <%%> or @{}@ blocks in MVC, in that way you get notice from IDE that something is wrong.

I don't know... One way your solution seems much cleaner than MVC, and much more friendly than ugly and bad UpdatePanels, but (there is always but! :/)
it could be hard to maintain and change and could bring more problems than it could solve.

So once more I vote for 3.

It's interesting but i can see some problems with that. And i wait for final version.
Keep up you doing really good work nonetheless.

Well, delivering raw string data to the HTML template by using trivial placeholders is the underlying idea of the entire framework. On one hand yes, you have to know which placeholders are allowed in the template (is this really such a problem ), on the other hand, this gives you maximum level of flexibility customizing your page look-and-feed WITHOUT primary development resource intervention.

And yes, I try to approach server-page rendering problem from another side, not having any Label, Literal or whatever controls in the markup or Html.WriteLabel(..) or similar in MVC. Again, I'm trying to get rid of flooding markup with functional controls. Although, they expose strongly typed properties which make your back-end programming "may be easier somewhere", they totally kill most of UI customization flexibility as well as thy destroy the idea of presentation-backend code separation.

Another problem to me are that conditional comments. What would happen if someone remove one? Or change name? Application would throw error? Certainly not in compilation.

It's important to understand that conditional areas is just a *nice* add-on, but not the DaST core. It simplifies the things in your DaST development, but you can achieve the same just using code tool. In the core DaST has only DIV scopes, placeholders and nothing else, which guaranties your output as long as you properly specify placeholders and scope tree structure in the template.

And, hey, I'm not saying that all flexibility given by DaST is given for "free" so you don't have to do anything Of course not. You still have to develop, you still have to think, you still have to watch for your markup, you still have to think of the back-end design etc

I too prefer <%%> or @{}@ blocks in MVC,

That's the main problem here. One of DaST pattern purposes is to allow such presentation engine where each page can be customized by an HTML designer not having any idea of "<%%> or @{}@" or any other server specific markup. The designer wants to work with pure HTML and nothing else using DreamWeaver. After working with huge web-based systems for a couple of years, the advantage of such approach became obvious to me. I don't have to spend time of expensive back-end developers to do a deep page layout customization (95% of client change requests are like this), unless it's a bigger functionality change.

Definitely, if you just want some application with input form and simple processing and you don't care about presentation customization, then you're better of using simple WebForms approach: drag-n-drop couple of controls on the page, bind then to handlers, and you're done. But if you're creating some agile software and you have to change/update something every day for tens of clients, then you better give DaST a second thought

Anyway, thanks for your comment. I'm happy to have a discussion here so that I know what to improve in the future
Source code will be available as soon as the project is setup on SourceForge: http://sourceforge.net/projects/aspnetdast/.

I understand all of that. Really. My english may be not perfect so my posts may sound like nonsense, but i am sure of one thing (it is also one of Murphy's Laws )

'if anything can be screwd up it eventually will be'

So who will be responsible for ommitting one letter (or adding - it's the same) in one of this magic string?
And I really don't think, from other hand, that anybody will remember all of this scope names in larger app.
And I know too that you can't write error proof framework, but this in my opinion will cause to many problems.

I cannot promise any dates, but I'll try to reproduce it. DaST is not my primary work, so I'm pretty short on time, but hopefully, I'll be able to get some help from DaST community on SourceForge in the future.

I don't really see the advantage this has over an MVC approach where we have full control over the rendered content and are not limited by viewstate and such. Analyzing the traffic sent / received by the demo page it seems pretty clear that under the hood this is still using a classic webforms model and shares many of its limitations (bloated markup, slow AJAX performance).

Also compared to MVC where I can generate a data model and send it straight to my html view it seems to require a lot of code to get things displayed.
As is, I don't think your claims about being simpler or more powerful than MVC really stand up to scrutiny. Perhaps this might be better implemented as a custom ViewEngine for MVC?

Anyway, as is I give you full marks for coming up with an interesting approach, but I think you are slightly overstating your case.

don't really see the advantage this has over an MVC approach where we have full control over the rendered content and are not limited by viewstate and such.

There may be types of applications that would not really have any advantages implemented on DaST vs MVC, may be. But, when I created this concept, I was mostly thinking about flexibility and architecture of the dynamic Web 2.0 sites, that have to be updated almost every day, and usage of pure HTML templates and clear and simple backend would be a key in such case. I used this concept already and based on my own experience I came to some conclusions about DaST vs. Forms vs. MVC. Definitely, I expect a big discussion on this in a couple of months after the project goes open-source and everyone is able to apply DaST in their real sites.

Analyzing the traffic sent / received by the demo page it seems pretty clear that under the hood this is still using a classic webforms model and shares many of its limitations

Yes, just as I stated in this article, current version of DaST is BETA1 and client Ajax part of DaST is just several strings of code that use the standard MS AJAX (which still works pretty well). To re-implement this in jQuery ajax is one of my development goals for a near future. Design-wise using Action / Handler approach in DaST is just so much simpler than Update Panel / Trigger / whatever approach in WebForms.

(bloated markup, slow AJAX performance)

What you mean by "bloated markup"? Markup that you're getting is whatever you have in your HTML template, nothing more and nothing less. If you dont like inline styles, just use the CCS and classes. I used the inline styles in DEMO just for simplicity.

Also, no, AJAX performance is NOT slow We reference WinForms AJAX as "slow" ONLY because the ASYNC postback executes the entire page lifecycle invoking tons of unnecessary code. DaST does not have any of such problems due to its architecture -- code is executed only in handlers for specific refreshed scopes. The only problem with MS AJAX + DaST is that standard library just "has" lots of unnecessary code which is never ever utilized by DaST, but this does not really mean anything in terms of performance. The postback just does whatever it should: invokes __doPostBack(), collects all form values (this part will go away in jQuery Ajax implementation) and add them as data for XmlHttpRequest, then server output is generated and sent to client, then MS AJAX parses the response retrieving content for updated panels and form values, then this content is simply inserted in corresponding containers by simple GetElementById(..).InnerHTML, and this is it. I dont see any "slow" part here.

Also compared to MVC where I can generate a data model and send it straight to my html view it seems to require a lot of code to get things displayed.

May be I misunderstand something, but to get things displayed in DaST you just need HTML template and generated data. This is the LEAST that you need -- the data and how to represent it. In MVC you generate your data and send it to the View, but you also have to have an ASPX to control the output, right? So how is this smaller then in DaST and where does it has "a lot of code" where MVC normally would not?

As is, I don't think your claims about being simpler or more powerful than MVC really stand up to scrutiny.

Oh, definitely, I agree. I'm not asking anyone to just believe my word, that's exactly why I created a working DEMO application so that everyone can run it and play with it. And it will be simple to compare, because DaST becomes an open-source with all code available at http://sourceforge.net/projects/aspnetdast/ (i will announce on my site when everything is ready)

Perhaps this might be better implemented as a custom ViewEngine for MVC?

Will see how it goes, but currently my purpose is to use DaST architecture INSTEAD of WebForms and MVC, not as part or add-on to any of them.

Anyway, as is I give you full marks for coming up with an interesting approach, but I think you are slightly overstating your case.

Thanks for your support. How good or bad the DaST is will become clearer after a couple of months after it's tried in at least a couple of real projects.

To expand a bit on my comments, I guess one of my big concerns is that most of the things you have highlighted as being improvements apply to webforms only, you haven't addressed any of the significant improvements MVC gives over webforms and seem to have lumped the two technologies into the same basket. MVC's improvements are not just in the architectural pattern but the massive improvements in terms of control over markup, working more naturally with the http request / response cycle and so on. I was left wondering if you really have a lot of experience working with the MVC framework to be honest.

Conceptually I don't see a lot of difference between your html templates and the page / user control or view / partial view model except I have to put a lot more custom attributes / comments in your templates. What is the big advantage of your approach?

What you mean by "bloated markup"? Markup that you're getting is whatever you have in your HTML template, nothing more and nothing less.
Specifically I mean the big blob of ViewState that increases the page size, if that's a beta issue then fair enough.

Also, no, AJAX performance is NOT slow Smile | We reference WinForms AJAX as "slow" ONLY because the ASYNC postback executes the entire page lifecycle invoking tons of unnecessary code.
Not really, we call Winforms AJAX slow because of the viewstate and large amounts of markup that is transferred to / from the server on every action, something your current implementation is also guilty of, typically transferring 10-15k+ per request. Having downloaded your demo I was quite disappointed with the performance running on my local machine, there are noticeable delays between performing an action and seeing the result.
This brings me to another concern I have. You seem to have approached this from the angle of 'wow, the asp.net page lifecycle is complex, it must be really slow' but I'm not convinced this is necessarily the case. Don't forget, aspx templates are not interpreted at runtime, they are pre-compiled into IL code which brings a lot of efficiencies. As MVC pages typically don't use server controls, the lifecycle complexity isn't really an issue there at all. If your pages are having to load the html templates and do the find / replace operation for every action at runtime I would expect them to be a lot slower than a normal asp.net page, have you done any kind of testing to confirm your improved performance claims?

So how is this smaller then in DaST and where does it has "a lot of code" where MVC normally would not?
Well, for example looking at your DetailsPopupController there's a lot of plumbing code to set up databinding, events etc that wouldn't be necessary. There is also a fair bit of JavaScript in the template to handle the AJAX which would be handled for me if I were using web forms AJAX. When using MVC I'd typically have very light controllers with more focus on client code to minimize the amount of work the server has to do and the amount of data being transferred. By way of comparison your controller for the details pop-up, which is just one small section, has over 300 lines of code in it, the MVC project I'm working on now my main controller that handles the input / output for 10 pages is less than 300 lines.

Hopefully a lot of these concerns are either misplaced or will be ironed out in future releases. I'll definitely be keeping an eye on it but mostly out of curiosity and admiration for people who try to do something different rather than an expectation that this is the future of web development as you claim.

Conceptually I don't see a lot of difference between your html templates and the page / user control or view / partial view model except I have to put a lot more custom attributes / comments in your templates. What is the big advantage of your approach?

Ok, the main conceptual difference from template point of view is that I COMPLETELY DENY using functional markup inside the HTML template. HTML template must stay HTML and nothing else - this is the idea. Advantages? Well, first, because I can give this HTML template to any HTML designer in my team (who has no idea about control statements) and she'll just get a simple HTML with WYSIWYG in any her favorite browser, and the main dev team will continue working on a project not waiting for the graphics designer. Second, this totally separates presentation from back-end. Yes, I know that MVC has beautiful TTDable controllers; however, my opinion is that DaST is the first pattern allowing true ui-from-backend separation.

viewstate and large amounts of markup that is transferred to / from the server on every action, something your current implementation is also guilty of, typically transferring 10-15k+ per request.

I used ViewState in the Alpha Version, in Beta1 there is a __UNITEDSTATE or __ASPNETDASTSTATE. This state contains ONLY parameters serialized into JSON. Roughly, there are only 2 ways to persist parameters. Do the client state transfer or use the server state transfer identified by session ID. Currently, I dont know how MVC3 does it honestly, I'll take a look, but there is no other way but using state transfer and blaming DaST for using state is strange -- you'll always have this data transfer in every possible super-framework, unless you deny persisting any data between requests. Obviously, data itself can be minimized by compression or other optimization techniques, but I did not do this for BETA1.

Having downloaded your demo I was quite disappointed with the performance running on my local machine, there are noticeable delays between performing an action and seeing the result.

Did you run it on you smart phone? Seriously, on your local machine it should run without any delays. Notice that I put Thread.Sleep(..) statements to physically delay the postback and simulate data retrieval. If you run it from Dev site here http://www.rgubarenko.net/dastdemo/VideoLibrary.aspx, then yes, it is slow, because my old server is slow.

Anyway, I don't just claim any performance increase, I'll let everybody to delve into the source code and go though the entire rendering process step-by-step to confirm or disprove whatever stated in this article regarding performance. Also, I plan performance testing, but it's a bit too early for BETA1, so this will most likely happen some time in the summer.

Well, for example looking at your DetailsPopupController there's a lot of plumbing code to set up databinding, events etc that wouldn't be necessary.

Well, this is just a boilerplate code which should be done by VS integrated designer and put inside the partial controller. Obviously, there is no designer yet for BETA1 and you have to do it manually inside the controller class. What do you think WinForms or MVC do compiling your views? They do the similar thing, producing code instead of markup. But, yes, currently in DaST you have to do this manually (which is not necessarily bad).

There is also a fair bit of JavaScript in the template to handle the AJAX which would be handled for me if I were using web forms AJAX

No, wait, there is no single line of JavaScript to support functioning of DaST core except for Action(..) calls. Look at that JS code again -- there's only the code to rebind or setup jQuery UI widgets. All jQuery stuff is extra, and you would HAVE TO do similar JS coding in every possible framework if you wish to use jQuery widgets and UI.

By way of comparison your controller for the details pop-up

The best way to compare MVC to DaST is to create VideoLibrary in both (if I have time I'll create VideoLibrary in MVC for comparison). BUT not just create ... Apply some support use-cases when client comes to you 5 times a day asking to move something here or there on the page, or put additional logo or banner (HTML templates come handy in this case)

Hopefully a lot of these concerns are either misplaced or will be ironed out in future releases. I'll definitely be keeping an eye on it but mostly out of curiosity and admiration for people who try to do something different rather than an expectation that this is the future of web development as you claim.

Thank you!

Keep an eye on http://sourceforge.net/projects/aspnetdast/ -- nothing is setup yet, but I hope to have everything ready soon. Most of your questions will automatically go away once you are able to work with the source code.

OK probably the last comment from me as I'm being quite negative and it's really not my intention to put down your hard work but the more I read about this the less convinced I am.

I COMPLETELY DENY using functional markup inside the HTML template.

So what are those comments if not functional markup? The designer still has to understand how to use those and understand about data-scopes etc. Most likely they would design the html then pass it to me to add this extra stuff, which is similar to how I'd do it anyway. There are MVC view engines that provide similar functionality to this without being so limiting. Far from increasing flexibility as you claim, your system limits me to your predefined control statements. Worse still I have to duplicate everything I do as instead of being able to just write my model value onto the page where I want it I have to add a placeholder to the page, then remember to manually tell the controller to replace this placeholder with the value I want. And as pointed out in another comments thread, if there is a typo in the placeholder then there is no compile-time checking to let me know. It's a potential testing and maintenance nightmare.

my opinion is that DaST is the first pattern allowing true ui-from-backend separation.

Based on what? You keep making these grand statements but offer little to back it up. As an example of testability lets consider submitting a comment on a video. In MVC my code would look something like this:

This is a nice self-contained easily tested piece of functionality, logically separated from the actual UI behaviour. On the other hand, your routine that saves the comment is tied up in framework specific event arguments and code to refresh various bits of the UI. There is no ui-from-backend separation at all and the code is simply not testable in a meaningful way.

This state contains ONLY parameters serialized into JSON.

This is not true. Profile your site using Firebug or something, it's transferring large amounts of Base64 encoded data, the same as Viewstate would.

blaming DaST for using state is strange -- you'll always have this data transfer in every possible super-framework, unless you deny persisting any data between requests

I don't know of any other frameworks that use a similar mechanism apart from webforms. Viewstate basically came about to create an abstraction of statefulness that allowed VB6 programmers to make web applications without having to understand how the web works. It wasn't a particularly good idea at the time and it's completely redundant now we have AJAX and JavaScript capable browsers, which is why MVC dropped it. Again, run Firebug against Facebook or some other AJAX heavy site, you'll typically find the request / response messages are much leaner.

What do you think WinForms or MVC do compiling your views? Wink | They do the similar thing, producing code instead of markup. But, yes, currently in DaST you have to do this manually (which is not necessarily bad).
Not sure what you mean by this, there is a difference between auto-generation of code and compilation. But given that you are promising that your framework is 'simpler and smarter' then yes it is a bad thing, your framework dramatically increases the amount of work I have to do. If using this really saves you 90% development time, I'd be fascinated to see how you used to do it.

Apply some support use-cases when client comes to you 5 times a day asking to move something here or there on the page, or put additional logo or banner (HTML templates come handy in this case)
Again, I fail to see how moving something in a template is significantly easier than moving something in a view (which is mostly just html anyway).

So anyway, sorry to be so negative but given that you are claiming to have created the future of web-development and a framework which beats both Webforms and MVC I think you have to expect a certain amount of scrutiny.

OK probably the last comment from me as I'm being quite negative and it's really not my intention to put down your hard work but the more I read about this the less convinced I am.

Discussion tends to finding better solutions, so I read and answer your comments with pleasure even though they have a certain amount of criticism. Honestly, if I were using MVC for a couple of years and then somebody just came up and say hey, there is something better and smarter, I might be even more suspicious then you are You're rising typical questions that every more or less experienced web developer would have, so from this point of view, our discussion is really useful for me and for everyone.

So what are those comments if not functional markup? The designer still has to understand how to use those and understand about data-scopes etc.

No, they don't have to understand anything about the scopes -- they only have to keep the scope tree structure i.e. DIV with "scope" attribute nesting structure which is a really simple task for them. Don't you see the difference between <DIV>...<DIV>..</DIV>..</DIV> and all those control statements that MVC has (i'm not even mentioning WebForms)? Something like:

Do you really think this MVC markup has the same level of complexity then repeated DIV with placeholders bound inside single data handler in DaST? May be for you yes, because you're the "expensive developer with skills", BUT not for the design team

Number two is that DaST functional markup is UNIFORM. Yes, it is always the same everywhere not depending what type of UI elements you're trying to output and you don't have to learn anything except DIV with "scope" attribute. Conditional areas separated by those special comments is a simplifying add-on -- you have an option to use it or not. This is the main reason I say that there is no functional markup in DaST.

It's probably more fair to say that DaST has a THEORETICAL MINIMUM of functional markup.

This is not true. Profile your site using Firebug or something, it's transferring large amounts of Base64 encoded data, the same as Viewstate would.

All parameters are saved to __UNITEDSTATE variable and you're wrong saying that it has something other than scope parameters. It contains only scope params serialized to JSON and nothing else. Decode that Base64 string here: http://www.motobit.com/util/base64-decoder-encoder.asp and take a look. Below I paste part of decoded data:

As you can see, it's just simple uncompressed JSON Compressing it will reduce state by about 45%, but I'm not sure if I want to do this, since no framework does it yet.

In MVC my code would look something like this:

Well, it's hard to say right now -- I did not work with MVC for a while. I'll try to reproduce this application in MVC using best practices so that everyone will be able compare. DaST separates action from data generation. Refresh(..) calls are needed only because raising Action(..) in DaST invokes async postback, BUT it does not refresh anything by default. You have to explicitly tell the system which parts of you scope tree you wish to refresh.

code is simply not testable in a meaningful way.

It's a little bit too early to say that it's not testable There is no DaST testing tools yet, but the testability idea is clear: you'll have some UI like NUnit showing your scope tree and giving the data (or list of data for repeaters) produced by the refreshed scopes. And this is it. You're getting just data not caring about the template and that data you can ASSERT. This is extremely simple and clear TDD approach.

Again, run Firebug against Facebook or some other AJAX heavy site, you'll typically find the request / response messages are much leaner.

I already answered this above, but just small addition. Some applications somewhere -- it's too abstract Let's be more specific. If you need to access certain data on the back-end after async post-back YOU MUST pass this data from the client. If data is not passed, it's not available, unless something is stored in your session (which is not applicable to ViewState mechanism). Agreed? Next, I pass the entire current state for the scope tree, because in that case you're able to access ALL scope tree parameters no matter where your action is occurred. You cannot access data of another controller, but you can raise controller action handled in that another controller, so potentially you may need all available params and to be on the safe side I MUST pass all scope tree params, which is exactly what I'm doing.

Not sure what you mean by this, there is a difference between auto-generation of code and compilation. But given that you are promising that your framework is 'simpler and smarter' then yes it is a bad thing, your framework dramatically increases the amount of work I have to do. If using this really saves you 90% development time, I'd be fascinated to see how you used to do it.

Recompilation is something that i don't want. One of my purposes was to allow the applications to switch HTML templates on the fly, read them from anywhere - IO, HTTP web requests, database repositories. Switch them even for refreshed scopes during async postbacks!

Regarding dramatically increased amount of work ... Well, it's hard for me to prove anything immediately. That's the reason why I'm setting up an open-source for ASP.NET DaST and PHP DaST so that everyone will be able to use them in ASP and PHP applications and make their own decisions. For now, there is only one DaST expert in the world (guess who ) and all I say about dev speed increase is based on my own experience. Best proof (or disproof) for my words will be to see DaST "in action" in a couple of months after open-source dev community plays with it.

Again, I fail to see how moving something in a template is significantly easier than moving something in a view (which is mostly just html anyway).

May be it also depends on the type of the application you're working on. As I said above, you can expose you DaST templates on external server for modification by your client. Imagine CMS engine built on DaST, where you can customize every single piece of your UI doing simple HTML? Compare to the best ones we currently have (DotNetNuke, Wordpress, etc.). I thing these are pretty strong arguments supporting DaST flexibility, but anyway, i hope to see it in action soon to have physical proof for my words.

So anyway, sorry to be so negative but given that you are claiming to have created the future of web-development and a framework which beats both Webforms and MVC I think you have to expect a certain amount of scrutiny.

It's totally fine to be negative as long as you have well argumented opinion. DaST came to public just a couple of weeks ago and, obviously, nobody is using it yet, so I understand that it's very hard to immediately give it an impartial assessment and there is still quite a bit of personal preference, because all of us are used to the platforms we're working with on everyday basis. Again, apart from my the arguments and thoughts supporting DaST vs. MVC, the best thing I can do to prove everything I claimed is to setup open-source and roll out the final production version of the framework, which I am doing already