Tag Archive for javascript

A Javascript-only, web based Dashboard library, that you can use to make eye-catchy Charts showing Transactions stats for your application, Server CPU, Memory, IO graphs etc. It offers libraries to convert different types of delimited text files into Charts. Those delimited files are generated from various sources, eg running SQL queries against a Database, or running shell scripts to collect system stats, or Powershell scripts to process IIS logs. End result is a nice-looking Twitter Bootstrap powered, responsive Web Dashboard, that you can get up and running in no time, on any platform. Since it is HTML and Javascript, you can customize it to show exactly what you want, how you want. Quite handy for earning brownie points and wooing your customers.

I have enhanced my streaming Ajax Proxy with POST, PUT and
DELETE features. Previously it supported only GET. Now it supports
all 4 popular methods for complete REST support. Using this proxy,
you can call REST API on external domain directly from your
website’s javascript code. You can test the proxy from this
link:

I am using MS AJAX here. You can use jQuery to perform the same
test as well. All you need to do is hit the URL of the
StreamingProxy.ashx and pass the actual URL in query string
parameter “u” and pass the type of the http method in
query string parameter “m”. That’s it!

Introduction

When you create rich Ajax application, you use external
JavaScript frameworks and you have your own homemade code that
drives your application. The problem with well known JavaScript
framework is, they offer rich set of features which are not always
necessary in its entirety. You may end up using only 30% of jQuery
but you still download the full jQuery framework. So, you are
downloading 70% unnecessary scripts. Similarly, you might have
written your own javascripts which are not always used. There might
be features which are not used when the site loads
for the first time, resulting in unnecessary download during
initial load. Initial loading time is crucial – it can make
or break your website. We did some analysis and found that every
500ms we added to initial loading, we lost approx 30% traffic who
never wait for the whole page to load and just close browser or go
away. So, saving initial loading time, even by couple of hundred
milliseconds, is crucial for survival of a startup, especially if
it’s a Rich AJAX website.

You must have noticed Microsoft’s new tool Doloto
which helps solve the following problem:

Modern Web 2.0 applications, such as GMail, Live Maps, Facebook
and many others, use a combination of Dynamic HTML, JavaScript and
other Web browser technologies commonly referred as AJAX to push
page generation and content manipulation to the client web browser.
This improves the responsiveness of these network-bound
applications, but the shift of application execution from a
back-end server to the client also often dramatically increases the
amount of code that must first be downloaded to the browser. This
creates an unfortunate Catch-22: to create responsive distributed
Web 2.0 applications developers move code to the client, but for an
application to be responsive, the code must first be transferred
there, which takes time.

Microsoft Research looked at this problem and publishedthis
research paper in 2008, where they showed how much improvement
can be achieved on initial loading if there was a way to split the
javascripts frameworks into two parts – one primary part
which is absolutely essential for initial rendering of the page and
one auxiliary part which is not essential for initial load and can
be downloaded later or on-demand when user does some action. They
looked at my earlier startup Pageflakes and reported:

2.2.2 Dynamic Loading: Pageflakes
A contrast to Bunny Hunt is the Pageflakes application, an
industrial-strength mashup page providing portal-like
functionality.
While the download size for Pageflakes is over 1 MB, its
initial
execution time appears to be quite fast. Examining network
activity
reveals that Pageflakes downloads only a small stub of code
with the initial page, and loads the rest of its code dynamically
in
the background. As illustrated by Pageflakes, developers today
can
use dynamic code loading to improve their web application’s
performance.
However, designing an application architecture that is
amenable to dynamic code loading requires careful consideration
of JavaScript language issues such as function closures,
scoping,
etc. Moreover, an optimal decomposition of code into
dynamically
loaded components often requires developers to set aside the
semantic
groupings of code and instead primarily consider the execution
order of functions. Of course, evolving code and changing
user workloads make both of these issues a software maintenance
nightmare.

Back in 2007, I was looking at ways to improve the initial load
time and reduce user dropout. The number of users who would not
wait for the page to load and go away was growing day by day as we
introduced new and cool features. It was a surprise. We thought new
features will keep more users on our site but the opposite
happened. Analysis concluded it was the initial loading time that
caused more dropout than it retained users. So, all our hard work
was essentially going to drain and we had to come up with something
ground breaking to solve the problem. Of course we had already
tried all the basic stuffs –
IIS compression,
browser caching, on-demand loading of JavaScript,
css and html when user does something, deferred
JavaScript execution – but nothing helped. The frameworks
and our own hand coded framework was just too large. So, the idea
tricked me, what if we could load functions inside a class in two
steps. First step will load the class with absolutely essential
functions and second step will inject more functions to the
existing classes.

I published a codeproject article which shows you 7 tricks to
significantly improve page load time even if you have large amount
of Javascript used on the page.

Dropthings
– my open
source Web 2.0 Ajax Portal has gone through a technology
overhauling. Previously it was built using ASP.NET AJAX, a little
bit of Workflow Foundation and Linq to SQL. Now Dropthings boasts
full jQuery front-end combined with ASP.NET AJAXUpdatePanel, Silverlight widget, fullWorkflow Foundation implementation on the business
layer, 100% Linq to SQL Compiled Queries on the
data access layer, Dependency Injection and Inversion of Control
(IoC) using Microsoft Enterprise Library 4.1 andUnity. It also has a ASP.NET AJAX Web Test
framework that makes it real easy to write Web Tests that simulates
real user actions on AJAX web pages. This article will walk you
through the challenges in getting these new technologies to work in
an ASP.NET website and how performance, scalability, extensibility
and maintainability has significantly improved by the new
technologies. Dropthings has been licensed for commercial use by
prominent companies including BT Business, Intel, Microsoft IS,
Denmark Government portal for Citizens; Startups like Limead and
many more. So, this is serious stuff! There’s a very cool
open source implementation of Dropthings framework available atNational
University of Singapore portal.

Get the source code

You will need Visual Studio 2008 Team Suite with Service Pack 1
and Silverlight 2 SDK in order to run all the projects. If you have
only Visual Studio 2008 Professional, then you will have to remove
the Dropthings.Test project.

New features introduced

Dropthings new release has the following features:

Template users – you can define a user who’s pages
and widgets are used as a template for new users. Whatever you put
in that template user’s pages, it will be copied for every
new user. Thus this is an easier way to define the default pages
and widgets for new users. Similarly you can do the same for a
registered user. The template users can be defined in theweb.config.

Widget-to-Widget communication – Widgets can send message
to each other. Widgets can subscribe to an Event Broker and
exchange messages using a Pub-Sub pattern.

WidgetZone – you can create any number of zones in any
shape on the page. You can have widgets laid in horizontal layout,
you can have zones on different places on the page and so on. With
this zone model, you are no longer limited to the Page-Column model
where you could only have N vertical columns.

Role based widgets – now widgets are mapped to roles so
that you can allow different users to see different widget list
using ManageWidgetPersmission.aspx.

Role based page setup – you can define page setup for
different roles. For ex, Managers see different pages and widgets
than Employees.

Widget maximize – you can maximize a widget to take full
screen. Handy for widgets with lots of content.

Free form resize – you can freely resize widgets
vertically.

Silverlight Widgets – You can now make widgets in
Silverlight!

Why the technology overhauling

Performance, Scalability, Maintainability and Extensibility
– four key reasons for the overhauling. Each new technology
solved one of more of these problems.

First, jQuery was used to replace my personal hand-coded large
amount of Javascript code that offered the client side drag &
drop and other UI effects. jQuery already has a rich set of library
for Drag & Drop, Animations, Event handling, cross browser
javascript framework and so on. So, using jQuery means opening the
door to thousands of jQuery plugins to be offered on Dropthings.
This made Dropthings highly extensible on the client side.
Moreover, jQuery is very light. Unlike AJAX Control Toolkit jumbo
sized framework and heavy control extenders, jQuery is very lean.
So, total javascript size decreased significantly resulting in
improved page load time. In total, the jQuery framework, AJAX basic
framework, all my stuffs are total 395KB, sweet! Performance is
key; it makes or breaks a product.

Secondly, Linq to SQL queries are replaced with Compiled
Queries. Dropthings did not survive a load test when regular lambda
expressions were used to query database. I could only reach up to
12 Req/Sec using 20 concurrent users without burning up web server
CPU on a Quad Core DELL server.

Thirdly, Workflow Foundation is used to build operations that
require multiple Data Access Classes to perform together in a
single transaction. Instead of writing large functions with many
if…else conditions, for…loops, it’s better to
write them in a Workflow because you can visually see the flow of
execution and you can reuse Activities among different Workflows.
Best of all, architects can design workflows and developers can
fill-in code inside Activities. So, I could design a complex
operations in a workflow without writing the real code inside
Activities and then ask someone else to implement each Activity. It
is like handing over a design document to developers to implement
each unit module, only that here everything is strongly typed and
verified by compiler. If you strictly follow Single Responsibility
Principle for your Activities, which is a smart way of saying one
Activity does only one and very simple task, you end up with a
highly reusable and maintainable business layer and a very clean
code that’s easily extensible.

Fourthly, Unity
Dependency Injection (DI) framework is used to pave the path for
unit testing and dependency injection. It offers Inversion of
Control (IoC), which enables testing individual classes in
isolation. Moreover, it has a handy feature to control lifetime of
objects. Instead of creating instance of commonly used classes
several times within the same request, you can make instances
thread level, which means only one instance is created per thread
and subsequent calls reuse the same instance. Are these going over
your head? No worries, continue reading, I will explain later
on.

Fifthly, enabling API for Silverlight widgets allows more
interactive widgets to be built using Silverlight. HTML and
Javascripts still have limitations on smooth graphics and
continuous transmission of data from web server. Silverlight solves
all of these problems.

Read the article for details on how all these improvements were
done and how all these hot techs play together in a very useful
open source project for enterprises.

ensure allows you to load Javascript, HTML and CSS
on-demand, whenever they are needed. It saves you from writing a
gigantic Javascript framework up front so that you can ensure all
functions are available whenever they are needed. It also saves you
from delivering all possible html on your default page (e.g.
default.aspx) hoping that they might some day be needed on some
user action. Delivering Javascript, html fragments, CSS during
initial loading that is not immediately used on first view makes
initial loading slow. Moreover, browser operations get slower as
there are lots of stuff on the browser DOM to deal with. So,ensure saves you from delivering unnecessary
javascript, html and CSS up front, instead load them on-demand.
Javascripts, html and CSS loaded by ensure remain in
the browser and next time when ensure is called with
the same Javascript, CSS or HTML, it does not reload them and thus
saves from repeated downloads.

The above code ensures Some.js is available before executing the
code. If the SomeJS.js has already been loaded, it executes the
function write away. Otherwise it downloads Some.js, waits until it
is properly loaded and only then it executes the function. Thus it
saves you from deliverying Some.js upfront when you only need it
upon some user action.

Similarly you can wait for some HTML fragment to be available,
say a popup dialog box. There’s no need for you to deliver HTML for
all possible popup boxes that you will ever show to user on your
default web page. You can fetch the HTML whenever you need
them.

You might think you are going to end up writing a lot ofensure code all over your Javascript code and result
in a larger Javascript file than before. In order to save you
javascript size, you can define shorthands for commonly used
files:

While loading html, you can specify a container element where
ensure can inject the loaded HTML. For example, you can say load
HtmlSnippet.html and then inject the content inside a DIV named
“exampleDiv”

Download Code

UFrame combines
the goodness of UpdatePanel and IFRAME in a cross browser and
cross platform solution. It allows a DIV to behave like anIFRAME loading
content from any page either static or dynamic. It can load pages
having both inline and external Javascript and CSS, just like an
IFRAME. But unlike IFRAME, it loads the content within the main
document and you can put any number of UFrame on your page without
slowing down the browser. It supports ASP.NET postback nicely and
you can have DataGrid or any other complex
ASP.NET control within a UFrame. UFrame works perfectly withASP.NET MVC making it an replacement forUpdatePanel. Best
of all, UFrame is
implemented 100% in Javascript making it a cross platform solution.
As a result, you can use UFrame on ASP.NET, PHP,
JSP or any other platform.

<divclass="UFrame"id="UFrame1"src="SomePage.aspx?ID=UFrame1"><p>This should get replaced with content from Somepage.aspxp>div>

Response from SomePage.aspx is rendered
directly inside the UFrame. Here you see twoUFrame‘s are used
to load the same SomePage.aspx as if they are
loaded inside IFRAME. Another UFrame is used to loadAnotherPage.aspx
that shows photos from Flickr.

What is UFrame?

UFrame can load
and host a page (ASP.NET, PHP or regular html) inside a DIV. Unlike
IFRAME which loads the content inside a browser frame that has no
relation with the main document, UFrame loads the content within
the same document. Thus all the Javascripts, CSS on the main
document flows through the loaded content. It’s just likeUpdatePanel with
IFRAME’s src
attribute.

The above UFrames are declared like
this:

<divid="UFrame1"src="SomePage.aspx"><p>This should get replaced with content from Somepage.aspxp>div>

The features of UFrame are:

You can build regular ASP.NET/PHP/JSP/HTML page and make them
behave as if they are fully AJAX enabled! Simple regular postback
will work as if it’s an UpdatePanel, or simple
hyperlinks will behave as if content is being loaded using
AJAX.

Load any URL inside a DIV. It can be a PHP, ASP.NET, JSP or
regular HTML page.

Just like IFRAME, you can set src property of DIVs and they
are converted to UFrames when UFrame library loads.

Unlike IFRAME, it loads the content within the main document.
So, main document’s CSS and Javascripts are available to the loaded
content.

It allows you to build parts of a page as multiple fully
independent pages.

Each page is built as standalone page. You can build, test and
debug each small page independently and put them together on the
main page using UFrames.

It loads and executes both inline and external scripts from
loaded page. You can also render different scripts duringUFrame
postback.

All external scripts are loaded before the body content is set.
And all inline scripts are executed when both external scripts and
body has been loaded. This way the inline scripts execute when the
body content is already available.

It loads both inline and external CSS.

It handles duplicates nicely. It does not load the same
external Javascript or CSS twice.

Download the code

You can download latest version of UFrame along with the VS 2005
and VS 2008 (MVC) example projects from CodePlex:

A web page can load a lot faster and feel faster if the
javascripts on the page can be loaded after the visible content has
been loaded and multiple javascripts can be batched into one
download. Browsers download one external script at a time and
sometimes pause rendering while a script is being downloaded and
executed. This makes web pages load and render slow when there are
multiple javascripts on the page. For every javascript reference,
browser stops downloading and processing of any other content on
the page and some browsers (like IE6) pause rendering while it
processes the javascript. This gives a slow loading experience and
the web page kind of gets ‘stuck’ frequently. As a result, a web
page can only load fast when there are small number of external
scripts on the page and the scripts are loaded after the visible
content of the page has loaded.

Here’s an example, when you visit
http://dropthings.omaralzabir.com, you see a lot of Javascripts
downloading. Majority of these are from the ASP.NET AJAX framework
and the ASP.NET AJAX Control Toolkit project.

Figure: Many scripts downloaded on a typical ASP.NET AJAX page
having ASP.NET AJAX Control Toolkit

As you see, browser gets stuck for 15 times as it downloads and
processes external scripts. This makes page loading “feel” slower.
The actual loading time is also pretty bad because these 15 http
requests waste 15*100ms = 1500ms on the network latency inside USA.
Outside USA, the latency is even higher. Asia gets about 270ms and
Australia gets about 380ms latency from any server in USA. So,
users outside USA wastes 4 to 6 seconds on network latency where no
data is being downloaded. This is an unacceptable performance for
any website.

You pay for such high number of script downloads only because
you use two extenders from AJAX Control Toolkit and theUpdatePanel of
ASP.NET AJAX.

If we can batch the multiple individual script calls into one
call like Scripts.ashx as shown in the
picture below and download several scripts together in one shot
using an HTTP Handler, it saves us a lot of http connection which
could be spent doing other valuable work like downloading CSS for
the page to show content properly or downloading images on the page
that is visible to user.

Figure: Download several javascripts over one connection and save
call and latency

The Scripts.ashx
handler can not only download multiple scripts in one shot, but
also has a very short URL form. For example:

The benefits of downloading multiple Javascript over one http
call are:

Saves expensive network roundtrip latency where neither browser
nor the origin server is doing anything, not even a single byte is
being transmitted during the latency

Create less “pause” moments for the browser. So, browser can
fluently render the content of the page and thus give user a fast
loading feel

Give browser move time and free http connections to download
visible artifacts of the page and thus give user a “something’s
happening” feel

When IIS compression is enabled, the total size of individually
compressed files is greater than multiple files compressed after
they are combined. This is because each compressed byte stream has
compression header in order to decompress the content.

This reduces the size of the page html as there are only a few
handful of script tag. So, you can easily saves hundreds of bytes
from the page html. Especially when ASP.NET AJAX produces giganticWebResource.axd andScriptResource.axd
URLs that have very large query parameter

The solution is to dynamically parse the response of a page
before it is sent to the browser and find out what script
references are being sent to the browser. I have built an http
module which can parse the generated html of a page and find out
what are the script blocks being sent. It then parses those script
blocks and find the scripts that can be combined. Then it takes out
those individual script tags from the response and adds one script
tag that generates the combined response of multiple script
tags.

For example, the homepage of Dropthings.com produces the
following script tags: