Life at the bleeding edge (of web standards)

Main menu

Post navigation

jQuery Pure: Call for contributors

This post is about an idea I’ve had for ages, but never found the time to actually start working on it. Maybe because it looks like a quite big project if done properly, so it’s scary to do it on my own without any help.

jQuery has a huge amount of code that deals with browser bugs and lack of implementations. For example, it needs a full-fledged selector engine, to cater for old browsers that don’t support the Selectors API. Or, it needs code that essentially does what the classList API is supposed to do, because old browsers don’t support it. Same goes for nextElementSibling (the .next() method) and tons of other stuff. However, not everyone needs all this. Some projects don’t need older browsers support, either due to the developer mindset or due to their tech-savvy target group. Also, some people only write demos/proof-of-concepts for modern browsers only and don’t need all this code. Same goes for intranet apps that are only designed for a particular modern browser. Last but not least, this code bloat makes the jQuery library hard to study for educational purposes.

However, even in a browser that supports all the modern stuff, the jQuery API is still more concise than the native methods. Besides, there are tons of plugins that depend on it, so if you decide to implement everything in native JavaScript, you can’t use them.

What I want to build is a fork of jQuery that is refactored so that all the extra code for working around browser bugs removed and all the code replaced by native functionality, where possible. All the ugliness removed, leaving a small, concise abstraction that only uses the current standards. Something like jQuery: The good parts. It could also serve as a benchmark for browser standards support.

The API will work in the exact same way and pass all unit tests (in modern browsers, in cases where they are not buggy) so that almost every plugin built on top of it will continue to function just as well. However, the jQuery library itself will be much smaller, with more elegant and easy to understand code.

So, who’s with me? Do you find such an idea interesting and useful? Would you want to contribute? If so, leave a comment below or send me an email (it’s in the about page). Also, please let me know if you can think of any other uses, or if there’s already something like that that I’ve missed.

Post navigation

Considering the amount of effort that goes into maintaining the official jQuery project, I think it would be much more productive for the community at large to contribute to the official codebase, rather than splitting everyone’s efforts.

Also, keep in mind that there are a LOT of cross-browser inconsistencies, which are definitely not limited to just IE6 and IE7, which are fixed as a matter of course in jQuery. If you take those fixes out, what exactly are you supporting? A theoretical “perfect” browser that just doesn’t exist?

http://leaverou.me Lea Verou

I don’t think there’s an issue of split efforts: People interested only in this probably wouldn’t contribute to the regular jQuery codebase and people contributing in jQuery would find it really easy to contribute to something like this (and may I say, pleasurable too).

I’m well aware that even modern browsers have bugs, I spend a big portion of my time reporting them. However, you’re missing the great educational value such a library could have. For example in a university course you’re not really interested in teaching cross-browser differences, because by the time the students put their knowledge to use, those will be very different. However, standards are (for the most part) timeless. Also, someone wanting to get a rough grasp of how jQuery works, would find it much easier to first study jQuery Pure, than to dive deep in all those workarounds that the regular jQuery library has to use.

Also, such a library would be just as timeless as the standards it implements. Browser bugs will eventually be fixed, but standards usually just evolve and don’t change much. The imaginary perfect browser of today will be the reality tomorrow, at least as far as those APIs are concerned.

http://benalman.com/ “Cowboy” Ben Alman

While jQuery has an elegant, easy-to-use API, part of the *point* of jQuery is the fact that it normalizes inconsistent and annoying behavior cross-browser. Not teaching future web developers about not only the necessity, but also the costs involved in handling issues like these does both them and their future employers a huge disservice.

This is because no matter how super-fantastic modern browsers get, there will always be minor (or major) implementation bugs which libraries such as jQuery will have to normalize. Web development is far less about theory and more about practical application.

To your point, I can see taking parts of jQuery and fashioning them into materials more appropriate for teaching standards and specifications like DOM or XHR. Teachers and trainers already do this, when necessary. Still, I regularly take students directly through parts of the jQuery source, which is great because they can see jQuery patterns put to work inside jQuery itself.. and they can also see some of the ugliness that is unavoidable when things don’t work exactly the same across all browsers.

http://leaverou.me Lea Verou

I don’t disagree that caring about browser inconsistencies is crucial to regular web development. What I argued for, is that if we teach university students the browser inconsistencies of today, it will be useless knowledge in a few years, when they actually start working. That’s why in the course I’m teaching, I only teach them the standards and feature detection techniques, so that they can work around the bugs of tomorrow themselves. Teaching the real jQuery source could be useful for some kinds of classes, but most definitely not all.

Also, such a simplified jQuery could serve as a benchmark for browser standards compliance.

http://twitter.com/zachleat Zach Leatherman

Personally, I learn by doing. Where would this theoretical library run, where I could actually use it and play around with it?

It’d have to tie to at least one implementation, and it seems inevitable that the implementation is going to differ from spec in some way, taking away some of the educational benefit of the library.

That being said, it would be fun code to look at and review. It kind of seems like the end result would end up to be very similar to zepto.js, even though the goals of the two projects seem very different.

http://twitter.com/paul_irish Paul Irish

So, I think there is a future where feature detection and UA sniffing go hand in hand.

Thus far one of the biggest dangers of UA sniffing is that everyone does their own sniffs. And they fuck them up. (Same thing with writing your own feature detects, unfortunately)

But one area where the two combine is something along these lines.. The JSKB project already hinted at the potential here:

So essentially the server would detect what UA is coming at it (with a reputable, community-driven UA parsing library), match that against the feature tests that have already been run (like what’s already in BrowserScope/Modernizr).. and then strip out whole entire chunks of code that are totally unnecessary to send down the pipe to this user agent.

This solves both the bandwidth problem along with the fact that we want to support different agents but send as few bytes as possible.

It’s definitely where I see things going and am looking forward to working with people on making it happen.

http://mathiasbynens.be/ Mathias

Isn’t the biggest danger of UA sniffing the fact that the UA string itself is unreliable? Even the most robust platform detection library won’t be able to change that.

https://profiles.google.com/adeel.e Adeel Ejaz

I agree with Mathias and mainly because of my experience. I did try UA sniffing. Yes, I hang my head in shame but I learnt something really nice from it. There was a toolbar for IE8 that used to add a piece of string to UA string which had “IE6″ in it! That’s when I learnt UA sniffing is not the way forward, for me at least.

Moreover, lets assume, we have an awesome UA sniffing system, still to make it work the server needs to a bit of heavy lifting. Over the top of my head, you will not want to read files on each request so you will probably have to use cache like apc or memcache (for PHP at least). Over on library like jQuery, you can end up with a may be a 10+ versions of the same file in the cache. Either this or you store 10+ versions of jQuery in the file system. Hmmm.

http://twitter.com/paul_irish Paul Irish

True. def does take a hit on cache and then your Vary header means proxy caching is busted. Indeed.

http://blog.yoav.ws Yoav Weiss

What if we’d think about it the other way around? Client-side UA sniff+feature detection snippet can determine the optimal/minimal JQuery lib and request that with specific URL parameters.
Then you can avoid downloading unused code and benefit from caching.
The main hurdle is creating the various JQuery builds that only have the code to support a specific browser/feature-set.

http://twitter.com/RonnyOrbach Ronny Orbach

That’s better IMO. Actually, the thing we’re *really * looking for is conditional comments. Sorry if anyone’s used to auto-bash any IE invention, but these, along with compatibility mode changer for testing, is actually pretty good.
Why don’t we turn CC into a standard? (apart from breaking the current cc_on patchy browser detections).

Other than that, MooTools and Ender are already modular and it’s awesome, and the closest things to what @LeaVerou:disqus talks about are MooTools and Modernizr 2.0 custom build URLs. But I’ve yet to see an implementation which tries to detect feature/browser and load a specific build accordingly.

Today you can be quite sure that this file is cached on most users’ machines: http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js.
I have a feeling that jQuery knowingly took the canonical file approach, which leverages server and client side caching better, instead of trying to modularize and make all these efforts which eventually might degrade performance instead of helping it.

http://twitter.com/_munter_ Peter Müller

That main hurdle is easily overcome with a proper build system. A guy at my company has been working some time on our now open sourced build system, which already builds multiple static versions of our javascript in different languages. If the jQuery syntax of browser conditions are consistent we should easily be able to build several versions based on those conditions.
The difficulty that I see is actually serving the right bundle to the right browser.

@ommunist

Is it really difficult? A technique like browser classes as it is featured by Kenneth Reitz in his skeleton theme for wp can do the job. Php detects and loads as well. Although there are purist opinions that one could find really difficult to maintain project that depends from UA string updates. Paul Irish here already mentioned that open source, community driven UA library is a right answer here.

http://twitter.com/xkit Garrett Smith

Couple the server and client. That’s great. And when you get a UA string that doesn’t exist in your lookup table, you do what? Send an apologetic “error” page? A “fallback”.

What do “Mozilla/5.0″ and “compatible” have to do with MSIE?

Attempting to derive meaning from something that should never be treated as a source of information, and cause browser manufacturers headaches in doing so causes headaches with browser manufacturer’s having to adjust their UA strings (Opera 10).

If the “bandwidth” problem is caused by using one or more bloated libraries, doesn’t it make more sense to figure out a way to not do that?

Get out from under that rock, Paul!

http://twitter.com/desandro Dave DeSandro

I’d recommend reading Javascript Microframeworks and The Future by Thomas Reynolds, published last month. It addresses the same concerns you voiced – code bloat due to legacy browser support, alternative frameworks, and what’s to come.

http://mathiasbynens.be/ Mathias

Sounds like you’re looking for a less WebKit-specific version of Zepto.js with some extra features

http://strangepants.com/ Jachin Sheehy

Bump. This was my first thought as well.

http://twitter.com/zachleat Zach Leatherman

Jinx. Though the timestamps say you win.

https://profiles.google.com/adeel.e Adeel Ejaz

I had been thinking about something “similar” for a long time. My thought process was as a developer sometimes I know which browsers I plan to support but I have to work the full jQuery library to get the benefits. The best example I can give you is sizzle. I have no doubt in believing its the best piece of code in the world for the job. But if I’m careful (and intelligent (thank you to Paul’s 10 things I learnt from jQuery Source)) enough to not use it, why exactly it is in jQuery to act as just extra bytes that I don’t need. The same I can say about lots of other stuff like in at least my professional projects, I hardly have ever used functions like fadeIn(), fadeOut().

So with that in mind, I think it will be uber awesome if we just don’t take stuff out that we don’t need but actually make an intelligent “custom” jQuery build tool! If I just need ajax, basic selections, and support for Chrome, Safari and Firefox 4, all I do is tick boxes that I need and my build is just of that code. I can understand it will be mammoth task but I’m really up for it if it gives me clean jQuery.

PS: I know my comment goes in a slightly different direction, but I think instead of just choosing what makes into a “clean” jQuery library, we let the developer choose. If you do not want to support those extreme level cases, the build will simply not have them.

http://leaverou.me Lea Verou

I believe this is the direction the regular jQuery project is heading anyway. So I’ve heard at least

http://twitter.com/rwaldron rick waldron

@LeaVerou:disqus , Just a heads up to clarify that the jQuery project has no plans either now or in the future to “modularize” the library. Keep writing the good write!

http://leaverou.me Lea Verou

Nice to know Rick, thanks.

http://twitter.com/rwaldron rick waldron

No problem

If you’re wondering what _is_ in the works, this is the 1.7 roadmap-approved ticket list: http://bugs.jquery.com/report/501

Some highlights include: events cleanup & api simplification, a goal to shim HTML5 elements, deferreds upgrade with abstracted callback list queue. Some quiet release goodies from 1.5 & 1.6 will be documented as well, including attrHooks, propHooks and ajaxHooks (need a special transport? this is the place).

http://twitter.com/Flocke Jens Grochtdreis

Its a pity that the jQuery-team doesn’t want to modularize the library. Most of it will be unused in projects. So why load unused and therfor unwanted code? Mootools and YUI lead the way. Please follow. In the past jQuery was bloated with every new release. That’s no good sign.

Vassilis Mastorostergios

jQuery is in talks with the Closure Compiler team (http://goo.gl/8UBO) to do just that. Have faith

http://paulirish.com Paul Irish

As Rick just said, jQuery has no plans to modularize the core of the library, bro.

Vassilis Mastorostergios

I’m just reporting what I saw in the last jQuery core meeting couple of weeks ago. Perhaps a core dev in that meeting could clarify.

http://twitter.com/rwaldron rick waldron

I was also in that meeting, as I am every Monday. The goals with Google Closure Compiler are to support Advanced Mode and all the capabilities that it affords. http://code.google.com/closure/compiler/docs/api-tutorial3.html

Vassilis Mastorostergios

Thanks for the clarification (although in my eyes, people were pretty enthusiastic about “modularity without the silly modules”). So we can’t expect a “modular” core even with Closure Compiler.

Why the middle step of having to check boxes? Why not just introduce a dependency system that you can hook into directly in your own code, and that a build system can read and build bundles from.

After all, what we are probably interested in in the end is to serve a custom built library with your own code as a single bundle. Have a build system build this entire bundle. We already do this a my office, where most of our current javascript code base is Ext JS. We hook into their dependency system with out open sourced build tool https://github.com/One-com/assetgraph

That pretty much solves your custom build, without the manual interaction step of checking boxes and going back to test if you checked the right ones. jQuery just needs to be modular with a clear dependency system, then any build tool can hook into that.

Christos Constantinou

Good news, it has been done already! It is called jQuery Mobile. Try it with Safari.

http://leaverou.me Lea Verou

That’s not what I’ve been talking about. jQuery Mobile still has to work around bugs and lack of implementations, it’s just focused on different browsers.

Artemi

One way I could see this implemented is if jQuery Pure was the first include, followed by inclusion of user-agent sniffing library that would decide to include the appropriate “jquery patch” for that browser. The jQuery patch would literally replace (and add) the pure functions with browser-specific versions. This may increase code size slightly for IE6 users, but will reduce it for Webkit users and so on. The goal would be to get browsers to pass all the pure tests over time, thus reducing their “patch” size.

Angel Grablev

Actually there have been a few jQuery rewrites for mobile which are basically an extremely overly simplified and clean implementation of jQuery and would be a much better starting point

Lucien Immink

jquery pure? native jquery!

http://profiles.google.com/bhofmann.hotmail.co.uk Bernhard Hofmann

There are times when the target browser is known and well defined. Our internal web app has to target, wait for it, IE6. Yeah, I know, FML and all that.

I’d welcome a customisation of the jQuery build that allows a list of target browsers to be specified, and then the build would only incorporate the features or workarounds for the target browsers. Rather than fork each one, I see the solution as compiler directives to include the IE6 hacks only if IE6 is in the list of target browsers.

http://twitter.com/Flocke Jens Grochtdreis

Hi Lea,
I don’t think it is good to get rid of browser normalizations inside jQuery. That is the most important part of this library. But I do think it would be very important to modularize jQuery. We should be able to pick the parts we need. Personally I think I never used more than 20% of jQuery.
So jQuery should walk the path YUI and mootools walk for years.
So jQuery would be leaner, smaller, better for performance and better for mobile.

http://leaverou.me Lea Verou

Hi Jens,
I’m not suggesting that it’s good to get rid of them in the regular jQuery, of course they are very important. I’m suggesting we do that in a separate branch, since it’s only useful for very specific purposes outlined in my post above.

http://twitter.com/mrgnrdrck Morgan Roderick

I’ve been toying with some similar ideas, under the working title of Inconvenient jQuery.

My idea was to see how much of the codebase that is infrequently used and can be safely removed.

Instead of investing a lot of time on working on something that ultimately will just become part of a discussion of how jQuery gets bigger and bigger, I’ve decided that my time is better spent trying to contribute to open source projects (like jQuery) and learning more pure JavaScript.

I agree with previous commenters, that there would be a lot of value in contributing to the original jQuery project. Contributions could be many things, I consider your blog post a contribution, as the discussion of jQuery is important. Other contributions could be to help modularise jQuery, so it would be easier to drop features in future versions or allow pick-and-mix scenarios in the future.

While there seems to be no ambition to modularise jQuery, I think that it’s important for the future of jQuery and would need someone driven and meticulous to spearhead the propaganda effort it would take.

http://twitter.com/higgo85 David Higgins

In the words of John Resig:

“if you’re particularly excited about
breaking jQuery down into little
chunks you can grab the individual
pieces from SVN and build a custom
copy.”

http://www.facebook.com/avryhof Amos Vryhof

I have been dreaming up an idea for a modular DIY JavaScript Framework “kit” that starts out with letting you choose from a few selector engines, or the selectors API, then pick and choose what functionalities you want to include with it.

Not really based off jQuery, but designed to mimic how it works.

Ideally, I would have radio buttons to choose a few different ways of implimenting certain functionality, and the ability to use/add your own. (that you can choose to add to the snippet library)

I’m hoping to have enough free time someday to start working on it because I think it would be very fun and useful for those who want to have a framework with just the functionality they want.

Barry van Oven

Before actually starting the fork, wouldn’t it be advisable to first set up a proof of concept? For example, strip the latest jQuery version from all cross-browser cruft and streamline the code to work with the most advanced browser engine to date including any and all performance optimizations. Then, compare both code and performance metrics to determine whether or not Pure jQuery has merit.

http://leaverou.me Lea Verou

Performance is not the only reason for jQuery Pure. Also, what you’re describing would take about the same effort as jQP itself.

http://twitter.com/i_like_robots Matt Hinchliffe

I often have similar ideas but I’m always dissuaded by the fact it would mean more project segmentation. Perhaps if ZeptoJS supported a wider range of animations it may fit your parameters quite well and if not then it can always be forked as the codebase seems pretty good. I’m looking to use it as a light jQuery alternative on some projects as it maintains a very similar syntax that’s easy for jQuery devs to jump in to.

http://twitter.com/i_like_robots Matt Hinchliffe

I often have similar ideas but I’m always dissuaded by the fact it would mean more project segmentation. Perhaps if ZeptoJS supported a wider range of animations it may fit your parameters quite well and if not then it can always be forked as the codebase seems pretty good. I’m looking to use it as a light jQuery alternative on some projects as it maintains a very similar syntax that’s easy for jQuery devs to jump in to.

Anonymous

I had a similar thought, which is why I started this: https://github.com/timw4mail/kis-js

http://leaverou.me Lea Verou

Nice! But like Zepto and many others, it doesn’t have exactly the same API and structure as jQuery, so the goals mentioned in my original post would fail with those.

http://twitter.com/gresleyalan Alan Gresley

This sound like a good project. It would mean that I can learn jQeury quicker. I can basically create the blocks needed in CSS3 animation that could be queried but I still a novice with script.

http://twitter.com/stofke72 Kristof Polleunis

Considering that a lot of sites these days use jquery served by a cdn like google or microsoft and that it’s in most browser caches anyway so it’s not always download over and over again, I’d say that little bit of bloat is a good tradeoff.

What is more important than size I guess is how much this bloat affects performance.

I would prefer that the browsers themselves would natively include the major javascript libraries and would actively update them. With an option of course to overwrite it with a specific version if you need to.

Then you’d never have to worry about adding it to your code. But for study purposes it would be easier if it were modularized.