Posts Tagged ‘INTERACTIVE’

Microsoft has finally laid out their plans. Silverlight is alive with Silverlight 5 / 3D powered by XNA (any surprise? guess it isn’t dead) and hardware acceleration throughout Internet Explorer for canvas/html5 focus apparently aloongside Silverlight for deeper graphical needs or changes. It was originally outlined back in April at MIX 2011, but it shows their path and answer to WebGL/Canvas and standards for interactive and game development on the web (plus I have been busy on two titles, one for iOS and one for both iOS and Android, gotta say it is nice to see how fast your app gets posted to Android store compared to iOS…).

In the latest Platform Preview we support all Canvas element APIs and most Canvas 2D Context APIs and attributes.

Features Partially Implemented

Canvas

globalCompositeOperation

The latest Platform Preview does not include support for the globalCompositeOperation attribute.

DOM Exceptions

The latest Platform Preview does not include support for Canvas 2D Context DOM Exceptions.

drawFocusRing()

The latest Platform Preview does not include support for the drawFocusRing() Focus management API.

This is pretty amazing even though it has been hinted at by other news (previously from AMD). Why should we care what Internet Explorer is up to? Well the dream of standards across web browsers seems to be materializing for html5 and more importantly, canvas 2d.

Even with Silverlight Microsoft has decided to join the party and upgrade the web on some great standards to build even more innovative platforms on top of. Some may see this as a death knell for Silverlight, Flash etc but I do not see it that way. I see <canvas> as a competing interactive technology but many times technologies bind together for a better experience, they also drive one another to innovate.

Much like Silverlight pushed Flash, and Silverlight was created because of Flash, those two technologies brought on canvas 2D and more graphical capabilities for the web in the interactive, game and application space. As javascript execution has sped up so has the graphical capabilities of browsers now. What is not to like about that if you are an interactive developer?

Canvas, Flash, Silverlight are all for the most part still software/CPU accelerated. The question is who will start the hardware acceleration of canvas and competing technologies even further to bring us closer to OpenGL ES/WebGL in the browser?

Ars Technica states that IE9 will have hardware accelerated canvas in addition to SVG but that doesn’t seem to be officially stated anywhere by Microsoft yet that I can find. AMD has hinted at it and previous news about SVG being hardware acclerated. Time will tell and it will be a HUGE boost to the browsers that do, of course we need all of them to do it to be worthwhile for mainstream content.

Ars on the hardware accelerated canvas support:

What does come as a surprise is canvas support. Microsoft has been promoting Internet Explorer 9′s support of SVG, which provides vector graphics capabilities within the Web browser, but thus far has kept quiet when asked if it would support the canvas bitmap graphics specification. Not only is canvas being supported, it is also being hardware accelerated, continuing Microsoft’s efforts to give Web applications the ability to exploit the extensive hardware capabilities of modern PCs.

Of course we should tread carefully here, there is still a big chance that portions of the canvas 2d spec will not be implemented exactly the same or some browser may have missing features much like CSS and javascript evolution. For instance the “most Canvas 2D Context APIs and attributes” is something I hope is addressed in the final IE9. If you are gonna spend the time implementing a standard, do it fully and right, don’t try to break it (an old Microsoft tactic). But this step was needed to again push interactive web technologies to more closely compete with desktop graphic technology which adds some really exciting times ahead.

Google is pushing Web Sockets into Chrome based on the Web Socket standards being developed at all major engineering standards groups. Web Socket is an interesting direction but it is great to couple that with O3G or WebGL for some multiplayer 3d game development with just a browser.

Starting in the Google Chrome developer channel release 4.0.249.0, Web Sockets are available and enabled by default. Web Sockets are “TCP for the Web,” a next-generation bidirectional communication technology for web applications being standardized in part of Web Applications 1.0. We’ve implemented this feature as described in our design docs for WebKit and Chromium.

Flash has long been the answer for sockets for web applications and once sockets were added to Flash it instantly became a better interactive and gaming platform for multi-user applications and multiplayer games. They started with XmlSocket then recently added Socket for raw binary data in as3. Silverlight and Java also have this feature but having this in script is pretty significant because many applications could really use a browser supported bi-directional communication link.

What is Missing

The biggie missing from Flash, Silverlight, etc and Web Sockets is UDP and preferably RUDP or Reliable UDP which allows UDP datagrams to be sent back and forth either verified delivery or broadcast. Unity does support UDP. The best socket layers are reliable UDP based because mixing TCP and UDP can lead to queuing and not all messages are critical so having just UDP isn’t enough, having TCP is too much. Reliable UDP is the way to go but so far no web layers are doing it well except Unity on that one (you still have to make your own RUDP implementation – libraries like Raknet or enet in C/C++ give you this but you can’t use that in Unity client only on the server). (Edit: Flash does have RTMFP which is based on UDP and uses FMS for nat for p2p but it is still not a true low level UDP socket just yet as it supports more features. A low-level UDP socket would also be nice in flash.)

Web Communication Evolving

I am a big Flash fan and have been developing it since 1999 among other platforms, I have recently watched other technologies nearly match the features and some go beyond it. The interesting thing about Web Sockets is that it does go after a core feature of flash; Canvas and WebGL or O3D also do. Flash still has the webcam, mic, sound mixers/tranform, and for now sockets which put it at an advantage in gaming and interactive. Flash used to be the sole greatest video player but Silverlight is doing a pretty good job of that as well so that is still an advantage but others are entering including possibly browser support in html5. I still think it is the best video but they would need to keep innovating.

Another interesting point about this is XMLHttpRequest objects. Originally “AJAX” was created by Microsoft for IE, pushing new features and innovating back when IE was a good browser and ahead in IE4. Mozilla and others adopted this feature (as well as editable text areas for html) because they were great features for web applications to evolve to. now Google is pushing with Chrome and Web Sockets is the next step that should be in web browsers even if it is only TCP based for now. This will add great capabilities and will probably be preferred over AJAX/XMLHttpRequest for really interactive and real-time tools/games should it take hold. Ian Hickson is running the table on the standards with this effort and it is a good one to get behind.

There are a few things that piqued my interest in Silverlight 3 beta notes and that is hardware acceleration on video, 3d canvas and pixel effects (unclear if these are hardware accelerated like shaders or pixel bender) similar to filters and with the possibility to write effects similar to shaders or Adobe’s Pixel Bender pbx shaders.

Support for Higher Quality Video & Audio. With support for native H.264/Advanced Audio Coding (AAC) Audio, live and on-demand IIS7 Smooth Streaming, full HD (720p+) playback, and an extensible decoder pipeline, Silverlight 3 brings rich, full-screen, stutter-free media experiences to the desktop. New and enhanced media features in Silverlight 3 include:

Live and on-demand true HD (720p+) Smooth Streaming. IIS Media Services (formerly IIS Media Pack), an integrated HTTP media delivery platform, features Smooth Streaming which dynamically detects and seamlessly switches, in real time, the video quality of a media file delivered to Silverlight based on local bandwidth and CPU conditions.

More format choice. In addition to native support for VC-1/WMA,Silverlight 3 now offers users native support for MPEG-4-based H.264/AAC Audio, enabling content distributors to deliver high-quality content to a wide variety of computers and devices.

Extensible media format support. With the new Raw AV pipeline, Silverlight can easily support a wide variety of third-party codecs. Audio and video can be decoded outside the runtime and rendered in Silverlight, extending format support beyond the native codecs.

Perspective 3D Graphics. Silverlight 3 allows developers and designers to apply content to a 3D plane. Users can rotate or scale live content in space without writing any additional code. Other effects include creating a queue in 3D and transitions.

Pixel Shader effects. These software based effects include blur and drop shadow. In addition, you can also write your own effect. Effects can be applied to any graphical content. An example would be to make a button appear depressed on rollover you could use a drop shadow effect on the pressed visual state.

Bitmap Caching. Silverlight 3 dramatically improves the rendering performance of applications by allowing users to cache vector content, text and controls into bitmaps. This feature is useful for background content and for content which needs to scale without making changes to its internal appearance.

New Bitmap API. With Silverlight 3, developers can now write pixels to a bitmap. Thus, they can build a photo editor to do red eye correction, perform edits on scanned documents or create specials effects for cached bitmaps from elements on the screen.

Themed application support. Developers can now theme applications by applying styles to their Silverlight 3 applications and changing them at runtime. Additionally, developers can cascade styles by basing them on each other.

Animation Effects. Silverlight 3 provides new effects such as spring and bounce. These make animation more natural. Developers can also now develop their own mathematical functions to describe an animation.

Enhanced control skinning. Silverlight 3 provides easier skinning capabilities by keeping a common set of controls external from an application. This allows the sharing of styles and control skins between different applications.

Improved text rendering & font support. Silverlight 3 allows far more efficient rendering and rapid animation of text. Applications also load faster by enabling the use of local fonts.

60+ controls with source code : Silverlight 3 is packed with over 60 high-quality, fully skinnable and customizable out-of-the-box controls such as charting and media, new layout containers such as dock and viewbox, and controls such as autocomplete, treeview and datagrid. The controls come with nine professional designed themes and the source code can be modified/recompiled or utilized as-is. Other additions include multiple selection in listbox controls, file save dialog making it easier to write files, and support for multiple page applications with navigation.

Deep Linking. Silverlight 3 includes support for deep linking, which enables bookmarking a page within a RIA.

Search Engine Optimization (SEO). Silverlight 3 enables users to solve the SEO-related challenges posed by RIAs.By utilizing business objects on the server, together with ASP.NET controls and site maps, users can automatically mirror database-driven RIA content into HTML that is easily indexed by the leading search engines.

Enhanced Data Support Silverlight 3 delivers:

Element to Element binding : UI designers use binding between two UI properties to create compelling UI experiences. Silverlight now enables property binding to CLR objects and other UI components via XAML, for instance binding a slider value to the volume control of a media player.

Data Forms. The Data Form control provides support for layout of fields, validation, updating and paging through data.

New features for data validation which automatically catch incorrect input and warn the user with built-in validation controls.

Support for business objects on both client and server with n-Tier data support. Easily load, sort, filter and page data with added support for working with data. Includes a new built-in CollectionView to perform a set of complex operations against server side data. A new set of .NET RIA services supports these features on the server.

Improved performance, through:

Application library caching,which reduces the size of applications by caching framework on the client in order to improve rendering performance.

Binary XML allows communication with the server to be compressed, greatly increasing the speed at which data can be exchanged.

Local Connection This feature allows communication between two Silverlight applications on the client-side without incurring a server roundtrip: for instance a chart in one control can communicate with a datagrid in another.

Advanced Accessibility Features. Silverlight 3 is the first browser plug-in to provide access to all system colors, allowing partially-sighted people to make changes such as high contrast color schemes for ease of readability by using familiar operating system controls.

Out of Browser Capabilities. The new out of browser experience in Silverlight 3 enables users to place their favorite Silverlight applications directly onto their PC and Mac, with links on the desktop and start menu—all without the need to download an additional runtime or browser plug-in. Further, the new experience enables Silverlight applications to work whether the computer is connected to the Internet or not—a radical improvement to the traditional Web experience. Features include:

Life outside the browser. Silverlight applications can now be installed to and run from the desktop as lightweight web companions. Thus, users can take their favorite Web applications with them, regardless of whether they are connected to the Internet or not.

Desktop shortcuts and start menu support. Silverlight applications can be stored on any PC or Mac computer’s desktop with links in the start menu and applications folder, and so are available with one-click access.

Safe and secure. Leveraging the security features of the .NET Framework, Silverlight applications run inside a secure sandbox with persistent isolated storage. These applications have most of the same security restrictions as traditional web apps and so can be trusted without security warnings or prompts, minimizing user interruptions.

Smooth installation. Because Silverlight applications are stored in a local cache and do not require extra privileges to run, the installation process is quick and efficient.

Auto-update. Upon launch, Silverlight applications can check for new versions on the server, and automatically update if one is found.

Internet connectivity detection. Silverlight applications can now detect whether they have Internet connectivity and can react intelligently including caching a users’ data until their connection is restored.

The great news is we have all major companies about software, mobile and the web are focused on interactive development. If you are an interactive developer with programming skills and design skills, this is the time. Adobe, Microsoft, Apple, even Google with Chrome (javascript engine ineteractive focused), and others are all on development that suits needs and requires skills of people that know the interactive and web platforms and are able to develop the best solution with the best technology for that solution. The programming depth is getting deeper (Adobe’s Alchemy, Silverlight/Moonlight/Mono/C#/Unity3D) but the capabilities are growing exponentially with what you can do with these new markets.

There are so many new, emerging and re-newed market forces in interactive development that things are going to shake up a bit and there is plenty of opportunity no matter what platform you might be locked into. However I recommend not locking yourself into one platform and exploring, but specializing in what you do best. There has never been a better time for developers looking to take the web to the next level with cooler game development visuals, more immersive virtual spaces and applications that have usabililty and design in new ways and mimic the great usable design of the iphone; for developers going mobile or specializing in web game development this is a good time to be in the game.

*drawlogic is authored by Ryan Christensen of *drawlabs and *drawcode, both dedicated to taking ideas to ship doing entertainment focused web, mobile and desktop game and interactive development projects.