Adobe insights about creating, delivering and monetizing digital content across platforms and devices

Tag Archives: palm

Yesterday I spotted a lady here in the UK showing off her Palm Pre to a friend. So today I recorded a video to take a look at Palm’s WebOS Browser with Flash Player 10.1. Please note that the engineering build on this device is not up-to-date, but you can certainly get an impression of how great the experience will be.

For the user experience professionals out there, check out the card metaphor for switching between applications; amazing.

Some of you will remember my earlier blog posts in August 2009 on the HTC Hero, the first Android device to ship with Flash. The goal of the Android experiment was to learn about the Android SDK, the development process and discover how Flash was enabled in the context of the browser.

As we edge closer to the release of Flash Player 10.1 in the first half of this year, it seems appropriate to revisit these posts. In this post you’ll see that there are hidden benefits to using a common runtime across device platforms, some of which are not that obvious.

The result of my week long investigation is that (using the beta version) I can hook up Flash Player 10.1 to the camera; but that’s just the start. Let’s look at how it’s done…

WebView

Those of you familiar with Nokia’s WebRuntime or the iPhone UIWebView will recognize WebView, because it’s the same thing. Really it’s an implementation of the browser made available as a UI component for native applications. So this means that you can create an application in HTML/JScript and manage the user experience through a native shell.

So why is WebView useful? Let’s look at the relevant classes:

WebView – Used to load and display web pages using the built-in device browser and chrome, embedded into your application.

WebViewClient – Enables the handling of various browser actions like page loading and error handling. Overrides the Activity in the built in browser.

WebChromeClient – Enables the replacement of the browser chrome for events like progress, alerts and for window controls. Can override the default Chrome.

So using a standard WebView I can use the core browser, or if I choose, extend this and replace the chrome and user experience. If you were reading carefully, inside WebView you will have noticed a method called addJavascriptInterface, and that’s where it gets interesting because this allows you to create a JavaScript front end to a native class.

In effect you can then write anything you want using native APIs, and add that functionality to WebView in the form of JavaScript interfaces. Wait! Doesn’t Flash have the ability to speak to JavaScript?

Flash Player – ExternalInterface

This is a long standing ExternalInterface API in Flash Player that enables communication with JavaScript in the browser. It’s primary purpose is actually to help Flash live alongside HTML, enabling Flash to signal JavaScript and pass messages back and forth. So in principle, if Flash can communicate with JavaScript and JavaScript with Native code, then we can start adding functionality.

Sounds really simple right?

PhoneGap

The folks at Nitobi have already made a huge head start creating PhoneGap, an open source set of cross-platform Javascript APIs for HTML applications, just some of which are below. The PhoneGap framework also enables anyone to create their own extensions based on the same principle, and although it’s quirky, the process is relatively simple.

In the git repository the team are also hard at work on functionality like Text-to-speech, Camera and File access for Android. Remember that this is an open source project, it’s changing just about every day and they need helpers. What’s really exciting is that PhoneGap workshops, including in person and online events; so I suggest attending!

iPhone

Android

Blackberry

Geo Location

yes

yes

yes

Vibration

yes

yes

yes

Accelerometer

yes

yes

pending

Sound

yes

yes

yes

Contact Support

yes

pending

yes

Porting PhoneGap

As you have read previously, we are working to bring Flash Player 10.1, which includes the ExternalInterface, to Android in the next few months. Now, my Nexus One is an Android 2.1 device, and that creates a slight problem because PhoneGap is only compatible with Android 1.6 APIs.

This means that I had to port code and complete some APIs, in fact I’ve completed the Contacts API, updated the Camera, Location and Accelerometer APIs and did a spot of bug fixing get things working. In total this took about five days, to get setup and familiar with the code, and then writing some code for a few hours here and there. Using Open Source kit often results in these sticky issues, hint, that’s why Adobe put so much effort into documentation

With the Android SDK it really couldn’t be simpler to perform tests once I got started, really it couldn’t be easier to do live debugging on Windows, Mac and Linux; a great SDK from Google. Once you get into the flow of debugging, coding and testing it’s all good.

My advice, as always, is to think about functionality that you’d like to add and go for it. There are lots of docs out there, and alot of helpful code available for hacking something together.

FlashCam

After playing with PhoneGap and testing out my changes/addition, I decided to put the framework to the test and build an application. The first API test was for the Android Camera, and I’m calling it FlashCam.

With FlashCam I have created a simple stub application that contains a WebView, much like the previous blog post. This WebView pulls down an HTML page that’s actually on my blog here, and instantiates the native code and applies the javascript interfaces. So you heard that right, I’m able to extend functionality to a live webpage and not just a static local page.

HTML Integration

Embedded in the example HTML file from PhoneGap is a JavaScript function called show_pic(). It simply wraps up another function inside the phonegap.js that is also referenced in the HTML file. If this function succeeds then dump_pic() is called, if not then the function fail() is called.

The code under getPicture() is really quite complex and beyond the scope of this post, but you can see it here. Essentially it calls a native function called takePicture() written in Java, and this launches a new Activity (like a Window) with the camera’s viewfinder surface.

When the photo is taken we call dump_pic() and it will contain the Base64 encoded JPEG image data from the camera. Then, all we need to do is return this to Flash in a callback and in this case “mySWF”, which is the name of the SWFObject embedded in the HTML file.

In the running SWF file we have a SimpleButton that can call the show_pic() function, but first we need to add a callback which is analogous to an event listener. In Javascript we call photoBytes(image_data), and once the ExternalInterface receives this event it will call onReceivedPhoto(image_data):

public function fl_MouseClickHandler(event:MouseEvent):void{

ExternalInterface.addCallback("photoBytes", onReceivedPhoto);

ExternalInterface.call("show_pic");
}

Here in the onReceivedPhoto function we decode the Base64 encoded image_data, which is just a very large string containing the image. I’m using the Base64 classes provided here by Jean-Philippe Auclair, and merely getting the ByteArray. I can then use Loader to take these bytes and produce BitmapData and do a little scaling in the imageLoaded handler before adding it to the stage within the holder_mc.

So that’s it, it’s really that simple to hook up native code to Flash Player 10.1. I suspect that this will also be possible on Palm’s WebOS, Symbian and Windows Mobile in time. I’m also loosely aware of a better bridge to JavaScript in Flash Player 10.1, so this might be a source of further investigation to improve performance for large data sets.

Now it’s your turn, below are the sources that I’ve used from PhoneGap with my own additions. You can also find the Flash FLA and AS files below, note that you will need at least Flash Professional CS4 to open the FLA.

Flash Player 10.1 will become available in the first half of 2010 for all supported platforms. In fact, the desktop beta 3 is already looking incredible and proving to be a huge hit with developers testing their content.

One of the most important parts of our work with our Open Screen Project OEM partners is to enable the seamless discovery, installation and update of Flash Player 10.1 on device platforms. I know that some have asked questions on this, and so I’m glad to bring you some responses, if a little late.

Extending the reach

In the “marketecture” diagram below you can see that Flash Player 10.1 is extending it’s platform reach, doubling it in fact. It’s worth noting that recent reports around minimum spec’s for Flash Player 10.1 are eluded to here also, because working with our partners, we are targeting the latest chipsets available.

To explain, smartphones have a typical lifespan that is less than half that of a desktop computer, and so hardware choices are made by planning for the future. Over the past few years we have shipped over 1.5Billion devices with Flash Lite using this simple rule.

Therefore the choice to target the ARM Cortex-A8 chipsets will result in greater efficiency, and most importantly a wider range of consistent experiences as uptake grows. To be clear, that uptake is already happening, and it will expand rapidly just like it does every other year.

It’s like a Moore’s Law of mobile phones

Yet some devices will not be able to support the full Flash Player 10.1 due to low hardware capabilities, and for many of those devices we have a new version of our optimized runtime, Flash Lite, to fill the gap. In fact the alpha version has already been spotted running Farmville on Android Eclair here.

Driving the Distribution

Working with our OEM partners we have enabled the Flash Player to be installed in a manner consistent with the desktop experience. When visiting websites that have Flash content, users can click on the “Flash Player required” images/links provided by content developers to begin the installation process.

As with the desktop, the browser then redirects to the Flash Player Download Center, and in the case of mobile phones we pass these requests to the requisite device application stores such as the Android Market. Today’s application stores have extended abilities to correctly identify devices, and to manage the update of applications and plugins like Flash Player. Users can of course visit their application store directly if they wish.

In addition, those users purchasing new devices from a retailer may already have Flash Player pre-installed, made available in over-the-air software updates or through the browser directly.

Delivering Enhancements

As with the desktop install process, with each version of the Flash Player various updates are applied throughout it’s lifetime to ensure a high level of quality. Users can expect these updates to be provided automatically on some platforms via their application store update process, as well as through over-the-air software updates.

In the example above you can see the update notifications that users are familiar with on Android devices, and it is expected that this will be used for Flash Player 10.1 during it’s lifetime. Though I should point out that Nokia have been providing their own update mechanism, directly in the browser for some time now with a huge user uptake.

During the past few days Palm have also begun to lay the ground for Flash Player 10.1 support by delivering their software update.

Player Detection

On the desktop today developers use a combination of methods to detect the Flash Player and version. Recognizing the need for a consistent approach, these same methods can be used on mobile phones in the future. Adobe recommends SWFObject2, an open source project that provides cross-browser support. It is also supported fully within our Creative Suite tools. SWFObject works across browsers and device platforms to detect the Flash Player and it’s version. Should an update been required, the tool can enable the ExpressInstall experience or provide fallbacks as required.

Historically I know some have invested considerable time and energy in device databases such as WURFL, which helped us to accurately distribute application installers to devices. With Flash Player 10.1 we need only be concerned about the browser use case, and so I would argue that we do not need to continue this effort long term.

I hope this information will help you build a picture of the huge efforts that our engineering and product teams have made. The mechanisms for discovery, install and update of the Flash Player 10.1 are a fantastic achievement, even though for some of you these may seem rather obvious.

With the Open Screen Project partners we are literally changing the ecosystem, enabling a more complete and consistent web experience on devices, and driving the industry forward with the Flash Platform in 2010.

It appears that there has been some confusion from the community at large surrounding battery performance. This was caused by my colleague Michael Chaize publishing an amazing video of Flash Player 10.1 demos on Vimeo.

Bloggers from Daring Fireball and Macgasm have spent a little more time than expected studying the battery indicators, as opposed to the incredible advancements in web browsing for mobile phones, netbooks and tablets. To be clear, the battery indicator changes being discussed are a function of video editing and not much else.

That said, let’s look at some mobile facts for fun.

Mobile phones are complicated mini-computers with extremely complex chip designs all working to produce a rich experience with maximum efficiency. It should be no surprise that using 3G, WIFI, Bluetooth, GPS or leaving a browser window open and showing even basic HTML can drain your battery. Additionally, distance from a cell tower is also a potential pitfall and some the travellers among you will note differing battery life in various cities, countries and networks.

For many years we have been working within these constraints, probably without many of you realizing it. Remember that our mobile optimized runtime Flash Lite (shipped on over a Billion phones) and has been used extensively for User Interfaces on mobile phones from Samsung, Sony Ericsson and LG, so this is something that we know quite a bit about.

During our testing of Flash Player 10.1 we have baseline tests against the following use cases (among others), and using a multi-meter to ensure that your content runs with acceptable battery consumption. We’re also testing against the web on sites like youtube, blip.tv and others with great performance reaching to hours of playback on the Nexus One.

Here are the actual combinations of test scenarios carried out at our offices, of course the real world result for you will be different:

Idle – No 3G, Wifi, Bluetooth, IR

Idle – No 3G, Wifi, Bluetooth, IR + backlight ON

3G enabled – Wifi, Bluetooth, IR off

WIFI + vanilla HTML. ’simple.html’

3G + vanilla HTML. ’simple.html’

3G + vanilla HTML file + swf: ’simple-swf.html’

To demonstrate battery performance on the Nexus One here is a recording of a large movie playing on Youtube. It lasts for some 17 minutes with little effect on the battery indicator, and just to ensure fairness I have included the battery usage chart data from the Android OS. Our own tests show that video can be played for well over 3Hours over WIFI from youtube in H.264 (Baseline 1.2).

The resulting battery usage is a mere 6% for the Browser which totalled 199Mb of data received:

Content Optimization

Poorly written content, Flash or otherwise, will perform badly on any platform. Our investments with Flash Player 10.1 and AIR are designed to provide the best results for the majority of existing content for web enablement on devices. However, all of us will have to consider the user experience for our new mobile users. Those of you that have created iPhone applications will know some of the tricks of the trade already

Thibaut (from the video above) has in fact written a fantastic document to lead you through the first steps in optimizing your content. Most of this is applicable to any of the Flash Platform runtimes, and certainly the desktop/AIR/netbooks/tablets etc.

Mike Chambers has also completed a great study on the Touch and Mouse events, and in particular how you can begin to optimize your content for this huge array of new platforms; and ultimately customers.

I arrived back from this years Mobile World Congress far more exhausting than normal, hence the complete lack of blogging during the event; apologies for that. So here I will aim to roundup the various announcements and hopefully add to the huge amount of blogging and articles written during the event.

In general this years event had a more positive mood with attendees and exhibitors all looking to the future of mobile and devices. New this year was the inclusion of Tablet and Netbook devices in the show many of which use the same hardware as high end smartphones. Our goal is to enable the Flash Platform to extend to these devices too, and so it should be no surprise that we had many partners previewing Flash experiences on their hardware, more on that later.

Open Screen Project Update – “Connecting Developers with Consumers”

At the conference we announced that we now have some 70 partners, with new additions like Burda Group, RTL, Stern, Addicting Games and EPIX on the content partner side. New technology partners include Freescale, MIPS, Vizio, Symbian, Imagination Technologies and Sonic. In terms of scale we have created one of the largest initiatives in the mobile ecosystem to build a foundation for the distribution of the Flash Platform across devices. Of course for the community much of this will appear like marketing, but rest assured that there are many hundreds of engineers working globally to bring Flash to a huge array of platforms.

As we begin to wrap up the Flash Player integration efforts our focus has widened to include enablement for web content. In total we’re working with over 140 content provider partners, owners of the largest and most popular sites that use Flash today. At MWC 2010 we showed applications and websites optimized for mobile viewing from the following:

As you can see the list is HUGE and this represents a small part of the overall efforts to bring the full web to devices using Flash Player. For the Open Screen Project to be truly successful we need our content providers and developers to play their part, as you can see, this effort is very much underway.

Of particular note we also joined the LiMo Foundation to deliver the Flash Platform on their Linux based devices. It’s an exciting announcement because it demonstrates that the Open Screen Project and other industry initiatives are aligned with a goal to the delivery of consistent and open platforms for developers and consumers.

Flash Player 10.1

On our stand at the Mobile World Congress we showed a beta version of Flash Player running on the Motorola Droid, Palm Pre and the Google Nexus One. The applications above were running incredibly well and attendees were very impressed with the performance and fluidity of the experience, Flash was built for the web and on devices it just makes sense. The most important addition to the mobile experience was the ability to playback Flash content in full screen mode, enabled by these powerful webkit browsers.

Of course the fun part for attendees was in visiting their own sites to understand the sheer power of the new player. In all, the thousands of attendees that visited our stand were extremely excited about being able to browse more of the web on their devices. For many the call to action was “How can I get started?”, well Thibaut has been crafting a great set of resources with our engineering teams here. The first step however is to create a plan of action, begin to plan a suitable user experience for mobile devices; and in particular those based on touch.

With Flash Player 10.1 we have worked to enable the same web content to run using less memory, yet with higher performance. The results are really incredible and should see a significant improvement, not only for the mobile devices but also for desktop computers and application running on Adobe AIR 2.0. Remember, we can only do so much and as a Flash Developer you have a role to play in ensuring that your content works well on these new devices.

For fun, my colleague Michael Chaize has created his own tour of the web on the Nexus One, he even pushes out all the stops by visiting a site based on Papervision!

In terms of availability we have required some significant patches to Android and to the Web OS to support the installation of the Flash Player over-the-air. We expect these new updates to become available soon to end users, at which time the Flash Player will become available either via their respective Application Stores, within the software updates or both. In either case the installation of the Flash Player will be seamless and ongoing we expect to see significant traction as we move forward.

Adobe AIR on devices

One of the big pieces of news at Mobile World Congress was the announcement of Adobe AIR on Android, and the tie-in with our Adobe Packager for the iPhone. At MAX 2009 we demonstrated the creation of iPhone applications using the upcoming Flash Professional CS5, and the twist is that these applications are based on AIR 2.0. To demonstrate the possibilities we showed some of the same applications created in Flash Professional CS5 that are available on the AppStore today, running on Android devices using Adobe AIR. The workflow couldn’t be simpler, it’s simply a repackaging effort with a little UX tweaking for basic layout and hardware capabilities.

Here is a nice video from Kevin Hoyt showing off Adobe AIR for Android:

Write One Run Everywhere? – In many ways the answer is yes, given a few measured choices. We have made it fantastically simple to reach across devices and platforms with Flash and AIR, and so the same code can now run on a huge array of phones, desktop computers and consumer electronics. Using the same assets and principle it will be extremely simple to tweak applications for varying screen sizes, input methods and user experiences.

So now our “marketecture” diagram for Applications looks like this (2010 is on the right):

Note that I’ve called out iPhone OS separately, this is because you will be required to use Actionscript 3 to target the iPhone. Once you package an application it will not be possible to process additional SWF files containing actionscript, such as those stored on the web, due to Apple’s restrictions around interpreted code.

Android, by comparison, is totally open and AIR runs beautifully on the platform; and dare I say a little faster on the Moto Droid, a device with iPhone 3Gs hardware.

The last time we had a Palm update we saw the Flash Player running in multiple cards (application instances) on Web OS. We had made great progress with the pause and resume functionality, to fit in with the unique nature of WebOs multi-tasking capabilities.

At CES 2010 Palm talked publicly about the availability of the Flash Player 10.1 for their devices. The intention is to ship it as part of an OS update, and also in their App Store so that users can install the player as required (if they already have the OS update for example). So you can start to see the promise of the Open Screen Project taking shape, Flash Player 10.1 updates over-the-air for all Palm users. Additionally Palm indicated that they are going to ship “soon”.

At MAX we still had some work to do with Palm to tidy up certain features, hardware acceleration for video and providing a full screen experience for games and video playback.

In this update video Adrian shows the progress made in the last few months around gaming site optimization.

Palm also demonstrated Flash content in the browser on a Palm Pre and expects a free Adobe Flash Player 10.1 plug-in to be posted to the Palm App Catalog in the coming months. Palm expects webOS to be one of the first mobile platforms to support the full Flash Player in the web browser.

That should mean that it will run on the Palm Pre as well as the new devices announced. Adrian did a demo of Flash Player 10.1 in action on the Palm Pre recently and you can check out the embedded video below.

Write once, deploy anywhere. Sounds nice, no? Sadly it’s not really a reality today. The currently available technologies are very fragmented and today there is no technology out there that makes that dream a reality.
I have a strong feeling that is going to change dramatically in 2010. Some will disagree or say that I am […]

CEOs from ARM, Broadcom, DoCoMo, Google, HTC, Motorola, NVIDIA, Palm, QUALCOMM, and RIM talk about how they’re bringing Flash Platform technologies to their devices and platforms as part of the Open Screen Project and why they think it’s important to have Flash on their devices and platforms.