Development Blog

Menu

The wearable revolution is upon us! This is something we believe will be great for Unified Remote since it is all about convenience. The goal has always been to create the one and only remote you need. However if you can not find the remote you need what’s the point then?

That is why we are putting Unified Remote on your wrist.

We have been looking into several different wearable platforms like Pebble that have an unofficial plugin for Unified Remote that you find here. However the first to get official support is Android Wear.

The challenges of small devices

Obviously there are several challenges to overcome when you design an app for a small device. However we did not want to limit the amount of features available to the user. The goal is that you will be able to do everything that is possible in the phone version from the watch. Even the mouse is available. In this beta release this is mostly true however there are some limitations.

Even if you can do everything we believe most people will not use the full app. Because that will in most cases take more time that finding the phone and pressing the correct button. We have solved this problem in two different ways.

Notifications

The big thing in Android Wear are notifications. These are the easiest features to access unlike apps that are hidden. Also most users only use a very small subset of actions. That is why we created an permanent notification where you can open the app, do an action by voice (more on this later).

It is also possible to access the Quick Actions from the phone by swiping to the right in the notification. If you like you can of course remove this notification whenever you like by swiping to the end and press the “Mute app” button. To activate it again you need to open the Android Wear application on your phone and remove it from the muted apps.

Voice

The second way to interact with the Wear app is by voice. Since voice is one of the primary input methods on Android Wear it is natural that Unified Remote has good support for voice. This is however not a trivial thing to achieve because Android Wear voice APIs are designed for natural voice and not for finding what application you like to use especially when the names of the applications are a bit strange. So we needed to create a method that would be able to guess what remote you like to use even if the voice APIs did not give the correct names.

What we ended up doing is evaluate the string coming from the voice APIs with the help of several phonetic algorithms and then compare the results to the real names using edit distance. This method gives a reasonably accurate (estimated to over 95%) result as long as the Voice APIs returns something that is close to what you said.

Voice can be reached from any screen in the app. If you press the speak button in the notification or in the list of remotes you can say the name of a remote and that remote will then open or you can say a name and an action in that remote and the action will be executed. However if a remote is already open and you press the speak button the voice action is context aware and you only have to say the action.

We hope you are as excited about this as we are. The app is available in our Unified Remote beta channel. Let us know what you like and even more what you do not like.

Building the widget

Apple’s Programming Guide for Extensions is quite good at providing a quick intro into how building extensions work. However, it falls somewhat short at explaining the details and especially some of the issues one might encounter. So in this post I will cover what I did to get the Widget we’re using to work and some of the pitfalls I found on the way.

Running the widget

Running the widget turned out to be quite the pain. XCode creates a separate target for your Widget but unfortunately it doesn’t provide good instructions on how to use it. First of all, running the widget target won’t unlock your phone like a regular app does, instead you have to have the device unlocked and to be safe have it on the home screen. While it’s gotten more stable with recent XCode betas there are still some issues and from time to time the widget simply won’t launch.

Once you launch the app, you’re presented with a dialogue asking you what app to run. At first this feels very odd but there is actually only one choice for a widget and that is the today app (If it’s not found at the top simply search for it).

A neat trick that works in the current beta of XCode at least is to edit the widget scheme and selecting Other… under executable then simply closing the dialogue. This will set the executable to Today-View, avoiding you the extra step each build.

Another interesting issue you can encounter is the error “No application exists at /Today-View”. This is also solved by editing the scheme making sure that “Debug Extensions and XPC services” is checked, as described on StackOverflow.

Sharing data between the extension and its app

This part is described quite well in Apple’s documentation however it’s lacking a clear step-by-step instruction. What you need to do is set up an App Group in the Member Center. This is a completely new feature which will force you to recreate all your provisioning profiles, since it involves changing properties in your App ID, so beware of that.

Once you’ve done that (The process is quite straightforward and the Member center guides you through it) you need to enable groups in your app. To do this, go to your app’s target and select the Capabilities tab. From here enable App Groups and check the group you created and XCode should do the rest for you. Don’t forget to do this in both your app’s target as well as the widget’s.

It’s important that the ID you enter as suiteName is the same as the one you specified in Member Center. You can then treat the NSUserDefaults just like you normally do, with the only difference that you create a new NSUserDefaults object each time instead of fetching it using [NSUserDefaults standardUserDefaults]. This confused me a lot at first, as I was looking for some method to get the singleton NSUserDefaults and recreating it each time made no sense. However this seems to be standard practice and it’s apparently the way it’s supposed to work.

Launching your application

To launch the containing application you use the URIs defined by your app. However, instead of calling UIApplication‘s method openURL: you have to use a method in NSExtensionContext. This is somewhat poorly documented but the object is available as a property through the category NSExtensionAdditions which is imported by default in your extension.
The following code opens a remote in our widget for example:

Conclusions

All in all, extensions are still a bit tricky and not that well documented. This part of Stack Overflow is not as well-stocked with questions and answers as one might be used to and even more notable is the absence of experts so far. Getting answers to a question is still hard. All of this, combined with a lot of XCode and iOS 8 bugs (Like Springboard crashes every 4 or 5 launches) means that working with extensions can be a quite frustrating experience. However it can also be an interesting one since it’s quite uncommon these days that there is something new to discover which no one else has tried before when it comes to iOS. I hope this blog can help some developers out there who are working with widgets and if you have any questions feel free to drop a comment and I’ll try to answer as best I can.

With the recent announcement of the iOS 8 beta we’ve started looking into how we can harness its new features for use with Unified Remote. One feature which caught our eye was the new so-called App Extensions. Perhaps this could finally bring our widgets over from Android, allowing you to use quick actions and all that other good stuff? As it turns out, we were not that lucky but I still managed to do some neat stuff. In this post I’ll describe a bit about what I did and some of the big pitfalls and difficulties you might face when designing a Widget (Or Today Extension).

Terminology

First off some terminology. Apple’s documentation and people’s usage of the different words is a bit confusing. Apple uses the term Extension to include anything that’s not quite an app in itself but rather extends the functionality of a different app. They come in many flavors and the one referred to as a Widget, in this article and by most people, is what Apple calls a Today Extension (Although they sometimes call it a Widget, just to confuse people I guess).

What a widget is (And isn’t)

You should see a widget as it’s own self-contained app. It has its own target in XCode, its own code and classes, its own resources, it’s own process on the host device and so on. As such a widget (Or any form of extension) is not really an extension of your app but rather a small, self-contained app in its own.

So what does this mean? And why should you care? Well, for starters it makes sharing code and data extremely cumbersome (More on that in a later post). It means you can’t simply call a method in your app and have it work. While there are some ways of sharing code, using embedded frameworks, none of them will allow direct communication to your app’s process. Apple’s idea is that widgets should launch its hosting app but never communicate silently in the background. This is also the reason why the Unified Remote widgets from Android won’t work for iOS, since there is no way to perform actions without opening the app and that completely breaks the purpose of the widget to begin with.

Solving the problem

Since the widget implementation we originally though of didn’t work out I chose to go for an alternative approach, something where we could still utilize the widgets but which didn’t feel contrived. Enter Quick Switch remotes:

While this feature is still in development, the basic idea is that you get a widget with your selected quick switch remotes. These are the same remotes as the ones you can see in the right menu in the app. From what we can tell, this actually works really great and it’s a neat way to get even quicker access to the remotes you want to use.

If you’re feeling a bit techy, join me for Part 2 of this post about the more technical details.

Since we’re had quite a few requests for the possibility to use Unified Remote with an Apple TV I started looking in to how this could be done. My first instinct was to look into the possibility of compiling the server for Apple TV. However this proved to be too much of a hassle and would require the user to jailbreak his or her Apple TV. All in all not really an option.

This left me stuck without ideas until a colleague pointed out that Apple has their own remote control app, maybe it can control the Apple TV. True enough, it had the option for remote controlling Apple TVs and I started digging in to how that worked.

What I found, using Wireshark for packet sniffing and a whole lot of Google Fu, was that it uses the same protocol as for remote controlling iTunes, a version of DAAP (Digital Audio Access Protocol) called DACP (Digital Audio Control Protocol). Apparently this protocol is far from well documented, so using Wireshark as well as some 5-10 years old Google Code Wiki articles I set out to investigate how the protocol worked.

A project called daap-client for Android had a very helpful article explaining some of the inner workings of the protocol. Basically it’s a very simple binary protocol over HTTP that uses tags to determine the data type of each packet. The packets look like this:

Tag (4 bytes)

Length (4 bytes)

Data

Now this leads to some interesting findings. First of all the fun fact that Length is always present, meaning that for instance a 32-bit integer has Length 4 (Just in case you happen to forget the length of an int) and in order to send a single byte you need 8 bytes overhead in the form of a Tag and Length = 1.

The other interesting thing is that this raises the question: How do we interpret the data? Obviously it has something to do with the tag but if you take a closer look at the tags provided by the article mentioned earlier the tags aren’t direct definitions of the data type contained in the packet. Instead there is a table in which one can look up the data type connected to a certain tag, however the table in the article lacked quite a lot of the types used by a more recent version of iTunes.

To figure out what tags correspond to what type I wrote a simple decoder based on the info I had. By pointing it to a computer running iTunes and fetching the URL /content-codes I managed to print out all content codes in a neat and orderly fashion. Based on this a dictionary of each tag and its corresponding data type could be constructed.

To conclude, we have made some progress as to integrating Apple TV support in to the server but now comes the really challenging part, actually sending data, keeping a session and finding out how the protocol actually works. Stay tuned for more!

As most of you may know by now our app for iPhone, iPad and iPod Touch is now out in the wild (In case you missed it, you can grab it here). The point of this post is to highlight some of the features in this app but before I do that, let me take a few moments to introduce myself.

My name is Richard Simko and I’ve been working with the iPhone app since the summer of 2013, some of you may have been in touch with me through our beta program (Thanks a lot for all the help testing there!). It’s great fun working with this type of app and I hope to hear from even more of you in the future.

New User Interface

Back to the app, the first major thing is that we’ve really focused on making the app feel at home on Apple’s operating system, iOS. As a result we decided to completely overhaul the UI compared to what is used to look like on Android. You’ve seen a few examples of it here previously and it’s also a part of the latest beta builds for Android but our iOS-app is the first to actually launch with this new UI. Among other things we added a slide-in menu which handily let’s you switch between different functions in the app, we improved the server manager to be even easier to use, and we added a completely new guide to get you up and running in no time. We also added some neat new features to the keyboard, allowing you to use more advanced keys when using the keyboard (Such as Ctrl, Alt, Esc, Tab etc.).

New purchasing options

Another great thing we did with this new app is to do away with the full and free versions as separate apps. Instead we offer it as an in-app purchase, not only making it easier for you to upgrade but combined with the other in-app purchases it’s a lot easier for you to tailor the app to your needs. One of these in-app purchases are the remote unlock tokens. Each token allows you to have one remote of your choosing unlocked forever. For example if you’re only interested in one of the premium remotes, you no longer have to unlock the entire full version, instead you can simply buy an unlock token and unlock the specified remote. You are free to switch between remotes as many times as you like and your unlocked remotes stay unlocked forever. Our goal with this is to make it easier for you to tailor the app to your needs. Right now this, along with the Extended Keyboard, are the only purchases you can make apart from the full version but stay tuned for more.

Mac server

We know that many iPhone and iPad owners also have a Mac and of course we want you to be able to fully enjoy all the features of Unified Remote. Because of this we now have the server application available for Mac. While it’s still in beta it’s still quite stable and works great (I use it myself a lot). To check it out head over to our download page and grab it.

Plans for the future

In addition to all these great features there is more to come soon! First of all we’re looking in to some more advanced features for the remote list which will allow you to sort in different ways, search the list, filter it and so on. The second thing is adding a menu for quickly switching between remotes, similar to the quick switch between servers in the right menu. We also have some plans for an exciting event this weekend which we will publish more info about really soon.

Those of you that have been following our development news will know that we have a lot going on right now. We are currently working on three different major releases simultaneously. We’re working extremely hard to get these finished as soon as possible, but we want to ensure high quality products by not rushing to release. Below is a quick summary of what we’ve been working on for the past few months, what the current status is, and what our roadmap for 2014 looks like.

Unified Remote for Android

The next major version of our Android app will be a significant improvement. Our main focus has been improving and optimizing the user experience. We’ve also put a lot of effort into customization. We’ve completely overhauled the app navigation and improved core functionality such as keyboard & mouse remote control, server management, widgets and quick actions editors.

At the moment we are working on finishing the last few features and getting rid of bugs. We have not announced any release date for the next version of the Android app simply because we do not want to make promises that we can’t keep. The same goes for our other releases.

Our roadmap at the moment is to release a beta within the next 1-2 months. We will then continue with the beta until we feel that the quality and satisfaction is up to our standards. If all goes well, the beta hopefully won’t last very long.

Unified Remote for iOS

Over the past few months we have been making very good progress with Unified Remote for iOS. We are currrently in an early beta phase where a limited group of testers are able to try out the app and give us feedback. There are still a few important features that need to be added, but overall the beta is going well.

Our roadmap at the moment is to continue with the beta until all the core features have been implemented and until the quality and satisfaction is up to our standards. Our goal at the moment is to have a first version ready for public release within 1-2 months.

Unified Remote Server

The next major version of Unified Remote Server is by far the biggest change. We have completely re-written the entire server software and re-implemented all of the remotes. The reason for this is that the current version of the server was developed for Windows only. By re-writing the server we were able to choose a different approach and target a much broader range of platforms including (but not limited to): Windows, Mac OS X, Linux. We are even working on providing servers built specifically for embedded devices such as Raspberry Pi and Arduino.

The other major change in the next version is that we are completely opening up the remote implementations. By embedding a powerful scripting engine we are able to provide a framework for developing remotes that can be used by both us and our users that are interested in creating their own remotes, or modifying our remotes to suit their needs.

At the moment the new server is available as a Developer Preview, intended for those who want to try out the new scripting functionality. We are currently working on finishing up the last core features before we are able to release an alpha or beta version.

Our roadmap at the moment is to release an alpha or a beta within the next 1-2 months. The beta is likely to continue for at least a few months due to the broad range of hardware and software that we want to support. Therefore our goal is release the new server as a parallel release until we feel that we can completely replace the v2 server.

Conclusion

As stated, we have a lot going on right now, and we are still a tiny team! With the work we are doing we hope that we can grow our team during 2014 so that can we provide more features and quicker releases. But, in the meantime, please be patient and if all goes as planned we should have betas of all 3 releases available within 1-2 months. The best way to support us is to purchase the full version of our Android app, or join the beta testing groups. Thanks, and have a great 2014 everyone!

Most of the major changes in V3 have been shown. The last important aspect that we’re still working on is the customization experience. For V3 we will be introducing a completely new editor for creating Widgets and Quick Actions. We’ve also improved the configuration/preview screens for these. More information about the new editor will be coming soon.

For V3 we’ve made some improvements to the remotes screen. There are now 4 different lists available: All Remotes, Most Used Remotes, Recent Remotes, and Recent Actions. The idea is to make it easier to find the remote you might be looking for. The Recent Actions list also makes it easy to send commonly used actions quickly.

Our plan is to add more lists in the future. For example, we plan on adding a Favorites list as well, where you can choose exactly what you want to include. We also want to make the screen more customizable in general as well. For V3 you will at least be able to select your “default” list, but we also plan to include an option to reorder the lists as well in the future.

Another core feature that we have made some improvements to is the Server Manager. Our goal was to make it quicker and easier to find and add servers in the app. One of the changes we’ve made is to combine to the list of saved servers and the list of detected servers into one screen. Previously these were in two separate screens.

The first section shows the list of saved servers. This includes servers added from the automatic detection and from adding manually. The second section shows a live list of servers that are automatically detected. They can easily be added to the list of saved servers by pressing them. Hitting the plus button in the action bar takes you to the screen to add a server manually. Also, as mentioned in a previous blog post, you will now be able to easily switch between servers using the left side menu, accessible from all the main app screens.