objects

Menu

Monthly Archives: January 2014

This is a rambling discussion about programming an app to use gestures on a desktop computer. It says that your instances of the Command class are more complicated when gestures can drive the command instances.

Apple HIG suggests that a ‘good’ desktop app should support gestures.

It wasn’t always this way; even today few desktop computers have multi-touch input devices. And multi-touch is the important word. To be clear, this Apple recommendation is about supporting those users who have Apple Trackpads and Magic Mouse ( which support multi-touch.) I don’t know how rare those are these days, whether Apple even ships new Macs with Magic Mouse standard.

To implement this recommendation requires that you look at certain commands (actions) in a different way:

they might need to be cancelable

they might need to be combined

Consider as an example using pinch and swipe gestures to control a view.

In my experience, both these gestures are active at the same time. That is, when a user places two fingers, both gestures start. Then a user can pinch their fingers, or pan their fingers, or both simulateneously, and the system delivers both gestures (as updates). Finally, the user lifts both fingers and the system delivers both gestures ascompleted. (See my post about testing gestures, to see how I determined that the above is more or less accurate.) Or, if the user places a third finger down, the system delivers both gestures as canceled.

Combined

In the usual app, there is a command to pan the view and a separate command to zoom the view. With gestures, those commands should be combined. Here I am using command in the Command pattern sense: some unit of action (which is often undoable, except that view change commands are not usually undoable.)

It is no big deal, after all both commands are simply changes to a viewing transform, and both zoom and pan can be combined in one transforming command. I’m just saying it is more general than separate commands, and many apps are not written to be general, but specific for separate view change commands.

Cancelable

In the usual app, a command to change the view is not cancelable. The action of the command is usually fed back (ghosted) as the user is performing the command. For example, as the user moves a scroll bar, the view (what the user sees) changes in real time (not after the user releases the scroll bar gadget.)

Using gestures, which might be canceled, you must be prepared to cancel the view change. This suggests a transactional command: one that seems to happen in real time but that can be rolled back. (in database parlance?: the command gives a consistent view to all users, and can be rolled back.)

For example, if a user puts two fingers down, moves them, and then puts another finger down, that might cancel the pinch and swipe gestures. If you were already ghosting the view change, your app now needs to roll it back. (And maybe handle the new three finger gesture?)

It is more complicated if the command is also undoable: then you need a transactional, undoable command.

The bottom line is: to support gestures adequately requires various flavors of command classes: combined, transactional, and undoable.

A previous post tells how to install and use pydroid. This post tells how to modify your template app (that pydroid created and deployed to Android.) It gives a series of small examples. The goal is to port a full fledged app you developed on another platform. But I want to test smaller examples first. I am worried that Qt support for Android is half-baked (there are many bug reports.)

General procedure

cd to the project directory e.g. ~/APP_NAME

modify the app’s code e.g. edit ~/APP_NAME/app/main.py

rebuild the project e.g. >pydroid deploy complete

After the last step, you wait while it rebuilds and deploys. On the AVD (which should be running), it reinstalls your app. Then in the AVD you can click your app icon to run it.

The default pydroid template app is a MVC, QML app

When you create a project in pydroid, it fills in a small app, a template app. The app uses a declarative GUI, i.e. QML, which is a language for specifying the form of a GUI without specifying the business logic, in the same way that, for example, you can use Glade to specify the form of a GTK GUI.

The template app also uses the Model View Controller paradigm, and provides simple classes for it.

But you can just throw that all away, or ignore it, especially while we are testing.

‘About Qt’ Example

In main.py, just in front of:

c = controller.Controller()
c.start()

put:

app.aboutQt()

The new app should display the standard (built into Qt) dialog that desplays information about the Qt version being used.

I found that it does display, but is larger than fits on the screen I used, and apparently does not scroll (or I don’t know how to scroll the screen in the AVD.) The “OK” button does function, ending the display of the dialog and resuming the app.

This indicates some corners of Qt are not ready for effortless porting to mobile devices: not allowing for smaller screens.

It did not behave as expected: after I ended the “About Qt” dialog, the app simply quit. To debug, in a terminal, enter:

adb shell “cat /sdcard/APP_NAME_log/python_log.txt”

That is, the template pydroid app includes logging to a file on the AVD’s SDCard, which you can read using ADB (Android Debugger Bridge.)

Reading the log, I found that the assertion failed. In other words, the app has no top level window until after you create the Controller.

So I moved the call to showWarning() to after the call to c.start(), and it worked as expected.

However, the title of the warning box did not appear on the AVD. That’s bad user interface. But I don’t know enough about the Android platform: whether the HIG tells how to display warnings in such a way that the user knows which activity it displaying the warning. It could be a deficiency of the Android platform, or it could be that deficiency of the port of Qt to the platform.

QGraphicsView Example

My app uses a QGraphicsView and a QGraphicsScene (part of the QGraphicsFramework). My app builds those procedurally. The pydroid template app uses QML to declare the view.

I tried to insert code into the pydroid template app to procedurally build a view. I could not get it to work. However, I found that QDeclarativeView ( that the pydroid template app already uses) is a subclass of QGraphicsView, specialized to read its components from QML. The pydroid template app thus demonstrates that it is possible to use a QGraphicsView on Android (I just need to adapt the view initialization from the source code for QDeclarativeView.)

So I simply edited view.py, adding an event() method to handle view events, and calling self.scale(0.9, 0.9), when an event of type QEvent.MouseButtonPress is received. It seems to work.

This is a simple log of my experience with pydroid. Not necessarily successful yet…still a work in progress. Jump to the end to see where I am stuck.

Pydroid is a build tool for python apps on Android, using the Qt GUI framework.

I started with a more or less clean computer that is not a development machine (few programming tools already installed).

As noted in the pydroid documents, it only works with Linux. I am using Ubuntu 13.10.

About Pydroid

Pydroid is a set of scripts, in the Python language. It automates a build process that alternatively is tedious typing into the command line. Python is a scripting language often used to automate such administrative or building work. Alternatively, the build process could be automated using shell scripts. Since you are a Python programmer, it makes sense to use Python for your build scripts, since you probably are weak at writing shell scripts.

Pydroid also seems to create a project for Qt Creator. And the Pydroid command seem to find their way into Qt Creator?

The pydroid README file says it is a framework, but that word means something else to me. Pydroid seems to be another SDK: a kit of tools (Python scripts) and template code, for developing an app. As an SDK, it depends on other tools.

Dependencies

These are pydroid dependencies:

setuptools

pip

adb

Necessitas NDK

Necessitas had its own dependencies, these are Linux (Debian) packages :

build-essential (a basic suite of program development tools)

openjdk-6-jdk (a java development kit)

ant ( a build automation tool similar to make)

The Necessitas NDK includes the Android SDK and the Android NDK. Thus you don’t need to install those. However, the Android SDK that is part of Necessitas SDK as installed is somewhat dated. The Android SDK package includes a tool for updating itself, the Android SDK Manager (a package installation manager.)

Install Dependencies

Install setuptools

The install script for pydroid needs setuptools. Download and install it with this command:

/PATH/TO/NECESSITAS (the path where you installed Necessitas, e.g. ‘/home/<loginName>/Necessitas’

ARM-VERSION e.g. ‘v7a’

Create a pydroid project

Enter:

pydroid create app name:APP_NAME domain:COM.EXAMPLE

This creates a project directory called “APP_NAME”. (You should make sure the current directory is a suitable home for that directory, and you probably will want to substitute for “APP_NAME.”)

The documentation says the project is now ‘already deployable.’ That doesn’t mean it is ready to distribute. It means it is ready to run the “deploy” command. The deploy command builds the app and uploads it to a device (virtual or real.)

Testing pydroid configuration

Invoke the “status” command of the pydroid app:

>pydroid status

It should display a list of configuration values. Instead, you get this message:

This functionality is available from the root directory of a pydroid project only.

Apparently the “check” command does not check an installation, it checks a project, which is contrary to what the README says:

You can always check your system or your project with the following command

Note that ‘check’ checks a project, while ‘status’ checks the pydroid system (the configuration values it will use to build.) Neither command will execute except when the current working directory is a pydroid project directory. Change directory to the project you just created:

>cd APP_NAME

Now the status command should work and show you configuration values.

Build and deploy your app

Invoke the pydroid deploy command:

>pydroid deploy complete

In my session, I had never created an AVD using AVD Manager. So I got:

Invoking the Android tool

The ‘android’ tool is a command line tool with two command line options (for the purposes above.)

avd

sdk

To use the tool, for example:

>cd ~/necessitas/android-sdk/tools
>./android sdk

Invoking the tool brings up a GUI tool.

Updating the Android SDK

Start the ‘Android SDK Manager’:

>./android sdk

The GUI is an expandable tree of package that can be installed, and their current state of installation.

Check the checkboxes of the packages you want to install or update, and then choose the “Install Packages” button.

In my session, it showed that the ‘Android SDK Tools’ and others could be updated. I did not choose to update them.

In my session, it showed that only Android API 8 and 10 were installed. At the time, those API’s represent a fraction of the installed Android base (a majority of Android devices in use in the real world are the ‘Jelly Bean’ API, i.e. version 4.x.x i.e. API 14 through 17. So I chose to install API 16.

The goal is an AVD that pydroid will be able to talk to. I reasoned that the the older API’s were probably broken (and my initial attempts to create an AVD resulted in an AVD that wouldn’t boot.)

Note that after installing a set of packages, the Android SDK Manager usually suggests a few more packages to install. I.E. installing is iterative.

Creating an AVD

Unfortunately, that describes starting the AVDM from inside the Eclipse IDE. You can also start it from a command line: (It is part of the Android SDK, which is bundled inside Necessitas, which you installed.)

>./android avd

AVDM is a GUI app. Use it to create an AVD. (For example, I used “Nexus S”, “API Level 10”, “ARM(armeabi)”, “500” MiB for the SDcard, and “Use Host GPU.”)

(I think it is important to have a large SDcard configured. I think your app is installed there, and the components can be quite large, and the error if it runs out of space might be obscured. Don’t chance it.)

After creating an AVD, choose the “Start” button to start the AVD. A window will appear showing the emulated window of your Android device. It may take many minutes for it to boot. It typically shows a blinking “Android” while it boots.

While or after that is happening, on your AVD’s screen, it shows “Installing”. After the install completes on the AVD you will see

This application requires Ministro service. Would you like to install it?

Choose the “OK” button.

Then the AVD should access the network and download a Qt library (that is what Ministro is, a package manager for an Android device that understands how to download in install dependencies. Note that this strategy is unusual for Android devices, and strange to users?)

Debugging your app fails to find the Ministro service

On my first attempts, on the AVD I got:

Can’t find Ministro service. The application can’t start.

On the AVD, I tried the browser app and found that the AVD could connect to the internet.

Ministro cannot be downloaded to an AVD because there is no ‘market’ for it?

Qt works better on a real device (eliminating the emulation layer)

Qt5.1.1 might behave better (and I’m not sure what version of Qt pydroid is using?)

you can install Ministro.apk and MinistroManager.apk in the AVD manually using adb (instead of going through Google Play i.e. a market)

you can install Qt in the AVD manually?

Figuring this out (whether Ministro SHOULD download itself, and why it doesn’t on an AVD) is not a high priority. Assume it WILL work on a real device, and for the virtual device, download it manually:

I don’t know how the Ministro service is visible to an Android user. Apparently not as an app.

Testing your app after installing Ministro service

Now in your AVD, click on the icon for your app “APP_NAME.” Now it says:

APP_NAME needs extra libraries to run. Do you want to download them?

(This is the Ministro service talking to you. Ministro is a broker. Your app asks Ministro for libraries. Ministro negotiates with the user and acquires the libraries, then returns the libraries to your app.)

Click on the “OK” button.

You see a sequence of various progress indicators: “Extracting …” and “Downloading …”. It takes several minutes to download many Qt libraries.

Apparently this step works on an AVD (whereas downloading the Ministro.apk does not on an AVD?) I suppose the Qt libraries are in a “repository” and not in a “market.”

After the Qt libraries are downloaded you will see:

Hello world example.

Edit the following files….

This is your pydroid app talking to you. Thats all it does: print out a message. Hit the Escape key to exit it.

Start your app again, by clicking on its icon. You will briefly see a “Necessitas” logo, then the message above. That is, on second runs of your app, Qt libraries are already installed on your Android device and the app starts quickly.

From here

I use the PyQt binding instead of the PySide binding. I hope to blog about swapping bindings in Pydroid.

Debugging: your app fails to ask for Ministro service

On my first attempts, on the AVD I got: “Unfortunately, APP_NAME has stopped.”

I just explored, and found I had:

made a typo in the build.config

had specified ‘v7a’ in the build.config but configured an AVD with the x86eabi

I tried to uninstall the app on the AVD

Using an AVD

Tips:

use the mouse (instead of a finger)

use the Escape key as ‘back’ i.e. the one physical button on many mobile devices.

use “Page Up” to unlock the device (when the lock icon is visible.)

To uninstall an app:

from the main menu (a grid of icons), choose “Settings”: a menu will

choose the Device>Apps menu item: a list of installed apps will appear

Context: you are a Python developer. You want to port your app to the Android platform. You are confused about the many projects that provide tools for this purpose.

Disclaimer: this is a work in progress. It is a log of what I learned trying to make sense of the subject.

Python on Android

Python is a language. Android is a platform. Most development for the Android platform is in the Java language. It is rare to develop for the Android platform in the Python language. You probably develop your Python app using tools on a development machine where Python is well supported. To port your Python app to Android, you need other tools.

Tool Projects for Python on Android

Developers create tools for porting Python to Android and publish projects for the tools:

There is some duplication/sharing between projects. The projects have similar names but are distinct projects!

The projects differ in:

what tools they provide

what development environment they support

whether they are active and supported

PY4A

This project is about compiling (for Android):

the Python interpreter

Python modules

The result is not very useful on its own, since the result doesn’t include much IO to a user on the Android platform (because the notion of stdin/stdout/stderr to a console is not in Android?)

The result is very useful as a component in the other tool projects.

Most developers don’t need to understand this project, unless they have custom (non-standard, C language (not pure Python)) Python modules that need to be compiled for Android.

The essential contribution of this project is:

the scripts (business process) for compiling Python interpreter and modules using the Android NDK

patches to the Python interpreter, needed because of differences in libc on Android?

This project is probably stable and relatively inactive. Once the project ports a particular version of Python to Android, the project is done until another version of Python is released (or the Android platform changes?)

SL4A

TODO

Android-python27

This project is about packaging (for Android):

an app written in Python

that uses minimal services of the Android platform ???

????

The project includes an Eclipse project that can be imported into Eclipse IDE. After telling Eclipse to build, the result is an Android package.

The project also includes source directories from other the other tool projects. It is not clear how to combine them? The project does not include an Eclipse project for packaging a Python app that uses Qt? It requires a certain amount of hacking?

Python for Android

This project is about packaging (for Android):

an app written in Python

that uses the Kivy cross-platform GUI toolkit

that uses services of the Android platform (that are not available through the GUI toolkit)

The development environment for this project is a Linux machine, using a command line.

Contributions:

scripts (business processes) for compiling components (distribute.sh)

scripts for packaging (build.py)

a binding from Python to/from Java (pyjnius)

Kivy cross-platform GUI toolkit

This project seems to suffer from a language/culture barrier. The project documentation in English seems like poor technical writing.

TODO who should use this project

PyDroid

This project is about packaging (for Android):

an app written in Python

that uses the Qt cross-platform GUI toolkit and the PySide binding to Qt

The development environment for this project is a Linux machine, using a command line.

The project IS a python package. You install it like you would any other Python package.

To use the project, you execute ‘helper’ or tool programs (scripts written in Python). For example:

>pydroid check

The project depends on TODO. During the installation of this project, it checks the dependencies?

Comparing the projects

These projects have the same end result, an Android package (.apk) that includes a Python interpreter:

android-python27

python-android

PyDroid

The projects use different environments (integrated development environments or IDE) on different development platforms:

android-python27: Eclipse IDE on Linux, OSX, or Win

python-android: command line on Linux

PyDroid: command line on Linux with support for QtCreator IDE

The projects have end results using different GUI toolkits on the target Android:

android-python27: minimal native Android GUI (toast?)

python-android: Kivy GUI

PyDroid: Qt GUI using the PySide binding

Note that the result GUI’s may look more or less native Android (conformant to platform human interface guidelines HIG, or having the native look-and-feel) . The projects use different toolkits to achieve that.

TODO PyQt binding

The Java bootstrap to exec the Python interpreter

Each project that produces a package includes a Java bootstrap. The Java bootstrap is a small program in the Java language. The Java bootstrap is the first thing to be executed on Android (inside the Dalvik VM, which is the Android OS and executes Java programs.) The Java bootstrap proceeds to exec (fork?) the Python interpreter, which interprets your app written in Python, which may call other Python modules (possibly written in other languages.) Some of the Python modules may be bridges, ‘system calls’ to the Android OS.

TODO the Ministro bootstrap

IDE Perspectives

Since each project includes a Java bootstrap AND app code in the Python language AND the Python interpreter in the C language, the projects are multi-language projects. An IDE (specifically Eclipse) may have different ‘perspectives’ for a project. One perspective is usually the best choice: the android-python27 project uses a Java perspective.

Context: you are trying to develop software for the Android platform, and want to test your software on an emulator.

About: this post is high-level. This post mostly ties concepts together by linking to Wikipedia.

Disclaimer: this post is a work in progress and may contain errors. I aim to organize concepts, such as in an outline, but the details are sketchy.

Developing for Android is cross-platform

Android is an OS running on an embedded computer. Android devices are so weak (little memory, disk, or speed) that usually you develop on a different, host device.

(But there is now a IDE that is native, i.e. runs on an Android device: AIDE. Probably it is most useful for small apps, or simple, command line tasks.)

Compiling a program is then cross-compiling. An Android device is the target, the development computer is the host.

The universe of Android devices

There exists a large variety of Android devices. The Android platform changes quickly, and there are many slightly different versions of the OS and hardware. Android is open source, so any vendor is free to sell a device running any version of Android, or even Android OS that they have modified.

Also, a vendor may sell a product in different configurations, e.g. with more or less memory, larger screen, etc.

Testing an Android program

Comprehensive testing of an Android app requires testing on many versions and products.

When developing cross-platform for Android (or in general) there are two options:

test on a real device

test on a device emulator

The main advantage of an emulator is: it’s free. You don’t have to buy the many, physical products to test on.

Uploading an Android app

To test on a real device, you must upload to the real device, across a connection (a cable or wireless.)

Uploading to on an emulator is fast, at the speed of memory transfers. But execution on the emulator might be slow (depends on the technology of the host machine and the emulator.)

Android Emulators

Android emulators are called Android Virtual Devices (AVD.) They are virtual machines. They do virtualization. They virtualize both the hardware and the software of the target device.

The virtualized hardware may include:

processor which may be a different architecture than the host, say ARM

keyboards, screens, and buttons

sensors (cameras, accelerometers, etc.)

AVD’s are similar to guest operating systems used with virtualization software such as VirtualBox. The software that runs them is called virtualization software or a virtual machine monitor. The Android Virtual Device Manager ( AVDM?) helps you manage AVD’s . In the Eclipse IDE, the AVDM is in the menu Window>View Android Device Manager .

AVD’s are persistent

You create an AVD and it is still there the next development session. You can install your software to an AVD and it will still be installed the next time you run the AVD.

The data for an AVD is kept in hidden files ~/.avd/… If the AVDM fails to manage an AVD (such as delete it) you can resort to fiddling with hidden files.

AVD’s are cheap. It is usual to delete them and start over to make sure you have a clean starting state.

Genymotion and Hardware-Assisted Virtualization

Genymotion has compiled the Android OS for the x86 instruction set. Genymotion uses VirtualBox. The Android OS runs as a guest OS. When the host OS is running on x86 hardware (as many PC’s do), and the CPU includes support for hardware-assisted virtualization (as most recent CPU’s do), then instead of emulating the instruction set of the Android device, instructions are executed directly in the CPU. For example, one virtual instruction may be executed by one real (CPU) instruction, instead of one virtual instruction executed by say ten real(CPU) instructions.

More specifically, Genymotion has compiled the Dalvik VM for the x86? The Dalvik VM is the interpreter for the Android OS written in Java.

This is an announcement of a project I shared on github: testQtGesture

Its an easy to change program for testing gestures on Qt, which is cross platform. It is a reduced app harnessed to log many events related to gestures.

Gestural GUI’s are horribly complex to implement. Some of the many factors:

platform OS

devices and device drivers

your app’s widget tree

your app’s interpretation for gestures

Its much simpler if you stick to one platform, maintained and documented by one company e.g. iOS. But Qt is cross-platform, including desktops, iOS, and Android. Qt gestures are not well documented. I use testQtGesture to explore gestures across platforms.