Use pyqtdeploy to generate Qt Creator project

Start pyqtdeploy

where pensool is the name of my/your app. The pyqtdeploy GUI should open. Here we made a new directory to build in. Pyqtdeploy can create many files in that directory.

Configuring your pyqtdeploy project

Glossing over the details, this is similar to pyqtdeploy that is NOT cross-compiling. (For example, nowhere do you see the word “Android”) The main difference is that under the “Locations” tab, the “Target Python Locations” section points to a directory which contains cross-compiled artifacts (e.g. a SYSROOT of /home/bootch/aRoot.)

pyqtdeploy ‘Build’

After you have configured pyqtdeploy, choose the ‘Build’ tab. Uncheck most of the options (i.e. ‘Run Qmake’ and ‘Run make’) since we will be doing that soon (I don’t know, maybe those would work, but I prefer to see any errors from inside QtCreator.) Choose the ‘Build’ button. This creates a qt creator project file (.pro)

Build using Qt Creator

Start Qt Creator

Navigate to the project created by pyqtdeploy (say ~/pensoolAndroidBuild/build/Pensool.pro) and choose the project.

The first time you open a project, Qt Creator tells you it has not been configured, and offers a choice of configurations. Configure the project for Android.

Choose the ‘Build’ button. (Now Qt Creator shows you a list of Android AVD’s. You should have one if you have tested Qt Creator’s ability to cross-compile a C++ helloworld example.)

Choose an AVD. (Now Qt Creator proceeds to compile and you may get errors. See below, the FAQ. )

FAQ Frequent errors

This is because Qt Creator is building (linking) for a different architecture than the libraries. It can be 32-bit versus 64-bit, or Intel versus ARM architecture? Typical error message from Qt Creator:

This might be wrong configuration of the Python build. The Python build is configurable for optional modules (extension modules, written in C.) This is briefly mentioned at the pyqtdeploy documents:

This will configure Python for a small sub-set of standard extension modules. Your application will probably require additional ones to be enabled. To do this you will need to make changes to the python.pro file and the config.c file (in the Modules directory). See the comments in those files for more information.

A minor quibble is that there don’t seem to be any pertinent comments in the python.pro file. I think you just need to add additional lines in the body as in this example:

Here, the names are names of source code files. You can also view these names in Modules/Setup.dist.

To edit Modules/config.c, you add a line for SOME of the modules which you added a line to python.pro: those modules whose error is like ‘PyInit…’

This process of figuring out what additional Python modules is similar to the process when you are not cross-compiling. That is, when you built your app for the desktop, you determined what additional Python modules your app uses. (The process there was iterative: build, see what Python complains about, add the module, and repeat. Here, the link phase tells you all the missing modules at once, but it doesn’t tell you the names of the modules. If you didn’t build your app for the desktop, you can determine the names of the missing modules by grepping the Python source code in the Pythonx.x/Modules directory for the function name given by the link error.

In this example, _posixsubprocess is a key (and also the name of the source code file) but note that this results in two underscores in the value field. In this example, ‘select’ is a key, but NOT the name of the source code file (which is ‘selectmodule.c’. In other words, this key must be the same as the key defined in Modules/Setup.dist.

After you make these configurations, you should not run pyqtdeploy again (because it creates a new python.pro, overwriting your edits) but you should run qmake, make, make install again because qmake reads the python.pro file you just edited.

It seems that you can rebuild and reinstall a static Python without destroying the installed PyQt and SIP components (so you can proceed directly to Qt Creator and retry your build.)

More patches might be needed for Python modules

Again, pyqtdeploy, when helping you build Python, applies patches to the Python source code to account for differences on the Android platform (that Python.org does not support.) If you configure Python with additional extension modules, you might need to make additional patches (until pyqtdeploy knows all the patches.) Here are some ill-defined hacks:

Next:

The pyqtdeploy directory contains a file DeveloperNotes.txt. This seems to be a discussion for a developer of pyqtdeploy (someone who is maintaining pyqtdeploy or wants to add a feature to it) but it also has useful hints about Android development.

These are brief notes about localization of PyQt (or C++) programs using Qt5. Use with caution, I’m learning too, and this could be wrong.

Localization is also known as:

i18n

internationalization

l10n (common on Qt sites)

The state of Qt translations

Briefly, they are haphazard. Because of the transition from Qt4 to Qt5, many translations files are waiting to be ‘blessed’ by a third party translator. You might need to use a mix of files from Qt4 and Qt5.

This page discusses how to contribute translations to Qt. If you go there, this link takes you to the current translation files and shows you their percentage completion (whether they have been blessed by a person who knows the native language.) That link also lets you download the current (or future) translation files.

Note that the ‘future’ translation files (for Qt 5.4 as of the time of this writing) are full of translations, but all are marked ‘unfinished’ (until someone blesses them, sometime during the release of Qt5.4?)

Differences between Qt4 and Qt5 translation files

In Qt4, all translations were in one monolithic file but in Qt5 they are split up into a file per Qt module. For example, in Qt4 translations were all in qt_xx.ts but in Qt5 translations for QtCore and QtGui are now in qtbase_xx.ts. This applies both to the templates (.ts) and compiled translations (.qm).

In Qt5, there still exists a translation file having the same name as the Qt4 monolithic file (qt_xx.ts or qt_xx.qm) but it ‘includes’ (in other words references) the split files (e.g. qtbase_xx.ts and qtdeclarative_xx.ts). In Qt5, that file is called a ‘meta catalog file.’ (Such a .ts file will not open in Linguist; Linguist says it is empty.)

So in your app code, you can still load a translator for the one name. As discussed here, for C++ it recommends to concatenate .qm files into one file having that same name.

However, for a PyQt app that only uses one module, it might make sense to load a translator with only one file, e.g. qtbase_xx.ts. But because of the transition from Qt4 to Qt5, a ‘blessed’ qtbase_xx.ts file may not exist. You might just rename the Qt4 monolithic file qt_xx.ts to qtbase_xx.ts.

It is part of your build process to compile the template files (.ts => .qm) and to concatenate them into one file (or not.) Your buld process may include a step of downloading the template files (.ts) for Qt library.

In the files at that link for Qt5.3, there are some files such as qt_pt.ts that are the ‘meta catalog file’ but the included split files are not present! So you must be careful where you get your files: sometimes qt_xx.ts is a Qt4 file of translations, and sometimes it is a Qt5 ‘meta catlog’ that Linguist says is empty.

Understanding the gitorious Qt website

Warning: I don’t really understand it yet. If you open the qt/qt5 repository, it doesn’t have much in it. If you read the Qt Wiki about building Qt5 from source, it says you first clone that repository, and then run a script that clones other repositories. That’s probably the best way to get Qt5 source. (But for example, the web site says the repository will have all branches, but I found that it only has the 5.3 branch (the stable branch), and is missing the ‘dev’ branch described by the Qt Wiki about Qt’s branching model ?)

At the gitorious qt website, if you open the qt/qtranslations repository, it appears to contain Qt4 translations files. I could be wrong. As evidence, the files there seem to be missing the ‘QPlatformTheme’ context, which is a Qt5 innovation.

Reiterating, you must be careful where you get your translations files (.ts).

What worked for me

I downloaded qtbase_xx.ts files from this link (an ftp site) under the ‘5.4’ section (which are ‘future’.) The files seems to be complete in that they have all the user-facing texts from the Qt5 source code. However, they are all (?) marked unfinished. As mentioned earlier, they are awaiting the 5.4 release and ‘blessing.’

But I ran an awk script to mark “finished” all translations that have non-empty translated text. Its not politically correct, since no human has reviewed the status of the translations. Some of the translated texts may themselves have been generated by a program (by the conversion from Qt4 to Qt5) and might not be correct. But I found that at least for my app, which uses a minimal but critical subset of the strings, it is better than the alternatives. The critical set that I need are the strings in the context ‘QPlatformTheme’, which makes standard buttons in dialogs use the platform’s theme. (This also very much depends on what language you choose, for example German may already be blessed and complete, but Portuguese not.)

Cut and paste can garble a template file!

I found the hard way that you must be careful: you can’t browse to a ‘raw blob’ template file at the gitorious qt website and cut and paste it to your computer. Somehow it garbles the Unicode encoding. You are better off downloading the files (say using the link above.)

The relationship between KDE and Qt

KDE (an alternative desktop to Gnome) is alive and well in Europe. It uses Qt instead of GTK as its windowing framework.

Unfortunately, KDE does their own translations of Qt user-facing strings. They use their own i18n system, including another format (.po) file. I don’t think they base their translations on Qt translations, and have little interest in updating the Qt translations. There doesn’t appear to be much flow of translations from KDE back into Qt. (Its another case of duplication of effort, and difficulty of coordination?)

Japanese still is not well supported on Ubuntu

I especially had trouble with Japanese language on Ubuntu. For example, in Linguist, many characters in translated texts don’t display properly. I think it is mostly to do with fonts. It might depend on many factors, such as whether you have exported LANGUAGE=ja in your shell environment, whether you have used System Settings>Language to set the language, whether you have rebooted, and what fonts you have installed. I struggled to find a combination that was stable.

Bottom line: use something other than Linux/Ubuntu (a Mac?) to use Linguist to translate Japanese (until the mess is straightened out.)

About pyqtdeploy

Pyqtdeploy is a tool for deploying (compiling) apps written in Python and using PyQt. Pyqtdeploy is actively being developed. In previous versions, you could deploy to the mobile platforms, but you were on your own for many steps such as compiling Python statically. In version 0.5, pyqtdeploy itself encapsulates some of those steps (you invoke pyqtdeploy to perform the steps, instead of a lower level tool such as make) :

building Python statically

building SIP statically

building PyQt statically

pyqtdeploy v0.6

This might be the last version before 1.0, in other words, the developer anticipates few new features .

Note that this blog was written for v0.5. In v0.6, the app was split into a GUI and a command line. Everywhere below where you invoke pyqtdeploy from a command line, you should instead invoke pyqtdeploycli.

Configuring the shell environment

At least on Ubuntu, as I installed PyQt5 (using ‘python3 configure.py…), PyQt5 (which is imported by pyqtdeploy) installs to /usr/lib/Python3.4/site-packages, and that must be on PYTHONPATH.

>export PYTHONPATH=/usr/lib/python3.4/site-packages

SYSROOT

Certain of the following commands take the parameter SYSROOT. This names a directory. While you are cross-compiling, it stands in for the root of an Android device. Certain commands ‘install’ files there (as if installing to a real Android device.) Some of the files ‘installed’ there are intermediates, e.g. Python.h header files, and won’t actually be copied to any Android device as part of any install. Some subsequent commands will build a directory tree under SYSROOT similar to a standard Unix/Linux file system, i.e. having subdirectories /lib and /include.

I named this ‘aRoot’, created it in home, and exported a shell variable for it:

>cd ~
>mkdir aRoot
>export SYSROOT=/home/bootch/aRoot

Installing pyqtdeploy

That fails with “ImportError: No module named ‘pkg_resources’ “. I was unable to fix it trying several things much discussed on the web. Note that pip3 is recently changed for Python3. My opinion is that this is basically a fault of Python: it is changing too much and is lacking a stable way of downloading modules, especially for multiple versions of Python. No doubt my problem started when I downloaded a newer version of Python than distributed with Ubuntu.)

Anyway, you can alternatively install pyqtdeploy from source:

>sudo apt-get mercurial # so you have hg
>hg clone http://www.riverbankcomputing.com/hg/pyqtdeploy
>cd pyqtdeploy # to the directory you just cloned
>make VERSION # ? not sure why next two steps are not dependencies in the Makefile
>make pyqtdeploy/version.py
>sudo python3 setup.py install

You can then test by

>which pyqtdeploy
>pyqtdeploy

A GUI app should open.

Updating pyqtdeploy

If you already have pyqtdeploy, but there is a later version. The new version might not read your old .pdy, so write down all it’s data. To update:

Building a host version of SIP

You must also have a host SIP installation that is the same version as the target version you will be building.

That means that after you downloaded the latest version of SIP (which supports target Android):

>cd ~/Downloads/sip*
>python3 configure.py
>make
>sudo make install

So now /usr/bin/sip (an executable) is the same version as the sip.a (a library) that we will later build for Android.

Building Qt statically

Although this step is described in pyqtdeploy instructions, you MIGHT not need to do this, i.e. it is optional.

When cross-compiling for mobile devices it is recommended that you use one of the binary installers provided by Digia.

This means that we will later be using QtCreator to package our app, and QtCreator will take care of bundling Qt libraries (dynamic?) into the Android package (APK) (or create an APK that uses the Ministro package manager at user’s runtime to download Qt libraries to the Android device.)

(If you have configured QtCreator for building for Android, you will have the Qt libraries cross compiled for Android in ~/Qt/5.3/android_armv7/lib )

(But keep reading, later I found that PyQt would not build, saying there is an issue with licenses. Possibly I need to build Qt from source.???)

Building Python statically

Downloading Python source

Navigate your browser to a Python download page and download the source (say the ‘gzipped source tarball’ for Python version 3.4.0).

Extract it, say to ~/Python-3.4.0 .

Using pyqtdeploy to build Python statically

See below. You can’t do this twice without starting with a fresh Python source distribution. Pyqtdeploy patches python and copies the original Python source files to a same named file with .original appended. So the patching will fail on a second attempt.

(target is one of: ‘android-32’, ‘ios-64’, ‘linux-32’, ‘linux-64’, ‘osx-64’, ‘win-32’, ‘win-64’, as I determined by reading targets.py in pyqtdeploy source. Subject to change.)
When that works, you see about 30 lines of output ending with the line: ‘Installing /home/bootch/Python-3.4.0/python.pro’. Also, pyqtdeploy creates a python.pro file in the current directory (which is the Python source directory.)

(If you see:

Patching /home/bootch/Python-3.4.0/Include/unicodeobject.h
pyqtdeploy: /home/bootch/Python-3.4.0/Include/unicodeobject.h:2266: line does not match diff context

I think this is a fatal error meaning that pyqtdeploy, while patching the Python source, used a patch that didn’t match the source. For me, I think this happened on a second attempt, when the source was already patched, and thus would not match the patch. I don’t see an option to pyqtdeploy to skip patching, so you might need to start over with a fresh Python source.

Explanation: pyqtdeploy maintains a set of patches for each Python version. This is fraught with maintenance difficulty. The Python org refuses to support Android, making patching necessary. )

Now we start cross compiling, so define (to qmake) where the Android NDK is:

( SYSROOT is where pyqtdeploy installs packages , and I choose to install to ~/aRoot)

(I am not sure if it is necessary to use the qmake from Qt that you downloaded, and the one under ‘android_armv7’, instead of the qmake that may be installed with your distribution, which for me on Ubuntu 14.04 was Qt5.2.1. I’m not sure whether it is necessary to export ANDROID_NDK_ROOT. )

That works very quickly, without any output to the console, and creates a Makefile in the current directory.

>make
>make install

That takes a few minutes (Python is not a large program.) You will see a stream of commands to the compiler. It finally installs the built (static, cross-compiled) Python library into wherever you specified for SYSROOT (which should be separate from your system’s installed Python.)

Using pyqtdeploy to build PyQt5 statically

Generate a configuration file

That tells the pyqtdeploy tool to create a configuration file: pyqt5-android.cfg. That returns quickly, without any console output.

Editing the configuration file

In my case, for subsequent steps, I got errors such as:

sip: QSslConfiguration is undefined

(When compiling the PyQt binding for QWebSockets.)

I don’t think I need that anyway. So I edit the pyqt5-android.cfg file, removing all Qt modules that I don’t use. (I know from previous experience with pyqtdeploy that my app uses a small set of Qt modules.) I edit it down to QtCore, QtGui, QtWidgets, QtPrintSupport, QtSvg, and QtNetwork.

This is a log of setting up a development environment. I am using Ubuntu 14.04.

My ultimate goal is to use pyqtdeploy to distribute a (Python, PyQt, Qt) app to Android (and iOS.)

Installing QtCreator

QtCreator is a cross platform IDE. I have been using Eclipse and PyDev, but QtCreator might be better, since it comes from the Qt project.

( This bug report explains that the version 3.0.1 of QtCreator shipped with Ubuntu 14.04 will give the error message:

Project ERROR: You need to set the ANDROID_NDK_ROOT environment variable

so I downloaded a later version of QtCreator 3.1.0. You can’t just use >sudo apt-get install qtcreator.

In my experience, Ubuntu lags behind on Qt, and Qt changes rapidly, so it is worthwhile to download the latest Qt.)

At this website I chose the ‘Qt Online Installer’, and the file ‘qt-opensource-linux-x86-1.6.0-4-online’ was downloaded. Then give it execute permission and click on the downloaded file. The ‘Qt Setup Wizard’ app opens. Accept the defaults, and newer versions of Qt and Qt Creator will be downloaded to ~/Qt directory. (This will include the Qt library built for ARM i.e. for Android.)

(As explained by the Qt project, Qt is supports iOS, but QtCreator doesn’t support iOS directly, you will need to use a .pro file AND the Apple Xcode IDE for that. This blog may apply to using OSX as the host and cross compiling to Android.)

Installing Ant

(I’m not sure what kind of tool ant is.)

>sudo apt-get install ant

Install a JDK

You can use the Ubuntu Software Center to find the name of the package and download it. Search for ‘JDK’ and also choose ‘Show xxx technical items’ near the bottom. I chose openjdk-7-jdk, but clicking the ‘Install’ button didn’t seem to work, so…

>sudo apt-get install openjdk-7-jdk

Configure QtCreator to use the tool chain: SDK, NDK, ant, and JDK

Start QtCreator and choose Tools>Options>Android. Click the ‘Browse’ buttons and navigate to the location where you downloaded the tools. (For many of them, Qt Creator leads you to the right place, but for the JDK I had to navigate to /usr/lib/jvm/java-7-openjdk-i386.

Configure QtCreator to use the Android kit

Create an example Project

I followed this tutorial. I don’t really want to use QtQuick, but this is just an example.

(I also took shortcuts with the example. Instead of displaying an SVG file and moving it, I just displayed the white rect that appears in the default QML.)

After creating the project, I clicked on the ‘Run’ icon. Now it says that I do not have an Android device. It offered to create one for me (a ‘Create Android Virtual Device’ button.) I clicked on that, entered a name for the device, and clicked OK. Then I got an error: ‘Error: invalid –tag default …’ (I suppose it is a bug in QtCreator that might be finessed by different choices.) But I decided to use Android tools to create an AVD….

Create an AVD using the Android AVD Manager tool

An AVD is a simulator, or Android virtual device. The Android project provides a tool ‘Android Virtual Device Manager’ for creating them.

>cd ~/android*/tools
>./android avd

A GUI app will open. Create an AVD. (Glossing the details, but the defaults usually work, and for a clean development machine, you will only see the choices from the things you installed earlier.)

I suppose the list of AVD’s is in a standard place that QtCreator knows about.

Testing

Now in QtCreator when I click the Run button, it:

lets me chose the AVD I just created

proceeds to build,

starts the AVD

downloads my app to the AVD

It takes a while for the AVD to boot itself. For a while you may see a black screen or a green arc going in circles. I don’t think the app can load while it is booting. (When it finishes booting, you can interact with the AVD like a real phone, using the mouse. It seems to start in Android’s ‘Getting Started’ help screens (several screens telling you that a swipe does) just as if you had just powered up a newly purchased and activated phone.)

I suppose QtCreator timed out waiting for the AVD to boot. I got:

Unable to start 'org.qtproject.example.accelbubble'.

But the AVD kept running. I tried the Run button again. This time, the application seemed to start on the AVD: it displayed a white rectangle. I tried to click with the mouse in the app to close it, that didn’t work. A swipe with the mouse (pressing the LMB while moving) killed the app in the AVD.