Bazaar2 Monthly Report - April 2017

April was a big month for us in terms of finishing up some big partsthat are directly visible to users, and easy to demonstrate. Thebiggest is the final 0.103 release of the F-Droid app which includesthe complete overhaul of the user experience, which feels simple,friendly and modern. This is one short step from a big 1.0 release,once we nail down the last features and get some more testingcompleted.

We also launched the first alpha of the new F-Droid Repomaker, asimple web tool for creating and managing collections of apps andmedia, and delivering them to users via F-Droid repositories (aka“repos”). Try the alpha demo! http://repomaker.grobox.de/

On top of those two launches, there are many other smallaccomplishments from this biggest and final development sprint forBazaar2.

All texts within F-Droid and graphics associated with apps are nowtranslatable, including all the strings within the app itself, all appnames, summaries, descriptions, video links, recent changes, andscreenshots. With release of F-Droid client 0.103, it will use anyavailable language. For the F-Droid client app itself, many languagesare completely translated, and many more have reached the functionallevel, thanks to the ongoing support from F-Droid community volunteersand the Localization Lab:

We have not received any Tibetan translations yet. We will be hiringtranslators to finish the Simplified Chinese and Tibetan translations.

For the per-app materials, we are now adding all the translatedmaterials for all the Guardian Project apps to the Guardian ProjectF-Droid Repository, which users can enable with the flip of a switchin F-Droid. We are also helping app developers to get theirdescriptive materials integrated for automatic inclusion inf-droid.org.

For reproducible builds, we started out by doing mass rebuilds of allapps in f-droid.org, as shown by https://verification.f-droid.org.This let us fix the most common issues without getting stuck on a fewhard issues. Now that we have reproducibly built over 300 differentapps, we’re turning to focus on reproducibly building the mostsecurity-sensitive apps. These tend to be the most difficult sincethey frequently include “native” C code, which is much harder thanJava to build reproducibly.

While the core tools for adding media files to F-Droid repositorieswere created months ago, we turned to focus on one specific use casein order to polish up the media file support: the F-Droid PrivilegedExtension “Over-The-Air (OTA) update”. This is a ZIP file that users“flash” to their device to install it with elevated privileges. Thisfile is now built, signed, and released using the full F-Droid stack,providing a trusted download method for users of any Android ROM toflash to their device:https://f-droid.org/repository/browse/?fdid=org.fdroid.fdroid.privileged.ota

That means the whole server-side deliver process is ready to handleany file you can copy into a folder. The 1.0 release of the F-Droidclient app will fully handle installing common file types so thatmedia players, etc. will automatically find and play them. As part ofthe Curation Tools section, RepoMaker already has some basic supportfor handling media, which we are now working on completing andpolishing.

In collaboration with Guardian Project’s Developer Square effort, weheld a workshop on the internet called GLOW2017:https://devsq.net/glow2017 . The videos are archived and availablefor anyone to learn from.

When the Bazaar2 project was defined, there were not well known toolsfor managing all of the localized files in Google Play. Now there aretwo: Fastlane Supply and Triple-T Gradle Play Publisher. Both arefree open source software, so instead of reinventing the wheel, weinstead integrated with those existing tools. fdroidserver nowautomatically detects the app store support materials in the app’ssource repo if it is already setup for Fastlane or Triple-T. So thereis now one place to put all of the app store materials (descriptions,graphics, etc) to publish them to F-Droid and Google Play. Thosedescriptions can be easily added to Weblate, Transifex, etc so thatthe translations can be automatically synced when they are complete.

RepoMaker has reached a functional level with the core featuresimplemented. It is currently being developed around the two basic setupmodes: as a hosted web app. Apps can be manually added or automaticallyfetched from other F-Droid app repos. RepoMaker can publish the reposin all the same ways that fdroidserver can, e.g rsync GitHub, Amazon S3,etc. There is a alpha demo of the multi-user mode for anyone to try:http://repomaker.grobox.de

We also began to build the foundations of the localization support.This current implementation strategy will also allow for standaloneinstallations like a desktop app following the web app model like Riot,Signal, etc.

The new user experience is functionally complete and a full release,v0.103, is now available via the normal release channels. We alsonailed down the full integrated experience using F-Droid PrivilegedExtension, which allows for installs without enabling Unknown Sourcesand automatic updates in background. It is now well tested andworking solidly on all Android versions. For the past month, we foundand fixed a number of issues specific to Android 7.x.

We ran two parallel user tests in Lubbock, Texas and Vienna, Austriaof the new user experience for the F-Droid client app. Overall, weare happy to say that they confirmed the general approach of the newdesign, and users overwhelmingly found it simple to use. There weretwo areas where users had difficulty: nearby app swapping and addingnew app repositories. This was not a surprise since, first andforemost, those are totally new concepts for most mobile users, whoare used to getting everything from one source: Google Play.

The new website is ready for launch, once we complete the secure,automated deployment procedure. The new website is generated usingJekyll and consists entirely of flat files with no code running on theserver side. On client-side, Javascript is only required for thesearch function. This makes the website work well with Tor Browser,and makes it easy for anyone to deploy their own app store usingsimple cloud file hosting services like Alibaba Cloud, GitHub Pages,Gitlab Pages, Amazon S3, etc. as well as simple appliance devices likeLibraryBox, FreedomBox, etc. We also began the process of making thewebsite fully translatable. The staging server is publicly availablehere: https://fdroid.gitlab.io/fdroid-website/

The fdroidserver tools for automated “collateral freedom” distributionare in place. The current options for automatic publishing to mirrorsare: GitHub, Gitlab, Amazon S3, and SSH/rsync for webservers and TorHidden Services. The F-Droid client app is already receiving themetadata about those mirrors, but it does not yet automatically act onit. Users can manually subscribe to individual mirrors now. TheGuardian Project app repo is currently setup for all of these types ofmirrors:

We added support for two sources of metadata about apps. Fdroidservercan now automatically upload all new release tohttps://androidobservatory.org and https://virustotal.com. These bothprovide rich sources of metadata about apps and malware, viewable viaweb pages or accessible via an API. They both are based on the SHA256hash sum as a unique ID, so it is easy to link an APK on a device tothe data on those services. This data will be used to alert the userto known malware in the new “Updates” tab of F-Droid client.

We have two prototype libraries for ensuring that apps have areliable, trusted update channel no matter where they were downloadedfrom. There are lots of custom versions of this, from Firefox toSignal. The libraries that we are creating are standardized, freesoftware libraries. They also integrate with the whole F-Droideco-system, using the same tools to manage the server-side as are usedfor F-Droid “repos”. This provides the flexibility for app developersto mix and match the features they need, like direct app updates via adedicated app repo, updates via https://f-droid.org, confirmedreproducible builds of releases, “collatoral freedom” mirrors, etc.

Our first test implementations for these new libraries will be Zom forthe direct updates, and Ripple and Location Privacy for the F-Droidupdate channel.

The results of the survey have been compiled, and the public report isnearing completion. We ran user tests of the fdroidserver tools in ahandful of locations. We were unable to run the tests in EasternEurope as we had hoped.