Subscribe

Tag: Susi

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login usingtravis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tarfastlane.json key.jks

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

SUSI.AI has 7 different user roles (bot, anonymous, user, account creator, reviewer, admin and bureaucrat). Users are classified based on these user roles. To distinguish each user’s identity, SUSI Server uses email id as the key. So before using any of the services user needs to login with SUSI. In this blog I discuss how SUSI makes login a smooth experience for the users.

Every app should provide a smooth and user-friendly login experience, as it is the first point of contact between app and user. SUSI Server uses DAO in which accounting object is stored as JSONTray. The Susi server provides the required API endpoints to its web and mobile clients.

Login service across various web, iOS, android and IoT client allows users to login, logout or to check their login status. Login endpoint uses the HttpServlet class which provides methods, such as doGet and doPost, for handling HTTP-specific services.Thus the LoginService class was inherited from AbstractAPIHandler and implement APIhandler interface. In Susi Server the AbsrtactAPI handler extends a HTTPServlet which implements doGet and doPost ,all servlet in SUSI Server extends this class to increase code reusability.

The AbstractAPIHandler checks the permissions of the user using the user roles of and comparing it with the value minimum base role of each servlet. Thus to specify the user permission for a servlet we need Override the getMinimalBaseUserRole method.
Below given is the end point at which requests are made.

http://api.susi.ai/aaa/login.json

Login servlet handles various types of requests. Requests can be made in following ways with different parameter lists:

In this type of request an access token is created and stored with some expiry time, by default the expiry time is set to one week. The generated access token is then sent as a response from server with success flags having value true

For this request server checks if the password is valid by comparing it with the hashed password, if the comparison fails an exception is thrown for client to display “Invalid Credentials” Otherwise response is returned for successful login of the user.

SUSI Web Chat uses Travis CI with a custom build script to deploy itself on gh-pages after every pull request is merged into the project. The build system auto updates the latest changes hosted on chat.susi.ai. In this blog, we will see how to automatically deploy the repository on gh pages.

The Travis configuration files will ensure that the project is building for every change made, using npm test command, in our case, it will only consider changes made on the master branch.

If one wants to watch other branches one can add the respective branch name in travis configurations. After checking for build passing we need to automatically push the changes made for which we will use a bash script.

SUSI Webchat,SUSI Android app, SUSI iOS app are various SUSI clients which depend on response from SUSI Server. The most common response of SUSI Server is in form of links. Clients usually need to show the preview of the links to the user. This preview may include featured image, description and the title of the link. Clients show this information by using various 3rd party APIs and libraries. We planned to create an API endpoint for this on SUSI Server to give the preview of the link. This service is called LinkPreviewService.

The TextCrawler function accept two parameters. One is the url of the website which is to be scraped for the preview data and the other is depth. To get the images, description and title there are methods built in. Here we just call those methods and set them in our JSON Object.

The getImages method takes the HTML document from the JSoup and find the image tags in that. We have given the imageQuantity parameter in the function, so accordingly it returns the src attribute of the first n images it find.

The SUSI skill CMS is an editor to write and edit skill easily. It follows an API-centric approach where the SUSI server acts as API server and a web front-end act as the client for the API and provides the user interface. A skill is a set of intents. One text file represents one skill, it may contain several intents which all belong together. All the skills are stored in SUSI Skill Data repository and the schema is Models -> Groups -> Languages -> Skill -> Intents.

http://api.susi.ai/cms/getAllLanguages.json

To get the list of all the languages stored in Susi Skill Data repository, we used the endpoint provided by SUSI Server.

One can access based on model and group. We can get the required parameters through call.get() method where first parameter is the key for which we want to get the value and second parameter is the default value. We make a json array of languages ISO codes and send it as a response for this endpoint.

Next, to display the actual name we used a ISO-639-1 npm package for language name translation from their respective codes.

and make an Ajax network call to get the ISO codes from the above endpoints. We parses the response in data object, using ISO6391 npm package we added the native language name in the drop down for the languages. We defined a Universal language with code ‘xx’, for skills which is related to mathematical and computational problems.

SUSI Skill CMS is a web application to create and edit skills. In this blog post I will be covering how we made the skill deleting feature in Skill CMS from the SUSI Server.
The deletion of skill was to be made in such a way that user can click a button to delete the skill. As soon as they click the delete button the skill is deleted it is removed from the directory of SUSI Skills. But admins have an option to recover the deleted skill before completion of 30 days of deleting the skill.

In this we get the model name, category, language name, skill name and the commit ID. The above 4 parameters are used to make a file path that is used to find the location of the skill in the Susi Skill Data repository.

Finally we add the changes to Git. DAO.pushCommit pushes to commit to the Susi Skill Data repository. If the user is logged in we get the email of the user and set that email as the commit author. Else we set the username “[email protected]”.

Then in the caretaker class there is a method deleteOldFiles that checks for all the files whose last modified time was older than 30 days. If there is any file whose last modified time was older than 30 days then it quietly delete the files.

Susi Skill CMS is a web app to edit and create new skills. We use Git for storing different versions of Susi Skills. So what if we want to roll back to a previous version of the skill? To implement this feature in Susi Skill CMS, we needed an API endpoint which accepts the name of the skill and the commit ID and returns the file at that commit ID.

In this blog post I will tell about making an API endpoint which works similar to git show.

In this we get the model name, category, language name, skill name and the commit ID. The above 4 parameters are used to make a file path that is used to find the location of the skill in the Susi Skill Data repository.

This servlet need CommitID to work and if commit ID is not given in the request parameters then we send an error message saying that the commit id is null and stop the servlet execution.

And then one can the loader to read the file. From the treeWalk we get the object and create an output stream to copy the file content in it. After that we create the JSON and put the OutputStream object as as String in it.

Signing key management of Android Apps is a hectic procedure and can grow out of hand rather quickly for large organizations with several independent projects. We, at FOSSASIA also had to face similar difficulties in management of individual keys by project maintainers and wanted to gather all these Android Projects under singular key management platform:

To handle the complexities and security aspect of the process, this year Google announced App Signing optional program where Google takes your existing key’s encrypted file and stores it on their servers and asks you to create a new upload key which will be used to sign further updates of the app. It takes the certificates of your new upload key and maps it to the managed private key. Now, whenever there is a new upload of the app, it’s signing certificate is matched with the upload key certificate and after verification, the app is signed by the original private key on the server itself and delivered to the user. The advantage comes where you lose your key, its password or it is compromised. Before App Signing program, if your key got lost, you had to launch your app under a new package name, losing your existing user base. With Google managing your key, if you lose your upload key, then the account owner can request Google to reassign a new upload key as the private key is secure on their servers.

There is no difference in the delivered app from the previous one as it is still finally signed by the original private key as it was before, except that Google also optimizes the app by splitting it into multiple APKs according to hardware, demographic and other factors, resulting in a much smaller app! This blog will take you through the steps in how to enable the program for existing and new apps. A bit of a warning though, for security reasons, opting in the program is permanent and once you do it, it is not possible to back out, so think it through before committing.

For existing apps:

First you need to go to the particular app’s detail section and then into Release Management > App Releases. There you would see the Get Started button for App Signing.

The account owner must first agree to its terms and conditions and once it’s done, a page like this will be presented with information about app signing infrastructure at top.

So, as per the instructions, download the PEPK jar file to encrypt your private key. For this process, you need to have your existing private key and its alias and password. It is fine if you don’t know the key password but store password is needed to generate the encrypted file. Then execute this command in the terminal as written in Step 2 of your Play console:

You will have to change the bold text inside curly braces to the correct keystore path, alias and the output file path you want respectively.

Note: The encryption key has been same for me for 3 different Play Store accounts, but might be different for you. So please confirm in Play console first

When you execute the command, it will ask you for the keystore password, and once you enter it, the encrypted file will be generated on the path you specified. You can upload it using the button on console.

After this, you’ll need to generate a new upload key. You can do this using several methods listed here, but for demonstration we’ll be using command line to do so:

The command will ask you a couple of questions related to the passwords and signing information and then the key will be generated. This will be your public key and be used for further signing of your apps. So keep it and the password secure and handy (even if it is expendable now).

After this step, you need to create a PEM upload certificate for this key, and in order to do so, execute this command:

After this is executed, it’ll ask you the keystore password, and once you enter it, the PEM file will be generated and you will have to upload it to the Play console.

If everything goes right, your Play console will look something like this:

Click enrol and you’re done! Now you can go to App Signing section of the Release Management console and see your app signing and new upload key certificates

You can use the SHA1 hash to confirm the keys as to which one corresponds to private and upload if ever in confusion.

For new apps:

For new apps, the process is like a walk in park. You just need to enable the App Signing, and you’ll get an option to continue, opt-out or re-use existing key.

If you re-use existing key, the process is finished then and there and an existing key is deployed as the upload key for this app. But if you choose to Continue, then App Signing will be enabled and Google will use an arbitrary key as private key for the app and the first app you upload will get its key registered as the upload key

This is the screenshot of the App Signing console when there is no first app uploaded and you can see that it still has an app signing certificate of a key which you did not upload or have access to.

If you want to know more about app signing program, check out these links:

Skills are basically a set of rules which respond to the user’s queries through any client app. All the skills are defined in the SUSI Skill Data repo where the user’s queries are matched with the already present skills and the server responds accordingly. Apps like Alexa, Google Assistant have an interface to view skills with a set of sample queries that can be used therefore, we are adding the same skill display UI in the SUSI iOS app.

Implementation

All the skills are arranged into categories or groups, so we first need to fetch all those groups followed by fetching skills from each group. For listing all the groups, we use the endpoint below:

Dynamic Table Views are used at places where there may be any kind of reusability of cells. This means that there would exist cells that would have the same UI elements but would differ in the content being displayed. Initially the Settings Controller was built using UICollectionViewController which is completely dynamic but later I realized that the cells will remain static every time so there is no use of dynamic cells to display the UI hence, I switched to static table view cells. Using Static Table View is very easy. In this blog, I will explain how the implementation of the same was achieved in SUSI iOS app.

Let’s start by dragging and dropping a UITableViewController into the storyboard file.

The initial configuration of the UITableView has content as Dynamic Prototypes but we need Static cells so we choose them and make the sections count to 5 to suit our need. Also, to make the UI better, we choose the style as Grouped.

Now for each section, we have the control of statically adding UI elements so, we add all the settings with their corresponding section headers and obtain the following UI.

After creating this UI, we can refer any UI element independently be it in any of the cells. So here we create references to each of the UISlider and UISwitch so that we can trigger an action whenever the value of anyone of them changes to get their present state.

To create an action, simply create a function and add `@IBAction` in front so that they can be linked with the UI elements in the storyboard and then click and drag the circle next to the function to UI element it needs to be added. After successful linking, hovering over the same circle would reveal all the UI elements which trigger that function. Below is a method with the @IBAction identifier indicating it can be linked with the UI elements in the storyboard. This method is executed whenever any slider or switch value changes, which then updates the UserDefaults value as well sends an API request to update the setting for the user on the server.