Somewhere along the way in July (after the second milestone), Jon decided to submit our project for the iCreate Mobility Challenge 2014, which was an NUS based mobile application development competition organised by the Computer Centre and sponsored by Microsoft BizSpark, M1, and People’s Association, just to name a few. I didn’t think that we were qualified to begin with, and I honestly didn’t think much of our project, which was essentially just a polling service. We later found out that there was already an existing company built on the idea.

Having neither a business plan for it (c’mon honestly, it was a summer project done to clear credits) nor was our web application a ‘real’ mobile application, I honestly wasn’t expecting this:

I would not try to justify or claim that these are the reasons why we won 3rd place, but going through with this taught me a couple of things:

You will never know until you try

Self-judgment does not give your product value. Other people’s judgment and perception of your product do.

A friend used to joke about this with a story: Let’s say some higher power wishes to reward you, you would probably want to make it easier for him to do so by buying a lottery ticket.

Only this time, it doesn’t just boil down to pure luck. You have worked on a product, and even if you don’t think much about it, someone else probably does. And when they do…

Appearances are all that matter very important

Feedbaker needed some marketing, and I had to do a pitch on its use-cases and features. When it comes to selling something, appearances matter a lot. Things that look better always get more attention than things that don’t (what applies in dating applies everywhere). This however, does not mean that dressing junk up nicely means that it’s going to sell.

There are just too many things that we worked hard to build upon this factor: our User Interface, colour schemes, layouts, posters. I had to also work on myself.

When people want to support/buy an idea, they are not buying just the product, they are paying for the person behind the idea.
It’s an entire package. Really.

Once again, I’d like to thank the organisers for this opportunity and the many people who have supported our project.

As we arrive at the third and final milestone of the Orbital Programme, we’d like to take stock of our lessons learnt and changes since the second milestone. Also, to review the decisions brought up in the previous post.

‘My Polls’ page (app-polls.html)

We have made some changes to improve the layout and usability. We felt that the polls interface is critical to the user’s experience of the application.

Firstly, the status of each poll is now dynamic with regards to the actions that the user can perform on each poll.

While there are no bulk action functions, we feel that the current application does not need housekeeping for the questions that the user creates due to a clear timestamp present in the interface.

Next, we have also included a ‘Help’ button that serves to guide new users. We felt that the application may not be as straightforward and self-explanatory to the user (as much as we have already catered for the usability of the application). The button sizes for the actions have also been enhanced to allow for easier clicking and interaction on mobile devices as well as on the desktop.

Some work has also gone into the inclusion of a column known as the Description – which we primarily conceived as a ‘Tagging’ feature we hoped to put in place for better organisation of different polls together to sort and display. Instead, the description is provided as a plain text attribute for each poll. This serves as an indicator for the user to differentiate between identical polls, but meant for different situations.

Final cleaning up and fixes

Since we have decided to get rid of some redundancies, we had to rebuild the routes on the front end to serve the correct pages for the appropriate links that we have provisioned for. Pages such as app-dashboard.html and about.html have been removed entirely from the app, keeping our focus mainly on the poll interface and the core functionalities of our question-and-answer features.

The results page now contains the option for users to toggle between different data visualisation methods on Google charts (incomplete, not ready for UAT) as well as an export function, which we have created to help presenters take stock of their responses offline to do operations such as a sum, min-max, etc for insightful analysis.

We’ve also provided for a hyperlink to the poll answering page on the ‘Get Link’ page for convenience of the presenter to navigate there should it be necessary.

Features that could not meet the final version

As much as we hoped to provide for a more complete Feedbaker.com to serve presenters, we were unable to finish up features that we have listed on our README:

Open project out to anyone to use

We have stuck to NUS OpenID despite the feedback to open it up to the public to use. We were looking at the option to completely remove the need for a login, where a user can immediately use the application freely and however he/she likes. However, we ran into a dilemma on whether we should go on to cater for anonymity in the polls, with how we should provide the option to bypass the authentication that is necessary to ensure that we have human responses.

Choice of Data Visualisation (Pie, Bar, Column)

As of now, we have provisioned for 3 different visualisation methods such as the Bar, Pie, and Column charts – all insightful for the quantitative sum of responses that would make sense on these charts.

(However, the visualisation techniques are not finished and we are unable to complete the implementation in time for this final milestone.)

“Youtube Live Comments” concept of open-ended results (comments)

This didn’t make the final cut as well as much as we wanted to. Solving the problem of the answer schema deemed to be too big a task for either one of us to deal with on ourselves without meeting up, and since we are miles apart, it was difficult to implement this portion given our priorities over the course of this project.

Self-assessment and reflections

Looking back on the 3 month journey (well actually 2, considering June was a time-vacuum which we completely left Feedbaker alone in its MVP state), we’ve touched and familiarised ourselves with the technologies we have summed up in this post.

Throughout the course of orbital, we found ourselves dabbling beyond the requirements and needs of Feedbaker as well to further our learning on other technologies.

For Jonathan in Berkeley, he has been actively engaging himself in conferences such as the Google I/O event, ForwardJS conference, as well as participate in the AngelHack and Outside Hacks Hackathon events which I am sure has further honed his skills and knack as a developer and product spokesperson.

For myself who has remained in Singapore, juggling a side project and the internship has been a strong practice in maintaining priority of tasks. I have also went for an AWS conference to learn more about Amazon’s cloud services and how to leverage on AWS to build, test, and develop applications that I may find myself doing in future. Besides that, I have also taken the time to dabble with Ruby on Rails using this guide to prepare for the Red Dot Ruby Conference in June. Feedbaker was my very first web development project and a fresh new endeavour that has taught me much over the many posts we have documented in this blog. I’d like to wrap up based on the important skills/traits I felt was important in seeing the successful state of this project in its current form:

Fact-finding

It isn’t about what we knew that was important, but how we go about finding what we needed to know. Most of the time, trial and error was an imperative in figuring out what works and what does not. There were things on StackOverflow that I could not understand at times, and very often find myself in a situation where the documentation was not enough. Knowing the appropriate terminology to the things we want to find out more about helped in getting to the pages and answers that we were looking for.

Building incrementally

Skills that I picked up from CS1010 are truly the basics that we could not have lived without. For every feature that we wanted to develop:

Get the most basic function to work first

Build on the content/data

See if it still works

Repeat from 1

Thanks to the tools such as JSHint, Grunt, and plugins on our text editors, this process was hastened and we got by on existing web scaffolds (Yeoman) to build Feedbaker up very quickly (3 weeks) based on this method.

Getting Feedback

Since we were building an application on getting feedback, we actually had to get feedback to know more about what we can do, what could be done better, and what we should not do. Maintaining a constant feedback loop was important in figuring out what was necessary and what was not. Things that got us to focus can be essentially broken down into 3 main tiers (see also the MoSCoW Method):

MUST-haves: What is essential (CORE features)

NICE-to-haves (Augmentations)

NO-NEED-to-haves

Going through the milestones (although we did not follow it strictly in its entirety) helped us to look at how a software development cycle feels like, and how our roles as a pair of developers are crucial in making sure that we trim the excess from the product to ensure that we meet our deadlines and expectations.

All in all, we feel that the Orbital experience has helped tremendously in accelerating and facilitating our growth as students in SoC, and as computing students that ought to sharpen our programming competencies.

We hope that this project blog has helped those who have accompanied us in this journey and has provided insights into our methodology and thoughts.

With my internship coming to an end, I have found more time to get back to working on the application. Jon has been helpful by being available through a local VoIP number he applied for for about SGD40-odd a year. I think it’s really useful considering how it charges 2-cents per minute of outgoing talktime with a really stable connection and audio quality.

Coming back to the application, we realised that there still remains much to be done, and that we had to iterate through the features and things we have initially reserved for. Also, I seemed to have forgotten quite a lot of what has been done, going through the commit logs was great for a refresher. And then there was this, a style guide for JavaScript I found especially enlightening and useful.

After having a few friends try it out, we have made some interesting observations about how we could improve our current user experience. We’ve figured that making Feedbaker NUS-centric for now is for the purposes of making the polls non-anonymous. Our focus in building this application, is to provide a solution for presenters (professors and teaching staff) to obtain live responses. We decided that we want to focus on optimizing the user story from the perspective of a lecturer, and everyone else who wishes to use it won’t matter as much.

As such, there were important decisions that had to be made as well.

The Dashboard had to go.

Indeed it had to go, there will never be enough data for us to meaningfully (semantically) create a dashboard page to give an overview on poll data. It just didn’t make sense to me.

We needed a revamp of the ‘My Polls’ page to truly encapsulate a single view/page interface.

Thinking more along the lines of the UX, we felt that with the Dashboard gone, we needed to beef up the My Polls page more, and to inform users more about what they were seeing at first glance.

Create a modal popup to guide the user on how to use the application –minor feature – additional functionality on the page gives bonus for interactivity.

Allow for more intuitive navigation and usability of the page

Some modifications were made to the schema to allow the lecturers to ‘tag’ descriptors to their polls, giving more meaning to the poll questions created and also to differentiate between identical questions that are meant for different classes.

An export feature

The export function was by far the most value-added feature that we felt had to exist in our application due a point raised early in the first peer-evaluation regarding our point of differentiation. With an export function, lecturers can now leverage on the .csv formatted data extracted from the system to log user activity that is non-anonymous – perfect for later implementing pivot tables on excel for decision making on all kinds of use cases. The possibilities seemed to coincide with our intentions.

We’re glad that Feedbaker has been shortlisted out of 13 applicants for the iCreate 2014 competition organised by the school. While we aren’t exactly a mobile application, we certainly hope the effort put into creating a fully responsive web application can survive the cut.

This year, I was really fortunate to have a chance to attend Google I/O, Google’s annual developer-focused conference. The two-day conference was held in San Francisco, and featured highly technical, in-depth sessions focused on building web, mobile, and enterprise applications with Google and open web technologies.

Apart from the exciting keynote address where Google’s latest innovations are announced, I also got to learn a lot from the many technical sessions that covered design, development and distribution.

While I was there, I also managed to get some UX feedback for Feedbaker from Matt Gaunt (Developer Advocate for Chrome) and another Googler whom I didn’t manage to get his name.

As Jon and I will be busy throughout the months of June and July, we have dedicated the past few weeks to work full-time on Feedbaker. In a short span of 3 weeks, we spent a total of more than 250 hours working on Feedbaker.

In building Feedbaker, we learnt and made use of many technologies and components that helped simplify coding and collaboration. The rest of this post will be a summary of the components we used, some of which may have been mentioned in previous posts with greater detail.

Our team has a git repository hosted at Github. This helps us to collaborate as we work together on different parts of the app and also logs each change made so that the other team member is aware.

Node.js is a platform built on Chrome’s JavaScript V8 engine. Our team made extensive use of Node.js as we built Feedbaker. Not only is Feedbaker served through Node.js, we also use other packages that run on Node.js to help build, test and develop Feedbaker.

Yeoman is a Node.js module that helps scaffold modern web applications. Our team used the angular-fullstack generator to kickstart our project. This generator helped us to scaffold a webapp that runs on an Express server, with AngularJS as the client-side JavaScript MVC framework and MongoDB as the database.

Grunt is a JavaScript task runner that automates the app’s development, testing and building process. As part of the build process, Grunt automatically compiles Sass into CSS and injects bower dependencies into the app. It does this by watching certain files for changes, and refreshes the webapp once the build process completes. When the app is being built for production, Grunt takes care of minifying HTML, CSS, JavaScript and even images. It also concatenates the many JavaScript and CSS files into a single file. All this is done to optimise the page loading time. Minification increases the page load speed by reducing the file size, while concatenation does the same by reducing the number of requests made to the server.

JSHint is a JavaScript code quality tool. It helps to detect errors and potential problems as we code, and at the same time enforces JavaScript best practices. JSHint works with Grunt to alert our team of any potential issues with our code.

Express is a web application framework for node. Feedbaker uses express to serve both static pages and dynamic content.

Our team also configured Nginx as a reverse proxy for our web application. This allows us to host multiple node applications on a single server.

To enable real-time display of poll results, Feedbaker uses WebSockets to allow for low-latency bi-directional connection between client and server. Socket.IO helps establish a WebSocket connection and allows data to be pushed to the client when fresh data is available.

Passport is an authentication middleware for NodeJS. It works seamlessly with Express to handle user authentication. In our app, we used Passport to authenticate users with their NUS OpenID.

Mongoose is a MongoDB object modeling tool designed to work in an asynchronous environment. Our team used Mongoose to interface with MongoDB, simplifying things such as validation of models.

Forever

Forever is a simple CLI tool for ensuring that a given script runs continuously. Our team makes use of this node module in our production server to keep the Feedbaker application running.

Bower is a package manager for front-end packages. It eliminates the need to commit vendor codes into the repository. Bower maintains a list of front-end packages the app needs in a file called bower.json. Dependencies are resolved automatically and are downloaded straight from the package maintainer’s git repository.

npm is the official package manager for Node.js. Similar to Bower, npm takes care of all the packages needed to build, test and deploy the webapp. npm also tracks these packages in the app’s package.json.

Having completed the implementations of the poll processes and actions to our satisfaction, it was time to do some layout for our front page to showcase our Minimum Viable Product (MVP). While browsing themes on wrapbootstrap.com and bootswatch, we’ve taken inspirations from two themes, namely one known as Qubico and the other is Delta. We’ve also decided to use Flatly as our base theme to begin our build. But before anything could start, we had to dig up early sketches in our design process to come up with our…

Home page layout

What went into our front page was rather simple and straightforward. We decided that it had to contain a few of the main things we had to talk about what Feedbaker does (which was rather difficult considering how simple the app was).

Introduction

Features

How to create a Poll

Story behind the app

Later on we added on more sections:

Technologies

Call to Action

Footer

From scratch

The first thing that needed work on was the top navigation bar on the homepage. We decided that we needed to produce a logo, and thus found flat-icons from iconfinder.com to use as a placeholder for our app icon. Followed by navigation buttons which we initially named and catered for ‘Home’, ‘About’ and ‘Login’/’Logout’. And thus we began with a fixed header that remains as the user scrolls.

The process was tedious but necessary, the development of the frontpage took us a total of 3 days because of deliberation over the design and numerous iterations over how the colours should look and over what icons should be used. Essentially, it boiled down to a simple 5 step cycle:

View inspector to edit in browser to verify the look (grunt was takes quite a while to compile changes for the .scss files)

Rinse repeat steps 3 – 4 until section is done.

Note: Our css file is written in the .scss format as it caters for a more ‘modular’ way at grouping css together. Our grunt scripts are configured to handle and compile the scss file into full fledged css file later on for the page will load from.

Icons, Graphics, Pictures

To make things slightly more interesting, we procured .png icons from various sources on the web and used photoshop to tweak the image sizes. Where image icons were heavy and unnecessary, we used glyphicons that came together in the bootstrap package for our graphical representation/aids in the front display as well. Initially we were trying out Font-Awesome, but it was broken in production and despite some attempts at fixing, it didn’t seem to compile due to the minification that was caused by the grunt build. Also, it didn’t seem to make a lot of sense to use Font-Awesome just for merely 4 icons.

Responsive Web Design and Media Queries

Jon shared an interesting book that really took the explanation and principles of responsive web design to another level with Ethan Marcotte’s Responsive Web Design.

With a brief skim on the book contents, and Bootstrap’s helpful CSS documentation on it’s grid system, things were very easy to configure when all we needed to do was to figure out how wide we want the content to be in each section, and correspondingly use col-md-1 to col-md-12 to play around with the columns in the row container. This helped us save a lot of time manually configuring the padding and layout positioning on the .scss file itself.

There are also some responsive utilities which I found very useful in hiding some things that may not make sense to show in the mobile interface that we might want to put inside the web page for a desktop view. Responsive utilities such as .hidden-xs allows for viewing on tablets and above, but not necessarily on small phone screens.

Revamp and Touchup

As we alternated through the sections with a blue colour scheme, we eventually moved on to fix up other parts of the home page, such as the Call to Action, which we felt should exist at the bottom after scrolling through the introductory information on the page (just like all respectable html5/css3 skinned product page you see on the web today). What was interesting to note here was how we decided to improve the flow of the user navigation from front page to login page to being redirected into the poll dashboard by revamping our initial login page, which was an extension of the home page on another view. This time, we decided to let the navbar go, and use a box to contain the text and buttons for the login page. This was done by providing a route for the Get Started button (our call to action on the homepage) for it to bring us directly to the login page itself. In any case, it would still be a 2+2-click through event for any user to get to the polls. (2 on our client, 2 on the NUS OpenID side)

Further plans

With 2 more months to go, we may have more to do and may improve on our functionalities and features for this app, and the frontpage may be due for more changes. Until then, we decided to forgo the about page and use the frontpage to capture all information intended for the user to read with a scrollspy implementation (see bootstrap/javascript) to get to the various section on our one-page slider layout. For now, we can’t get any happier than we already are.

Since we were looking into making things go real time, I took the opportunity to read up more on network related topics and this post shall cover what it was that I read through and learnt.

TCP/IP vs UDP

Taking the opportunity to look into the existing internet protocols, I had to touch on TCP/IP with all the OSI mumbo jumbo. I skimmed through this resource from Yale on TCP IP, and found out cool things like how it’s basically first invented for military purposes.

Then also after getting a better understanding of that, I also wanted to find out more on why not UDP? (Transport layer)

Websockets

Jon explained that there is this new thing in HTML5 known as websockets and how it has made internet awesome. There were several other ‘primitive’ and arcade ways we could have made our polling system “live” with techniques such as long-polling, which basically sets the page to refresh every few seconds (this sends requests to the server for the page to load the new results). Doing so may actually DDoS the server.

Socket.io

Thankfully there are also libraries out there for us to implement. Discovered Socket.io earlier on when we were discussing on how to implement the real-time/live results in our results page. Finally arriving at the point where we actually have to do it, we found ourselves back here.

However, getting Socket.io to work with our model layer was not easy, and we had run into several problems. Jon took over on this part to configure and fix up the entire websocket implementation for socket.io in this commit.

I guess the important take away here is that we have to think in terms of a ‘chat room’, that each time a socket is open, it serves the data posted by a user on another client to all the other users who are listening in the room. Which is essentially represented in this following snipper of code:

While integrating Feedbaker with NUS OpenID, I ran into some problems with mapping the OpenID identity to a unique user in the app’s user database. I found out that NUS OpenID accepts the NUSNET id both with and without the network domain, and depending on whether the domain is entered, the system returns a different OpenID identity.

For example, if I were to log in with “a0123456” as my NUSNET id, my OpenID identity would rightly be “https://openid.nus.edu.sg/a0123456“.

However, if I log in using “nusstu\a0123456” as my NUSNET id, I end up getting a different identity.

You are logged in as https://openid.nus.edu.sg/nusstu\a0123456.

When this happens, 3rd party applications that make use of NUS as an OpenID provider would identity this user as another different unique user.

After sending in a bug report to the NUS OpenID Developers group, I got a reply from Wai Peng, systems engineer from NUS SoC.

Thanks for finding this. It is _not_ desirable behaviour. I should probably code some checks into this. Will update when I fix it.

Meanwhile, as a temporary fix for the Feedbaker app, I decided to just replace out the domain portion of the identity.

With what I thought was effectively a 50% complete prototype, I decided that I have to meet up with Jon on the weekend to catch up on the project and get some more things ironed out together.

To give more breadth to the things Jon covered in the previous post, I’ll touch on more detail to the best of my memory.

We started off by reviewing the code that has been committed over the week, and discovered some interesting things about what JSHint finds error-prone in the existing scripts, such as naming conventions, as well as blank parameters.

‘My Polls’ page

Our initial focus was to also complete and sign off on the app-polls.html page, which is the page where the user views all the polls listed under his/her user ID. We revamped the poll page to just display actionable icons, which provided a more simple user interface and experience, and got rid of the confirm() prompt when activating/deactivating the polls (turns out JSHint does not like them either).

‘Get Link’ page + Short Links

We then went on to look into the ‘Get Link’ page, whereby we have previously decided on providing a short link to anyone to access the poll questions and answer interface.

The poll id as stored in the database is MongoDB’s ObjectId, a 24-byte hexadecimal string. This would be extremely inconvenient for typing out, especially on small mobile devices.

Initially, the decision was to interface with Google’s url shorterner (goo.gl) API to generate our short links for us. The other alternative was to use bit.ly. However, upon more deliberation, we realised that implementing our own custom short link would be more effective and would serve as a better learning point on how they work.

We also tried using ShortId and Hashids to come up with the short link, but eventually generated it randomly.

Math.random().toString(36).substring(2, 8)

What this gives us is a 36 character range with digits 0-9 mixed into a set of lowercase alphabetical letters.

‘View Results’ page

It was the second last actionable item that we have left to resolve (the last being the ‘Delete Poll’ action, which didn’t require a view page)

From experience gained in creating a new poll and the capturing of answers in the database schema. There was a better understanding of how to create the API for the results view. We set them up following the general steps of creating the respective API sections on the client side as well as the server (Express) side.

Quickly, we managed to also hack up a view for displaying the answers, and decided to use Google Charts as data visualisation. Initially the thought was to implement our own data visualisation diagrams, but figured that it would be much easier and more practical to simply employ Google Charts. The challenge however is making this page responsive, and to cater for a larger view of the charts on the desktop. Also, we are looking at the possibility of allowing users to choose between Pie charts, Line charts, and Bar graphs to suit their fancies.

Delete Poll

With some hands on, I was looking into implementing the deletion of a poll from the database. Having secured the API and set up of the respective client and server end requirements for allowing a DELETE request to be posted, I found myself going through a very tedious testing process of manually using the Mongo shell to test for three scenarios that might occur:

Deleting a poll but not deleting the answers

Deleting a poll that does not belong to the user

Failure to deleting a poll and end up deleting its the answers

Fortunately, with appropriate chaining of the callbacks, we managed to sort through securing the API to prevent the above scenarios from occuring, which leaves me with finding out how to implement a Bootstrap Modal through Angular to act as a confirm prompt (better UX) so as to delete the poll from the system. Reflecting the change immediately after deletion is also something to consider.