Overview of my work at KDE within GSoC

Hello again and welcome to my blog! In this post i am going to cover what happened since the first GSoC evaluation and give you some overview on the status of my work.

Since the last post I’ve been working on the implementation of the guitar plugin, along with adjusting the existing piano plugin to better suit to the new framework.

As you remember from my last post, minuet currently supports multiple plugins to display its exercises. To change from one plugin to another, all you have to do is to press on the desired instrument name: for now, only “Guitar” and “Piano” are available.

In the past couple of weeks, I’ve been deciphering the guitar notes representation and also the guitar chords. I don’t want to discourage anyone from learning how to play the guitar, but man.. It was so hard and tiresome.. Nevertheless, my previous piano experience helped me better understand the guitar specifics and get up to speed with the theory needed to complete my project.

Then I talked with my mentor on Hangouts and, using http://chordfind.com as a base (which is indeed a great start for beginners who want to learn guitar/piano and many other 4-strings instruments chords), we agreed on two specific representations for each cord: Major, Minor, Augmented, Diminished, etc. for chords with the root note in the C-E range or in the F-B range.

Then i started working at the core of the plugin guitar: to keep functional the piano keyboard, i had to implement the exact same methods used by the piano plugin. I won’t go into too much coding detail (the code is available on GitHub on my fork of Minuet and on the official GSoC branch when completed), but with a little twitch to the current ExerciseView component, I managed to create a guitar plugin that runs the Minuet’s chords exercises.

Hi everyone!
As the title of this blog post suggests, the first evaluation has been successfully passed, the project accomplishing all the proposed goals. Thus, let me present you the progress I’ve done for the first evaluation.

The very first thing I’ve took care was allowing multiple-instrument visualization framework, by modifying the old core architecture of Minuet. Instead of the one and only keyboard instrument, Minuet is now providing an instrument tab bar, enabling the end user select the desired instrument. The tab bar is currently loading the instrument within the first tab as default. After I’ve discussed with my mentor, Sandro, I decided to keep the piano keyboard as the default instrument contained by the first tab. In order to keep all the functionalities available and a simple architecture at the same time, I’ve created an InstrumentView wrapper above all the instruments. Its methods are called by the core ExerciseView, calling in turn the methods that each instrument should implement.

After changing the Minuet’s core in order to provide a multi-instrument framework, I started refactoring the piano keyboard view as a separate plug-in. Basically, for Minuet, a plug-in is nothing more than a bunch of QML files and a simple JSON file parsed in order to learn about the QML files’ location and load them dynamically in the instrument tab bar.

The next goal was to install each instrument separately as a plug-in, besides all other Minuet’s plug-ins. Like any other KDE application, Minuet, keeps its data files into /usr/share/ file-system directory. Thus, this was the place where our plug-ins had to be installed; actually, the exact path is /usr/share/minuet/plugins/. Installing a plug-in is simply done by using a CMakeLists.txt file telling the build system where the plug-in files should be installed.

Plug-ins are loaded as follows: firstly Minuet searches for plug-ins location and if the search succeeds, it will iterate through each instrument’s JSON file, building a QJsonArray containing the plugin name and path. This QJsonArray is passed to the InstrumentView, setting the contents of the Instrument Tab Bar. Having all the tabs ready, the TabBar will just wait for a tab to be selected and a QML file to be loaded.

Having the multiple instruments framework and piano plug-in ready, the next step was to test and analyze the code for possible technical issues or bugs. Also, a general clean-up was required before submitting the code for the review and first Google Summer of Code evaluation.

These are the achievements I’ve met for the first evaluation. Let’s discuss now about the current progress for the next evaluation:

I wrote a proper and comprehensive documentation for the piano plug-in, enabling anyone to create his/her own instruments plug-ins and loading them dynamically into Minuet.

I started implementing a new instrument from scratch. As I am very passionate about guitar, I chose to integrate it into Minuet’s instruments suite. Thus, I have already designed the visuals for the Guitar plug-in and added it into Minuet’s instrument tab bar. Now, I need to link the GUI to the core, work on the guitar’s specifics and make the plug-in fully functional.

In conclusion, GSoC proves to be a great experience thus far, being a pleasure to work on a project focused mainly on a subject I’m in love with: music. For further cool news, stay tuned!

It is a long time since I posted on my blog and frankly, i missed it. I’ve been busy with school: courses, tones of homework, projects and presentations.

Since last year i had a great experience with GCompris and KDE in general, i decided to apply in this year’s GSoC as well, only this time, i chose another project from KDE: Minuet.

Minuet is part of KDE-edu and its goal is helping teachers and students both novice and experienced teach and respectively learn and exercise their music skills. It is primarily focused on ear-training exercises and other areas will soon be available.

Minuet includes a virtual piano keyboard, displayed at the bottom of the screen, on which users can visualize the exercises. Using a piano keyboard is a good starting point for anyone who wants to learn the basics of musical theory: intervals, chords, scales etc. Minuet is currently based on the piano keyboard for all its ear training exercises. While this is a great feature, some may find it not quite suitable to their musical instrument.

My project aims to deliver to the user a framework which will support the implementation of multiple instrument views as Minuet plugins. Furthermore, apart from the piano keyboard, I will implement another instrument for playing the exercise questions and user’s answers.

This mechanism should allow new instruments to be integrated as Minuet plugins. After downloading the preferred instrument plugin, the user would then be able to switch between instruments. It will allow him to enhance his musical knowledge by training his skills using that particular instrument.

At the end of summer, I intend to have changed the current architecture to allow multiple-instrument visualization framework and refactor the piano keyboard view as a separate plugin. I also intend to have implemented a plugin for at least one new instrument: a guitar.

A mock up on the new guitar is shown below:

I hope it will be a great summer for me, my mentor and the users of Minuet, whom i want to offer a better experience by using my work.

Every journey eventually ends and though not expected nor desired, Google Summer of Code 2016 is officially over. These three months have been full of fun and good times but also hard work. I have learned the “Secrets of the Git”, how to approach and work with a big project, give myself smaller tasks and solve them in order to achieve the greater goal.

It’s been a great experience working with different people having the same objective: creating a reliable and educational piece of software but at the same time with a friendly interface for children. I interacted with teachers, students and developers from all around the worlds and I learned a lot from each and every one of them.

Participating at GSoC with GCompris was a very educational period for me and I hope I brought some improvement to the community as well. I hope my work will help children of all ages learn something new and have fun while interacting with my activities.

Update on my work:

In the second part of GSoC (after the mid-term evaluations), I finished Crane, Alphabetical Order and Share and started working on the hardest task of all: porting TuxPaint. A presentation of the old TuxPaint can be accessed here: http://gcompris.net/screenshots-en.html#tuxpaint .

First of all, I must admit that this is the hardest project I have ever worked on, mostly due to the lack of documentation on the subject and also because of its complexity, mixing many tools and functionalities.

That being said, in developing TuxPaint, I took the liberty to redesign the old version and give it a new look. So let me introduce… TuxPaint:

TuxPaint presents itself to the user with many useful tools like different types of brush, eraser, size selector, shapes: circle, rectangle, lines, text, undo and redo, load children creation or other images and the possibility of saving the current painting.

I’ll start presenting the brushes:

The pencil is used in the first image, where I drew the smiling face. The next three are pattern based brushes: you use them to draw circles, horizontal and vertical lines. Next are a spray brush and a pen tool, followed by a circle line and a crossing lines pen. The last tool is a blurred line, with extra shadow.

All tools can be customized to have any color the user needs. All he has to do is pick the desired color from the top part of the screen and the size he wants from the right panel.

Next tools on the list are represented by the shapes:

circle and rectangle

two types of lines:

Free-angle lines and Horizontal and Vertical lines

This slideshow requires JavaScript.

Another functionality TuxPaint brings to the user is the ability of writing text onto the painting:

The text color can be set by picking any color from top and the size can be changed by pressing on the “size” button, just like the customization of the brushes.

Next tools are “Undo” and “Redo”, which are self-explanatory: when pressed, “Undo” removes the last addition and “Redo” brings it back.

The user also has the ability to save his creation by pressing on the last tool, represented by a blue disk. This feature enables him to go back to any previous creation and start working at it from where he left it.

Pressing on the “Load” button brings to the view a side screen with images that can be imported to the main board.

The “TuxPaint” button brings to the view another screen which contains users’ previous creation.

Another feature that TuxPaint brings is the “reload” button. It resets all the board by erasing everything and painting the board in white. For security reasons, if the user has anything drawn on the board, a Dialog Message will appear, asking him if he wants to save his work. He can choose to save it or not, or to go back to painting. A similar Dialog Box will appear when pressing on the Home button, to prevent him from losing his painting.

This slideshow requires JavaScript.

In the development of TuxPaint, I used Qml Canvas 2d, and besides the qml documentation, I had to go through many HTML 5 canvas 2d documentation. Since this is uncharted land for me, it took me a lot of time and research in order to fully understand the context and the way canvas2d works.

Though finished, TuxPaint is not bug-free. It will take some time for me (and mentors) to continue testing it, but for now, it is functional and meets all the expectations.

My work for GCompris as part of GSoC is over now (except being fixing minor bugs on the not yet merged activities: Share, Alphabetical Order, PhotoHunter-Help feature and Paint) but my contribution is not over. I enjoyed my time spent with the KDE community, especially with the GCompris team so I plan to actively participate on the chat, give help to the new contributors anytime I am able to and help building great activities furthermore in the future.

Last but not least, I would like to thank the mentors and especially Johnny Jazeix for his 24/7 help and support.

Two months into GSoC, I must say that this summer is the best in my life. I have learned a great deal of programming, went jogging every day and had fun overall in my vacation.

If you followed my blog posts, you already know that before the actual start of the GSoC coding period, to better familiarize myself with the programming languages GCompris uses and also to prove my commitment and dedication, I contributed to GCompris by solving various bugs and developing two activities. “PhotoHunter” is an activity ported from the previous GTK version of GCompris. Its goal is finding differences between two similar images.

The second activity is “Share”, an authentic creation of mine. In this activity, the user is presented with a problem about a child who has some candies and wants to equally share them with his friends. The user has to drag the exact amount of candies from the left panel to the child friends’ areas on the main screen.

Now for the GSoC part of my work, I am happy to announce that “Crane” is already merged, and “Alphabetical Order” is coming fast from behind – everything is finished, but it needs some more testing.

Crane

In this post I will just mention the updates brought to Crane. For a more detailed presentation on the functionality of Crane, please check my previous posts.

For adding a pedagogical side to this activity, I decided to use as starter levels some easy words consisting of image letters connected one to the other. The user’s goal is to reproduce the model and by doing so he automatically learns new words.

To better teach the association between the two models, the levels are carefully designed to gradually increase the difficulty: for the first levels, the two boards are at the same height and the grid is displayed, then for some levels, the grid is removed, for others, the height of the two boards is changed, and at the end, both changes are made: the grid is removed and the heights differ.

This slideshow requires JavaScript.

At last, there are levels with small images that the user must match. As the levels increase, there are more and more images on the board, making it more difficult to match.

Alphabetical Order:

The second activity on my list for Google Summer of Code was “Alphabetical Order” – an educational game in which the user learns the alphabet by practice. Its goal is to arrange some letters in their alphabetical order: on top of the screen is the main area, where some letters are displayed from which some are missing. The user has to drag the missing letters from the bottom part of the screen to their right place in the top area.

As the difficulty increases, the levels become more and more complicated:

more letters,

more missing letters,

the letters from top are not always in the good order.

Pressing on any letter triggers a sound and the letter is read so the user (children) learns its spelling as well. In the configuration panel, these sounds can be turned on or off, by pressing one button. The language used by the activity can be changed in the same configuration panel.

Here we can find two more buttons: “Easy mode” and “Ok button”.

In “Easy mode”, when the user drags a letter to its place, if the answer is correct, a sparkle is triggered. If the answer is wrong, the letter will turn red for a few seconds.

If the “Ok button” is activated, the level will be passed only when the user presses the “Ok button”. If the answer is good, a sparkling effect will appear on all the letters, else, the wrong letters will glow red.

If the user’s answers are mostly correct for the first level, an adaptive algorithm will increase the difficulty and as he passes more and more levels, the algorithm will dictate the difficulty of the game. On the other hand, if his answers are mostly incorrect, the difficulty is lowered so he can better learn the alphabet with easier levels.

PhotoHunter

After finishing “Crane” and “Alphabetical Order”, I went back to “PhotoHunter” and “Share”: for the first one, I added a new feature – a Help Button. Pressing it once will move the two pictures to the center, one on top of the other one. A sliding bar will appear and as the user drags it to right, the two images will combine and reveal the differences. In this “Help Mode”, the user cannot press on the differences; he has to press the “Help Button” to exit the “Help Mode” so the images will go to their initial positions and then press on the difference again in order for it to be counted.

This is how the slider bar works:

A portrait view:

Share:

For “Share”, I added a new type of levels and a new feature: “Easy mode”. In “Easy mode”, the user can use the maximum number of candies given in the problem. If he gives more candies to a friend on the board, then he won’t have enough left for the others. On the other hand, if the “easy mode” is deactivated, the user can drag more than the exact amount of candies to each friend on the board needs. This addition forces the user solve the given problem and find its answer instead of guessing it by dragging the candies from one child to another.

Problem:

Solution:

The new levels consist of placing candies in some friends’ areas before the user starts playing. This feature makes him take in consideration the candies already added to the board and compute again to find the new solution.

I am currently working on porting TuxPaint, a paint activity in which children can have fun drawing lines, rectangles and circles or free-drawing their own creation. The next post will mainly cover the development of TuxPaint.

Being a Google Summer of Code student is one of my greatest achievements and until now, it has been up to my expectations. This is a “two week in the coding period” blog post and i hope you’ll find it interesting and detailed enough.

At the end, i added links to my GitHub’s GSoC branch and to an apk – an Android build (installer) of GCompris, containing my current work on Crane.

The community bonding period was very productive for me. Besides my usual school stuff, I have done research on my tasks and read more about qml and javascript languages. I also talked to the mentors regarding my activities and what path I should approach.

In this period Sagar helped me install an IRC bouncer, in order to receive all the messages in a chat room even though I am not online. This is a great tool that helped me to never miss a message that was addressed to me or important messages from the channel’s irc chat.

Because of school’s end i was very busy these last weeks. I had to both prepare for final tests, finish projects, solve some homework in order to enter the final exam session and work for my Google Summer of Code project. It was an intense period and hard to focus on both sides, but up until now, i managed to work it out pretty well.

Having already one activity finished and merged and another one 90% finished helped me in my early days of work. The first activity I decided to work on is Crane, a puzzle game which aims to develop children’s imagination and logic thinking. Crane was part of the previous GTK version and needed to be ported in the new Qt version. In this activity, there are two grids displayed, each one containing the same number of items differently arranged.

The one from right (or bottom in portrait view) represents the model, and the one from left (or top in portrait view) is the one the children “work” in. A building Crane is placed in the middle – it allows the children to move the items from the working grid by using the left, right, top and down arrows.

My new Qt version of Crane (looks pretty similar to the old one to me 🙂 )

The crane’s arm can be controlled with the command buttons displayed next to the crane’s body and switching from one item to another can be done by pressing on the desired item. When using a keyboard and a mouse, the player can press TAB or SPACE to switch to the next item and use the arrow keys on keyboard to actually move the items in any direction inside the grid.

Because on some Android phones with smaller displays the command box is too small and the player cannot properly press the moving arrows, I came up with an alternative. To improve the user experience, I added a Swipe effect: on a touchscreen device, in order to move the items, the player can use his fingers to swipe in any direction.

Video demo of Android version of GCompris’ Crane:

As you can see, the buttons are hard to press. As a side note, I still need to work on the swipe effect.

In the old version of GCompris, the dimensions of the window were locked, so there was only one mode: desktop mode. In the current Qt version, the window can be resized in any way the user wants. To address this issue, i had to adapt the activity to the new demands.

In my version of Crane, the graphics are dynamically linked and as a consequence, the layout will adapt to any screen size. See below two screenshots of both landscape and portrait mode on a Samsung Galaxy S6:

The levels are randomly generated in order to keep the game fresh and new. The difficulty changes though with each level. It starts with four items and it increases with every level that the user passes.

For the first levels, this version of Crane currently misses a grid on each board. This addition will make it easier for children to find the exact spot where the item has to be placed.

After a while, i finished my first GCompris activity. Merging my code to the master makes me happy and motivates me to continue contributing. It’s nice to see that my work is appreciated and know it will help children from all around the world develop their skills while having fun at the same time.

Let me introduce you to PhotoHunter:

PhotoHunter is an activity which aims to develop the user’s observations skills and enhance his attention for details. The objective is to find a given number of differences between two similar images displayed on the screen. The two images are placed one next to the other in Desktop mode and one above the other in portrait mode.

When the user finds a difference, he can tap on it, and it will be revealed in the right/bottom image, to match the model image. On the bottom-right corner, there is a counter, which indicates how many differences you discovered and how many you still need to find.

While working at PhotoHunter, i encountered several problems and difficulties but with research and help from the GCompris community, especially from the mentors, i managed to solve them. It has been a great experience and i learned a lot during its development.

Having my code accepted by the mentors and developers of GCompris means a lot. It gives me a sentiment of fulfillment and satisfaction and it makes me feel more positive about finishing my other activity – Share. Regarding my future work at GCompris, porting PhotoHunter gave me a better idea of how i should approach my tasks, especially during the Google Summer of Code program.

PhothHunter will be available in the 0.6 release, but if you want to see it and test it or give it a try, you can build GCompris from sources, by cloning the GitHub repo.
A video preview of PhothHunter is also available on YouTube.

P.S. Thank you for all the help and suggestions i received during the development of PhothHunter.