Blog Archives

About

For the Maps Web Application I was tasked with creating a system that could show the location and contact information of all the registered users from the Ida Institute’s website. Moreover, the system had to have a way to show the different groups that users belong to.

People within the countries had to be searchable by name.

There were several important decisions that had to be made to proceed with the development of this project:

What kind of map visualization technology can be used?

How to retrieve user data from the Typo3 CMS that is used for Ida’s website?

How to modify the features in the Maps project

How to show the groups that users belong to?

How to show the members in each of the countries?

How to embed the solution in the Ida’s website and control access to the data.

1. Map Technologies

Looking at the Map display technologies Google Maps usually immediately comes to mind. After examining their API it was decided not to use this solution because the location that was provided by the CMS was way too coarse (only the country was known).

I started looking at the alternatives and came by JVectorMap. This solution is JavaScript based and includes all the maps that the system requires.

2. Data Retrieval

Typo3 CMS provides a functionality to export the list of registered users in a CSV file.

First iteration of the system required manual uploading of the CSV file. This way of providing the updated user data worked but was really cumbersome on the person that had to keep the user’s list in the Maps system up-to-date.

I was not given access to the main database where user’s data is stored and there we no API to retrieve it by other simple means. This is where an idea to move the file retrieval to the Maps application was born.

I managed to login into the Typo3’s admin panel from the Maps code and retrieve the CSV file.

This was done by analyzing how the login process is handled in the CMS’s login page.

After filling in random Username and Password values and trying to login while capturing browser traffic using Fiddler the following WebForm values we determined to be required to perform a login into the CMS’s backend and get access to the people list.

The challenge and the userident fields were changing when trying to login with the same credentials.

Closer inspection of the page source revealed that there is a form that has the required fields set up as hidden inputs.

The challenge value can be extracted by executing a GET request to the login page and parsing the returned text to find the challenge value. The only unclear bit was the way that userident field is used and where it’s value come from. Looking at the form definition there is a call to doChallengeResponse() javascript function before the form is submitted.

It seems that the way the userident field is calculated consists of building a string that contains the username, a MD5 hash value of the password as well as the form’s challenge value separated by the colons (str = document.loginform.username.value+”:”+password+”:”+document.loginform.challenge.value) and calculating a MD5 hash value of the aforementioned string. So I just re-implemented the userident calculation in C#.

After all the required values are ready to be submitted to the Typo3 a POST request is executed using System.Net.Http.HttpClient class and the authentication cookie is used for the next request that retrieves the the CSV file.

The whole process is logged in real time and can be viewed later by the system administrators (emails in the image below are obfuscated for privacy reasons).

3. Settings Panel

To make the web application customizable a simple settings panel was created that allowed to manually upload the CSV file, perform country name mapping, select visible groups (explained later) as well as forcing the data refresh (usually it happens every few days).

4. Groups

The groups are created by parsing the CSV file and extracting all the unique groups that people belong to. All new groups are then stored in the Maps database as separate objects with their visibility set to hidden. This can be changed form the Settings Panel and it makes the group visible in the front page.

After website users click on the group map results are filtered to include only the Ida Members belonging to that particular group.

5. Ida Members

Once a country is selected on the map website users are redirected to a page showing a list of all the members in that country. Since the height of the user boxes could not be predetermined because of different image size and different amount of data the layout had to adapt to different size boxes. Masonry library was used to take care of it. It arranges the grid elements into the most fitting positions.

6. Embedding the Solution

The Maps Web Application was included in Ida CMS’s page as an iframe. The only problem is that the height of the iframe is not know before the page loads. This issue was mitigated using IFrame Resizer libary. After the page is loaded it dynamically adjusts the frame height to match the height of the content inside the iframe.

Conclusion

At the time of writing this post, the project is more or less finished and waiting for the approval to be integrated into the main Ida’s site. The solution for automatically getting the users from the CMS is probably a bit hacky, but without access to data or asking the CMS administrators to build an API for it (which introduces additional cost) this is probably the easiest way to do it.