Performance in mobile Business Apps – #1: Verify Unique Index

This is the first part of a series about performance in mobile business apps. Developing apps for the enterprise always is a great challenge: you have to deal with servers from backend, Online / Offline – mode, complex workflows and your app should help the employees to get their work easy done. Performance is one of the keys to success. I’ll talk here about some scenarios you may run into if developing business apps.

Getting data from a server via REST or SOAP you have to rely on data sent to you and sometimes it’s not as expected. Per ex. in one of my Enterprise projects from time to time I had to sync master data where 150k records must be fetched from server and stored locally into SQLite. From API documentation there was one field as unique index declared.

Persisting bulk data into SQLite should be done in a batch transaction. (More about this in another part of this series)

Did so and suddenly SQLite reports an error: INSERT failed because of “Index not unique”. From batch transaction SQLite doesn’t tell you which records are duplicates. To verify this I inserted record by record and found 20 duplicates and could report to IT. Inserting record by record was very slow: 80 minutes instead of some seconds. So what to do ?

To verify data it would be the best to check if the indexes are unique before inserting them into the SQLite. In this case index was a String field and my first idea was to use a QStringList:

If there’s something slow in your app – it’s always worth to search for the reason. I never would have thought that a QStringList is so much slower in this usecase as a QVariantMap.

… updated 2015-09-26 ….

thanks to feedback from twitter @ICODeRUS and @qtproject I added two more tests to make it even faster 🙂

Using a QMap<QString> instead of a QVariantMap reduces time by 10% and using QHash<QString> speeds it up to 400 mS (3 times faster as QVariantMap). see the details from GitHub project, here are the results: