Denis Mingulov's blog

Friday, February 24, 2012

I am trying to publish one more application for N9, original link to my blog as a support website is not enough.

So I have created this page for the support.

Drop Cache for N9 - an utility to flush disk cache to
prevent some data loss and to allow increase a free memory by dropping
out an outdated low level cache (non-destructive). Free. Open source, GPLv3.

If you have any question / problem or just a feedback - please write a comment here.

True random number generator. The most interesting feature for me. :)
I am the current maintainer of mssf-qt library package, which is a part of Meego 1.2 Harmattan Security Qt API (more info - Security domain overview). Password Generator uses this library, so briefly for the random data - a request is done, which is transferred to Trusted Execution Environment (TEE) Service - cryptographic and secure one, which generates the needed data. It is not a multiple requests, for example, 8 for 8 characters, but just 1 request.

Also I had an opportunity to become Nokia Qt Certified Specialist on 28 November 2011, I passed 2 exams - 010-002: Qt Essentials and 022-001: Core C++ for Qt developers.

So I can compare some aspects of Nokia Qt and Microsoft Professional Developer certifications.

M

N

Available at least since 1993

Since 2010

Score report contains exact received points + information of strength and weak topics, what should be improved

Currently - just Pass or Fail, by forum - might be improved in future

Notification from the company is received on the next day (as exam is provided at a special test service partner - might take some time)

The same

No paper certificate by default, PDF or XPS can be downloaded from a special MCP site, paper certificate can be ordered for an additional money

Certificate is delivered in 4-6 weeks

It is possible to check status anytime on the special site: mcp.microsoft.com, available almost immediately (see 2 rows above)

No any special site, it should be possible to add a special badge to account on developer.qt.nokia.com site by an additional request. For me - request is sent more than 1 week ago, still no any reaction/reply

A lot of different exams

Qt without Widgets seems is impossible, despite of QML is a modern way. I hope it will be improved later for Qt5

So briefly, Nokia Qt Certification is not mature enough, and I hope it will be significantly improved in future.

Thursday, October 27, 2011

Due to the Nokia switch to Windows Phone - I got an opportunity to become MCPD by my company.
I am still mostly working with Linux and Qt, but Nokia, Nokia... :)
Here is my notes about different book as preparation materials on my way to Microsoft Certified Professional Developer (MCPD).
(I decided to become certified Web Developer 4 on Visual Studion 2010 as I think the mobile devices' future is really tightly related to Web/Cloud)
Book images might be blocked by ad blocker, but this is the only way to get legal images which I know, so please take my apologies :)

There are no Microsoft Press books currently available.
So I decided to take a look to the book for previous version of the exam, 70-503 for .NET 3.5. It is really outdated and can be used for preparations for this exam just in case if you really have enough time, a lot of subjects are not covered at all.

Professional WCF 4: Windows Communication Foundation with .NET 4. Quite good book to recall some part related to the topic and improve knowledge, but does not cover everything what is needed for the exam. Otherwise the book itself can not be considered as something useful for specialists, it is mostly 'for beginners'.

Official book. Really useful and "must have" for such exam. Just Entity Framework is described quite briefly.

Pro Entity Framework 4.0 (Expert's Voice in .NET).Really nice book about Entity Framework. The author real is a professional and expert. Also "must have", as Entity Framework knowledge is really needed to pass the exam.

It was the most complex exam for me, despite of both "must have" books here. :)

70-519 Exam Ref.
I am still preparing to this exam, but seems it is a nice book which I would like to use to prepare.

I am not sure can I say anything exam content, but I just want to advice - use any available test simulations (at least that which are available on MCTS Self-Paced Training Kit disks), does not matter how strange they are.

Wednesday, October 13, 2010

I prefer to use latest Qt and latest Qt Creator - to use new features and also to see that nothing is broken in the current version. So I recompile Qt and Qt Creator from the scratch quite often - to be sure that there are no any bugs due to the incorrect compilation.

Briefly - do not compile a lot of modules which will not be needed for me, and also specify Qt namespace (to see any errors related to misuse this feature). I use Ubuntu 10.10, so I had just to create a copy of config.status - and copy it to any new shadow build folder.

I think that most interesting thing with such Qt config is that the Qt compilation requires almost the same time like the Qt Creator compilation. About 22-23 minutes on my home laptop and about 16-18 minutes - on the work laptop. (I compile Qt Creator like any usual Qt application: qmake -r; make -j 2)

What might be the reason? Even reduced Qt has to be much more than Qt Creator.

I decided that the answer is a usage of precompiled headers in Qt sources by default and lack of usage this feature in Qt Creator.

What are precompiled headers?

Quote from Qt documentation:
"Precompiled headers are a performance feature supported by some compilers to compile a stable body of code, and store the compiled state of the code in a binary file. During subsequent compilations, the compiler will load the stored state, and continue compiling the specified file. Each subsequent compilation is faster because the stable code does not need to be recompiled."

Also by the documentation - it should be very easy to add precompiled headers to any Qt project, almost everything will be done by the framework:

Create a special header file which has to contain stable and static code

Define the PRECOMPILED_HEADER variable in the project file

And that's all.

There is just some note, that "all platforms that support precompiled headers have the configuration option precompile_header set". It looks like a bug in the documentation, I became confused by this sentence, at least in the current Qt master branch grep -r precompile_header qt/mkspecs shows that precompile_header is predefined only for WinCE and Win32.

Ok, anyway it can be enabled manually by:

CONFIG *= precompile_header

.

Which exactly precompiled header has to be used for Qt Creator? Lets check what is used for Qt.

Suddenly for me - just a small amount of headers are included. I think that almost the same might be used for Qt Creator.
Lets just add there, for example, QPointer and QScopedPointer headers. Small shell command to see the most popular includes (has to be improved, of course):

So general Qt headers will be used in any libraries (by qtcreatorlibrary.pri), and extended with Gui part - in any plugin (by qtcreatorplugin.pri). Also it is possible to use any other precompiled headers, like it is done for Botan (it is pure C++ library, so even general Qt header are almost useless).

As it is already said, precompiled headers are not enabled by default for linux gcc, so it has to be done manually like here:

With precompiled headers: ~20 minutes (~2 GB)
Without precompiled headers: ~70 minutes (~1 GB)
So the compilation time is reduced to ~30%. Amazing.
And I recalled why I switched to use Linux. :)

About a required additional hard disk space for precompiled headers. Again, Qt Creator consist of a lot of subprojects, so any of this projects - will create its own precompiled header, even if it is the same qtcreator_pch.h. It requires time and disk space. I hope later Qt will be able to handle such situation also (if supported by compiler).

Ok. I hope I will be able to check Mac OS X results also, it is interesting - is it worse or better than Windows. :)

So, the conclusion: a lot of space for contributions to Qt Creator in different areas. E.g. I think a plugin-helper to create precompiled headers might be useful (at least count the used headers - or more useful, up to the special wizard).

Update 25.10:
It looks like precompiled headers sometime are not the good choice, especially on 'cool compiler PCs'. E.g. Linux PC which might compile Qt Creator by make -j 10 in 12 minutes - with enabled compiled header might became totally disappointing, compilation time might grow to 30-40 minutes. It looks like the reason is cache - requirements to the available free memory are much more stricter now.

I think the solution might be something like 'global' precompiled headers (for most Qt Compiler subprojects - precompiled headers are almost the same, so it really might be compiled just once). Problem - different defines for such precompiled headers, has to be investigated how exactly it has to be done correctly.

Sunday, September 19, 2010

I use Qt Creator daily (to be more specific - one of the latest builds).
Every week it becomes better, bug fixing and feature improvements are ongoing.
Qt Creator (and Qt) is Open Source and almost anybody can contribute it. It is available on Gitorious.

What is the most annoying in Qt Creator now? For me it is the time I have to wait before Qt Creator will be loaded and ready to work. On the clean PC with just cached previously started application - it takes about 800 ms only, but otherwise - it can take 5-10 seconds, and some time - up to 30 seconds (I use Ubuntu 10.04 x64 with T7500 CPU and 3 GB RAM). Can it be somehow improved? I decided to check this issue.

Qt Creator consist of different plugins (a lot, the list + description is available in Help - About plugins). Source for the plugin library is here: src/libs/extensionsystem. It is possible to profile plugin loading by using an option '-profile'. After loading - every plugin is initialized by initialize (initializePlugin in the report) and extensionInitialized (initializeExtensions) calls. Difference is the order of dependent plugins' execution, it is easy to see by the '-profile' output.

So what is the real result for Qt Creator loading? What is the slowest plugin?TOP5 plugins by loading time

Welcome - suddenly it can take up to 20-30% of full Qt Creator loading time.

Core - this is really a core for Qt Creator, so it is hard to optimize it.

Debugger - also a lot of features / sub-plugins.

Find - quite interesting why it is here

Qt4ProjectManager

Why it is TOP5? It had to be TOP3, but there is a Core plugin, and it is very complicate. :)

Find plugin - short investigation shows that it is due to X11 specific problem, QApplication::clipboard() has to initialize X11 desktop also. So it is X11 problem only, can be investigated more later.

Ok, lets try to check Welcome plugin. Most time is consumed by WelcomePlugin::initialize. Sources can be find here: src/plugins/welcome. What is going on there? Lets profile it more carefully.
Most work is done here:

m_welcomeMode = new WelcomeMode;

So what is the WelcomeMode::WelcomeMode()?

Additional profiling - and the next 2 lines require most time for the execution:

2nd line is just an usual UI setting up (from Designer), it is executed quite long time but it is hard to investigate why (is it related to Qt - to styleSheet usage?..)
So lets try to check 1st line - what is exactly done there and why it is so slow? ImageWidget::ImageWidget() is almost empty, so the longest time is required for QPixmap creation. ImageWidget does not use this QPixmap directly anyway, and uses it for the scaling source only.

So it is quite a big image (~2 mb unpacked). What might be the problem with the performance? Try to check Qt documentation: "QImage is designed and optimized for I/O, and for direct pixel access and manipulation, while QPixmap is designed and optimized for showing images on screen." As mentioned already, ImageViewer uses the source image (m_bg) for the scaling source only, how the scaling is implemented? Check Qt sources, QPixmap for the scaling uses QPixmapData::transformed. It is implemented so:

QPixmap::fromImage(toImage().transformed(matrix, mode));

So lets try to change QPixmap as a source for the background in ImageWidget to QImage (it will be faster even for the scaling). It will be so:

And ImageWidget has to be changed accordingly.
Patch is available here.

Ok, lets test will it be really faster or not.

Welcome plugin does not depend to any other plugin except Core, so to do the testing more real - Qt Creator will be compiled in the release mode for the next Qt 4.7 version: 4.7.0~rc1+git20100917-0ubuntu1.How the testing will be done?

Start old Qt Creator, wait until it will be ready, kill it, wait 5 seconds

Start new Qt Creator, wait until it will be ready, kill it, wait 5 seconds

Go to 1

Test results (time in ms):

modified

original

initialize

extInit

total load

initialize

extInit

total load

1

94

68

927

377

50

1155

2

94

68

1006

386

69

1137

3

76

50

845

310

69

1137

4

70

47

842

364

41

1222

5

70

49

929

450

50

1230

6

71

48

868

365

69

1292

7

94

64

997

146

71

1053

8

69

39

1141

410

63

1318

9

71

45

873

359

65

1276

10

75

65

843

387

49

1221

11

72

49

911

418

86

1192

12

117

58

916

340

69

1249

13

84

72

996

392

100

1294

14

76

74

849

209

69

1362

15

78

74

897

367

44

1146

average

81

58

923

352

64

1219

both

139

both

416

Together: 139 ms (81 + 58) for modified Qt Creator vs 416 (352+64) for original one. Reduced to about 33%, or +200% to the loading speed for particular Welcome plugin. Full load time - about 300 ms faster or +33%. 900 ms vs 1200 ms - it is palpably faster.

Is it really so? Yes and no.

Yes, at least for X11 Welcome plugin became faster per 200% (on the particular PC). And for any platform - in ImageWidget::paintEvent due to the scaling.

No, the test is synthetic. There is enough available memory, and a lot of files might be cached, so even 2 different Qt Creators start faster than usual. E.g. here Qt Creator starts in 25 seconds: QTCREATORBUG-2396. 250 ms (or even 5 seconds) faster for 25 seconds - it is almost invisible.

A test on the much faster PC (the same Ubuntu 10.04, T9600 @ 2.80GHz + 4 GB RAM, Video: Quadro FX 770M) does not show any speed up at all. So this is even a more specific patch, related to something else (some resource?...)

For the created patch - a merge request is created. Lets hope it will be accepted. :)

WelcomePlugin::extensionsInitialized (and other plugins) will be investigated and considered later.

It just has to be done, and of course to be done correctly. And there are a lot of other, more important tasks for Qt Creator team. What is one of the main difference between official Qt Creator team and any independent developer? Priorities. An independent developer can spend a lot of hours for some minor suggestion due to any reason, for example, because it is interesting.

Except improvements for the current solution, any other possibilities might be considered, e.g. boost showing the main window with base plugins only - and continue the loading in the background (e.g. Settings widgets) etc.

Ok, what is the main reason for this article? To show that anybody can help Qt Creator to become better. Also I am looking for a new challenge, and I decided to try to speed up Qt Creator loading. Do you want to participate?..