Blößmrt

History

When I looked at the state of graphical IMAP e-mail clients several years ago,
I was not really impressed. KMail from then-current KDE3 did not do a proper
job for me (numerous IMAP bugs like its inability to work as about every other
IMAP client when deleting messages, bug 26986 -- there were
more issues than that, but years have left my memories washed out a bit),
Thunderbird would crash for me every once a week, at least, and I just
happened to like KDE applications more than Gnome stuff, so I did not spend
much time looking at Evolution. Many MUAs looked like a classic generic
e-mail clients designed with POP3 in mind with IMAP added late in the
development cycle, while others supported wide range of IMAP features, yet
lacked in the GUI part of the problem. In short, using none of these
applications made me feel happy.

A programmer not feeling happy is a receipt for disaster. I was about to
finish my high school, so I had plenty of time at hand. I was experimenting
with Python, so that seemed like a natural implementation language, too. In
the end, I started a project called trojita whose remnants could
still be seen in an abandoned SVN
repo.

Coding in Python was fun. I tried several different approaches to the design
of my pet program, I was playing with technologies I had no experience with, I
even showed my "IMAP library" at my final exam as an example of a project I
made. It did not have much functionality, in fact, only the IMAP parser had
been completed, but it was an educative experience nonetheless and I passed the
exam.

After some time, however, I discovered Qt and C++ and felt in love. I
joyfully returned to the realm of statically-typed languages and suddenly felt
a lot better. I began porting my Python library to Qt/C++. It was not really a
port, rather a first complete rewrite of my project. Anyway, it did not take
long and the C++ version suddenly offered more functions than the old Python
branch, with unit tests as a nice added bonus.

Qt's Interview architecture, the Model/View classes, seemed like a decent
implementation of the MVC patter I was poking around to use. Several months
have passed, and suddenly trojita was able to show a tree of
mailboxes stored on a remote IMAP server, listing messages contained therein
and showing message bodies. I choose to finish the program as a part of my
bachelor's thesis, and ultimately, I succeeded.

The Code

So, in a few blogposts starting with this one I'm going to introduce a new Qt
IMAP e-mail client to the world. I hope I will get some attention and folks
looking at the code and trying to run the application. I'd love to get some
feedback on program design, code quality and general usability as well.

The code is hosted at Gitorious, and
a bachelor thesis about
Trojitá (PDF) (mirror)
which explains its design and compares it against several alternatives is
available, too. Perhaps the most interesting part is Chapter 3 which describes
the architecture of the application, and Chapter 4 in which I compare Trojitá to
several other MUAs on the market. All information about Trojitá are also
aggregated on Trojitá's homepage (any
web designer listening? :) ). Here is the obligatory screenshot:

Trojitá's Features

Some highlights of Trojitá are:

A pure Qt4 application with no additional dependencies

Robust IMAP core implemented using Qt's Model-View framework

Standards compliance is a design goal

Support for bandwidth-saving mode aimed at mobile users with expensive
connection

IMAP over SSH -- instead of going over an SSL socket, the server could be
accessed via SSH

The thesis was completed several months ago. Since that time, I've removed
the dependency on std::tr1::shared_ptr and switched to Qt's
QSharedPointer which in turn requires Qt-4.5 or newer. There
wasn't much more changes since then, as I enjoyed quite a long vacation, but I
guess I can tell the development is getting faster again.

This weekend I had the pleasure of travelling to a beautiful city of Schlitz.
After some six hours on train, I arrived to KWest GmbH, a cool company which
specializes in manufacturing of embedded systems, and which is nowadays busy
developing an Internet tablet for a German ISP. It turned out that the company
likes Qt and free software and is not particularly happy with the availability
of Qt-based IMAP clients. As it happens, I was not happy with the world of IMAP
clients, either, and I chose that as a subject for my bachelor's thesis a few
years ago.

After having met KWest's representatives, we came to conclusion that Trojita is indeed a suitable IMAP e-mail
client for them, and that while they do have a skilled development team, it
would be beneficial for both me and them to work together. We have agreed
that they will focus on bringing the GUI of the application up to speed with
modern standards, adding important features like address book and a decent
platform integration to the mix, while I will focus on what I'm good at, that
is, improving the IMAP support, especially the offline mode.

The good thing for the community is that Trojita will remain under the GPL and
that I'll be able to spend much more time on its development. Given enough
time, Trojita will mature, and if everything works well, we will see a nice Qt
e-mail application pretty soon. As usually, both the bug tracker and
all source code remains freely
available without any restrictions.

I'd like to thank KWest GmbH for giving me an opportunity to work on a free
software project that I'm interested in, especially to Sebastian for inviting me
and to Markus for being a nice guy to work with. And if you have some spare
time and are looking for a nice little city to visit, be sure to stop in Fulda
and Schlitz, their city centers look as if they came straight from a fairytale.

I've spent more than 10 hours in total changing the build system of Trojitá from CMake to QMake upon a request
from KWest. The process was very painful for me, so I think it's
worthwhile to include some comments about what were the major obstacles.

The first step, "getting the beast to build", was actually pretty easy -- I
(KWest actually) simply created a list of all files in the tree, added them
into a single .pro file, removed the #includes for
MOC, commented out the unit tests and rebuilt the project. That was the easy
part; qmake simply built bunch of .o files and linked them
together.

Problem with unit tests is that they all define the main()
function. Therefore, as far as I know, one has to create a separate
.pro file for each unit test, use the template =
subdirs and put each test into a subdirectory. That's slightly annoying
when compared to CTest (see the Trojita's git repository for how the
CMakeLists.txt looked before we switched), but doable and
actually pretty straightforward. Now, a much bigger problem is persuading
QMake to create static libraries and using them properly and in a
cross-platform way. I care about the Unix platform, some users want to play
with Trojita on Windows, and there's that secret device KWest is building.

Trojita currently consists of several parts; we have the core IMAP stuff
(which itself consists of three or four components), the GUI layer, some
third-party modules even (like Witold Wysota's qwwsmtpclient,
the Qt's iconloader, a unifying layer for QProcess
and QSslSocket etc). The unit tests for the IMAP protocol
clearly should not care about icons at all and shall ignore GUI classes, too,
but they do need to link against the IMAP protocol implementation. A
reasonable way to express that in the build system is to create a static
library for the IMAP stuff and link it to the rest of the GUI when
building the application and to each of the unit tests when building tests.
That's where problems with QMake start to hurt.

Unlike CMake, under which the static libraries are extremely easy to
write, QMake's support and documentation for static libraries leaves much to
be desired. A reasonable request is, for example, expecting the build system
to isolate me from stuff like library placement -- I just want to tell it that
I need to link against bunch of .arelease and
debug when caring about Windows builds.

Another expectation is that QMake should relink each binary if any static
library it depends on gets rebuilt. Too bad, using QMake, you have to include
the library name in three places for this to happen: at first, you have to use
LIBS += -Lpath/to/your/lib/directory, then you got to actually
link to it by LIBS += -lnameOfTheLibrary, and finally you have to
take care of the rebuilding by PRE_TARGETDEPS +=
full/path/and/the/libIdentifier.a. Oh, and please do not forget about
CONFIG += ordered, or the PRE_TARGETDEPS won't affect much stuff
anyway. Yuck!

There's the CONFIG += create_prl and CONFIG +=
link_prl, but they did not help me. I guess they are used for
specifying dynamic libraries on which the currently processed
static library depends. They certainly did not fix my problems when I
played with them, though.

That's what I call ease of use. Anyway, the QMake change just had to be done
(when the customer asked if I could migrate to QMake, I said I had no
preference) and it's been done now. This blog post is just a rant, and
hopefully might eventually make its way to Google's results for "qmake static
library".

There's always the possibility that I'm just too dumb to miss a completely
obvious way to work with static libraries in QMake. If that's indeed the case,
I sincerely apologise to QMake designers. Also, I offer one beer as a sign of
appreciation to the first person who shows me that I'm indeed missing
something from the big picture. In the meanwhile, have fun -- QMake has a lot
of nice features and I'm no longer afraid to use it, now that the static
libraries are used properly. The make test is still missing, but
I guess I can live without that for a while.

Today, on the 3rd day of the month, 333 days since the initial announcement, it's
my pleasure to announce the immediate availability of a new release of
Trojitá, a Qt IMAP e-mail client.
This is a technical preview release not designed for production use, but
rather a demonstration about what's been done in the past months. It should be
safe to point it to your production IMAP server (it has never eaten any mail
for me), but one shouldn't send real e-mails or otherwise rely on it apart
from testing.

New features in this release include:

Much improved offline support with proper on-disk caching

Tons of bugfixes (and I really mean it -- there have been plenty of them)

Improved interoperability with various IMAP servers, including Microsoft
Exchange

Parallel connections to the IMAP server for increased performance when
dealing with multiple mailboxes

It's my pleasure to announce that version v0.2.9 of Trojitá, a fast
and lightweight IMAP e-mail client based on the Qt library, is now
available. For more details of what Trojitá is and who should use it,
please see below.

Most of the changes in this version happened under the hood, but there
is a bunch of user-facing improvements -- the most prominent one is
probably the support for display of message threading (ie. "conversations").

Sadly, the Maemo version which first appeared as a beta during fall of
2010 could not be merged into the main repository before this release,
but you should stay tuned for future updates.

xTuple integration

Trojitá is going to be featured as an addon for the xTuple business management software. Using Trojitá will enable xTuple customers to
integrate e-mail messaging into their ERP systems. Please note that
xTuple users should use packages provided by their vendor and not this
source version.

What is Trojitá

Trojitá is a very fast and lightweight IMAP e-mail client written using
the Qt library. While it is not intended for use as a regular e-mail
client just yet, it could be interesting to play with it to see if the
speed looks compelling to you.

What could make Trojitá interesting for you:

It's a pure Qt4 application with no additional dependencies; it builds
in two-and-half minutes on a five-year-old laptop

Robust IMAP core implemented using Qt's Model-View framework

Standards compliance is a design goal

On-demand message list and body part loading ("lazy loading")

Offline IMAP support

Support for bandwidth-saving mode aimed at mobile users with expensive
connection

IMAP over SSH -- instead of going over an SSL socket, the server could
be accessed via SSH

Safe and robust dealing with HTML mail

Please note that the "message sending" and "message composing" features
of Trojitá are a bit lagging at this point and therefore it is not
recommended to use Trojitá for these tasks apart from testing.

Certain features of Trojitá depend on the IMAP server's functionality.
Trojitá is written from the bottom-up as an IMAP client and is designed
around its feature set. For example, threaded message viewing (ie.
"conversations") is supported only if the server implements an
appropriate extension for now.

Many of the optional IMAP features are not utilized at this point; there
is no support for searching, for example. We have tickets opened for
these, so please Cc yourself at the task tracker if you would like
to follow the progress here.

Finally, as with any software, Trojitá has some bugs which are already
known and reported in the issue tracker and some which are still waiting
for discovery. That said, it should be safe to use to for *reading*
mail. I've been doing that for several years on a production account,
and I have never lost a mail with Trojitá. Please do not send e-mails
with current version, though, as it is known to produce non-standard
messages in certain circumstances.

Trojitá is known to work on Linux, Mac OS X and Windows. It should also
run on all platforms supported by Qt. Optimizations for mobile devices
and the Maemo/MeeGo are in the works.

Reporting bugs and wishes

Please use the Redmine portal to report issues with Trojitá. If you
do not want to be bothered by a registration, please at least send bug
reports via e-mail or report them at the #trojita IRC channel on Freenode.

Community

Trojita could always use more people in the community. Areas in which
people are needed most, as well as general guidelines about how we
prefer to work are documented at Contributing to Trojitá.

The IRC channel is #trojita on irc.freenode.net, the mailing list is
trojita@lists.flaska.net (moderation required for non-subscribers,
archived at various places etc).

Acknowledgment

I'd like to thank the KWest GmbH. and OpenMFG LLC, dba xTuple for their
support in delivering the exciting new features of Trojitá. Another
acknowledgment goes to the helpful folks at the #qt IRC channel and the
qt-interest mailing list and also to everybody who reported an issue
with Trojita or tested it against another IMAP server.

I was thinking that it might be a good idea to share my view on how Trojitá is going on, and what my plans
for its future are. For a quick overview of Trojitá, a fast and lightweight IMAP client,
please look at the project homepage or
see the FAQ.

So, with version 0.2.9.3 out
(what a nice number, isn't it?), it's time to take a look at what needs to be
done before people can actually use Trojita on a daily basis. For my use
case, the following is really lacking:

Quick search through
messages in the current mailbox. I often need to find a particular message, and
simply typing a part of its subject, or a sender or recipient, is what I use
most often. A full-blown multi-mailbox search is probably not needed at this
point, but a basic, quick input box for searching would be very much
welcome.

I'd like to let you know that I've succeeded in building Trojitá for
MeeGo 1.2 Harmattan, as available on the Nokia N950. It is completely
unusable for now due to the lack of a real mouse, but shows pretty nice
that it *can* work :). A screenshot:

So, now that I know that the port actually builds and that the device
can run my code, I'll plunge into the QML world and create a proper
touch-friendly GUI. I'll keep you posted about my progress.

I'd like to say a huge thank you to Nokia and its Developer Launchpad
team, especially those that are responsible for the Community Device
program. Nokia was kind enough to offer a loan of the N950 at no charge
for the purposes of Trojitá development, and I really appreciate it.

It's my pleasure to announce that version v0.3 of Trojitá, a fast and
lightweight IMAP e-mail client based on the Qt library, is now available. For more details of what Trojitá is and who
should use it, please see below.

Changes since version v0.2.9.4

This release is a major one, bringing new features, plenty of bugfixes and two new contributors to the table. A quick
summary of what has happened:

Drastic reduction in memory usage

Tagging e-mails (contributed by Shanti Bouchez)

Faster fetching of data

Much more efficient support for threading

More robust IMAP support

A new Debug menu

Autocompletion of e-mail addresses (contributed by Thomas Gahr)

Reporting server/client configuration through ID

Support of multipart/related for full rfc2387 compliance

Showing Trojita's homepage on start

GUI fixes (now finally works in dark themes)

SMTP works over SSL (contributed by Shanti Bouchez)

Expanded unit test coverage

Plenty of bugfixes and further improvements

What is Trojitá

Trojitá is a very fast and lightweight IMAP e-mail client written using the Qt library.

What could make Trojitá interesting for you

It's a pure Qt4 application with no additional dependencies; it builds in two-and-half minutes on a five-year-old laptop

Robust IMAP core implemented using Qt's Model-View framework

Standards compliance is a design goal

On-demand message list and body part loading ("lazy loading")

Offline IMAP support

Support for bandwidth-saving mode aimed at mobile users with expensive connection

IMAP over SSH -- instead of going over an SSL socket, the server could be accessed via SSH

Safe and robust dealing with HTML mail

Please note that the "message sending" and "message composing" features of Trojitá are a bit lagging at this point
and therefore it is not recommended to use Trojitá for these tasks apart from testing.

Certain features of Trojitá depend on the IMAP server's functionality. Trojitá is written from the bottom-up as an
IMAP client and is designed around its feature set. For example, threaded message viewing (ie. "conversations") is
supported only if the server implements an appropriate extension for now.

Trojitá is under heavy development, features are added on almost daily basis and the codebase is rapidly maturing.
Certain useful features are still missing, there is no support for searching, for example. We have tickets opened for
these, so please Cc yourself at the task tracker if you
would like to follow the progress here.

Finally, as with any software, Trojitá has some bugs which are already known and reported in the issue tracker and
some which are still waiting for discovery. That said, it is safe to use to for *reading* mail. I've been doing that for
several years on a production account, and I have never lost a mail with Trojitá. Please do not send e-mails with
current version, though, as it is known to produce non-standard messages in certain circumstances.

Where I can get it

Our web has all the required information, but if you are
impatient and just want to grab the tarball for v0.3, download from Sourceforge:

Trojitá is known to work on Linux, Mac OS X and Windows. It should also run on all platforms supported by Qt. A MeeGo
version is still pending, though.

Reporting bugs and wishes

Please use the Redmine portal to report issues with
Trojitá. If you do not want to be bothered by a registration, please at least send bug reports via e-mail or report them
at the #trojita IRC channel on Freenode.

Community

Trojita could always use more people in the community. Areas in which people are needed most, as well as general
guidelines about how we prefer to work are documented at the wiki.

Acknowledgment

A huge thank you goes to two new contributors who have submitted patches to make Trojitá better. It's my please to
introduce Shanti Bouchez who is responsible for the new feature of tagging messages (and fixed STARTTLS for SMTP
subscription in the process, among other things). The second contributor is Thomas Gahr who added e-mail auto-completion
and fixed bugs.

Since its inception in 2006, many other people have contributed to Trojitá as well. I'd like to mention patches from
Benson Tsai, John Rogelstad, Andrew Brouwers, Gil Moskowitz, Jiří Helebrant, Jun Yang, Justin J, and Tomáš Kouba, who
have all sent patches in. Finally, another huge thank you goes to anyone who has reported bugs or helped make Trojitá
better in any way.

I'm sitting on the first day of the Qt Developer Days in Berlin and am pretty
impressed about the event so far -- the organizers have done an excellent job
and everything feels very, very smooth here. Congratulations for that; I have a
first-hand experience with organizing a workshop and can imagine the huge pile of
work which these people have invested into making it rock. Well done I say.

It's been some time since I blogged about Trojitá, a fast and lightweight IMAP
e-mail client. A lot of work has found the way in since the last release; Trojitá now supports
almost all of the useful IMAP extensions including QRESYNC and
CONDSTORE for blazingly fast mailbox synchronization or the
CONTEXT=SEARCH for live-updated search results to name just a few.
There've also been roughly 666 tons of bugfixes, optimizations, new features and
tweaks. Trojitá is finally showing evidence of getting ready for being usable as
a regular e-mail client, and it's exciting to see that process after 6+ years of
working on that in my spare time. People are taking part in the development
process; there has been a series of commits from Thomas Lübking of the kwin fame
dealing with tricky QWidget issues, for example -- and it's great to see many
usability glitches getting addressed.

The last nine months were rather hectic for me -- I got my Master's degree
(the thesis was about
Trojitá, of course), I started a new job (this time using Qt) and
implemented quite some interesting stuff with Qt -- if you have always wondered
how to integrate Ragel, a parser generator, with qmake, stay tuned for future
posts.

Anyway, in case you are interested in using an extremely fast e-mail client
implemented in pure Qt, give Trojitá a try. If you'd like to chat about it,
feel free to drop me a mail or just stop me
anywhere. We're always looking for contributors, so if you hit some annoying
behavior, please do chime in and start hacking.

I'm happy to announce that Trojitá, a
fast IMAP e-mail client, has become part of the KDE project. You can find it below
extragear/pim/trojita.

Why moving under the KDE umbrella?

After reading the KDE's manifesto, it
became obvious that the KDE project's values align quite well with what we
want to
achieve in Trojitá. Becoming part of a bigger community is a logical next
step -- it will surely make Trojitá more visible, and the KDE community will get
a competing e-mail client for those who might not be happy with the more
established offerings. Competition is good, people say.

But I don't want to install KDE!

You don't have to. Trojitá will remain usable without KDE; you won't need it
for running Trojitá, nor for compiling the application. We don't use any
KDE-specific classes, so we do not link to kdelibs at all. In
future, I hope we will be able to offer an optional feature to integrate
with KDE more closely, but there are no plans to make Trojitá require
the KDE libraries.

How is it going?

Extremely well! Five new people have already contributed code to Trojitá, and
the localization team behind KDE got a terrific job with providing translation
into eleven languages (and I had endless hours of fun hacking together
lconvert-based setup to make sure that Trojitá's Qt-based
translations work well with KDE's gettext-based workflow -- oh boy
was that fun!). Trojitá also takes part in the Google Code-in project; Mohammed Nafees has already added a
feature
for multiple sender identities. I also had a great chat with the KDE PIM
maintainers about sharing of our code in future.

What's next?

A lot of work is still in front of us -- from boring housekeeping like
moving to KDE's Bugzilla for issue tracking to adding exciting (and
complicated!) new features like support for multiple accounts. But the important
part is that Trojitá is live and progressing swiftly -- features are being
added, bugs are getting fixed on a faily basis and other people besides me are
actually using the application on a daaily basis. According to Ohloh's statistics, we have a well
established, mature codebase maintained by a large development team with
increasing year-over-year commits.

Interested?

If you are interested in helping out, check out the instructions
and just start hacking!

There's a lot of people who are very careful to never delete a single line from an e-mail they are replying to,
always quoting the complete history. There's also a lot of people who believe that it wastes time to eyeball such long,
useless texts. One of the fancy features introduced in this release of Trojitá,
a fast Qt IMAP e-mail client, is automatic quote collapsing. I won't show you an example of an annoying mail for obvious
reasons :), but this feature is useful even for e-mails which employ reasonable quoting strategy. It looks like this in
the action:

When you click on the ... symbols, the first level expands to reveal the following:

When everything is expanded, the end results looks like this:

This concept is extremely effective especially when communicating with a top-posting community.

We had quite some internal discussion about how to implement this feature. For those not familiar with Trojitá's
architecture, we use a properly restricted QtWebKit instance for e-mail rendering. The restrictions which are active
include click-wrapped loading of remote content for privacy (so that a spammer cannot know whether you have read their
message), no plugins, no HTML5 local storage, and also no JavaScript. With JavaScript, it would be easy to do
nice, click-controlled interactive collapsing of nested citations. However, enabling JavaScript might have quite some
security implications (or maybe "only" keeping your CPU busy and draining your battery by a malicious third party). We
could have enabled JavaScript for plaintext contents only, but that would not be as elegant as the solution we
chose in the end.

Starting with Qt 4.8, WebKit ships with support for the :checked CSS3 pseudoclass. Using this feature,
it's possible to change the style based on whether an HTML checkbox is
checked or not . In theory, that's everything one might possibly need, but there's a small catch
-- the usual way of showing/hiding contents based on a state of a checkbox hits a WebKit bug (quick summary: it's tough to have it working without the
~ adjacent-sibling selector unless you use it in one particular way). Long story short, I now know more
about CSS3 than I thought I would ever want to know, and it works (unless you're on Qt5 already where
it assert-fails and crashes the WebKit).

Speaking of WebKit, the way we use it in Trojitá is a bit unusual. The QWebView class contains full
support for scrolling, so it is not necessary to put it inside a QScrollArea. However, when working with
e-mails, one has to account for messages containing multiple body parts which have to be shown separately (again, for
both practical and security reasons). In addition, the e-mail header which is typically implemented as a custom
QWidget for flexibility, is usually intended to combine with the message bodies into a single entity to be
scrolled together. With WebKit, this is doable (after some size hints magic, and I really mean magic -- thanks
to Thomas Lübking of the KWin fame for patches), but there's a catch -- internal methods like the findText
which normally scroll the contents of the web page into the matching place no longer works when the whole web view is
embedded into a QScrollArea. I've dived into the source code of WebKit and the interesting thing is that there
is code for exactly this case, but it is only implemented in Apple's version of WebKit. The source code even says that Apple needed this for its own
Mail.app -- an interesting coincidence, I guess.

Compared with the last release, Trojitá has also gained support for "smart replying". It will now detect that a
message comes from a mailing list and Ctrl+R will by default reply to list. Thomas has added support for
saving drafts, so that you are not supposed to lose your work when you accidentally kill Trojitá anymore. There's also
been the traditional round of bug fixes and compatibility improvements. It is entertaining to see that Trojitá is
apparently triggering certain code paths in various IMAP server implementations, proprietary and free software alike,
for the first time.

The work on support for multiple IMAP accounts is getting closer to being ready for prime time. It isn't present in
the current release, though -- the GUI integration in particular needs some polishing before it hits the masses.

I'm happy to observe that Trojitá is getting features which are missing from other popular e-mail clients. I'm
especially fond of my pet contribution, the quote collapsing. Does your favorite e-mail application offer a similar
feature?

In the coming weeks, I'd like to focus on getting the multiaccounts branch merged into master, adding better
integration with the address book (Trojitá can already offer tab completion with data coming from Mutt's abook) and general GUI improvements. It would also be great to make it possible
to let Trojitá act as a handler for the mailto: URLs so that it gets invoked when you click on an e-mail
address in your favorite web browser, for example.

And finally, to maybe lure a reader or two into trying Trojitá, here's a short quote from a happy user who came to
our IRC channel a few days ago:

17:16 < Sir_Herrbatka> i had no idea that it's possible for mail client to be THAT fast

One cannot help but be happy when reading this. Thanks!

If you're on Linux, you can get the latest version of Trojitá from the OBS or the usual place.

Jos wrote a blog
post yesterday commenting on the complexity of the PIM problem. He raises an
interesting concern about whether we would be all better if there was no Trojitá and I just improved KMail instead.
As usual, the matter is more complicated than it might seem on a first sight.

Executive Summary: I tried working with KDEPIM. The KDEPIM IMAP stack
required a total rewrite in order to be useful. At the time I started, Akonadi
did not exist. The rewrite has been done, and Trojitá is the result. It is up
to the Akonadi developers to use Trojitá's IMAP implementation if they are
interested; it is modular enough.

People might wonder why Trojitá exists at all. I started working on it
because I wasn't happy with how the mail clients performed back in 2006. The
supported features were severely limited, the speed was horrible. After
studying the IMAP protocol, it became obvious that the reason for this slowness
is the rather stupid way in which the contemporary clients treated the remote
mail store. Yes, it's really a very dumb idea to load tens of thousands
of messages when opening a mailbox for the first time. Nope, it does not make
sense to block the GUI until you fetch that 15MB mail over a slow and capped
cell phone connection. Yes, you can do better with IMAP, and the possibility
has been there for years. The problem is that the clients were not
using the IMAP protocol in an efficient manner.

It is not easy to retrofit a decent IMAP support into an existing client.
There could be numerous code paths which just assume that everything happens
synchronously and block the GUI when the data are stuck on the wire for some
reason. Doing this properly, fetching just the required data and doing all
that in an asynchronous manner is not easy -- but it's doable nonetheless. It
requires huge changes to the overall architecture of the legacy applications,
however.

Give Trojitá a try now
and see how fast it is. I'm serious here -- Trojitá opens a mailbox with tens of
thousands of messages in a fraction of second. Try to open a big e-mail with
vacation pictures from your relatives over a slow link -- you will see the
important textual part pop up immediately with the images being loaded in the
background, not disturbing your work. Now try to do the same in your favorite
e-mail client -- if it's as fast as Trojitá, congratulations. If not, perhaps
you should switch.

Right now, the IMAP support in Trojitá is way more advanced than what is
shipped in Geary or KDE PIM -- and it is this solid foundation which leads to
Trojitá's performance. What needs work now is polishing the GUI and making it
play well with the rest of a users' system. I don't care whether this
polishing means improving Trojitá's GUI iteratively or whether its IMAP
support gets used as a library in, say, KMail -- both would be very succesfull
outcomes. It would be terrific to somehow combine the nice, polished UI of
the more established e-mail clients with the IMAP engine from Trojitá. There
is a GSoC proposal for integrating Trojitá into KDE's Kontact -- but for it to
succeed, people from other projects must get involved as well. I have put
seven years of my time into making the IMAP support rock; I would not be able
to achieve the same if I was improving KMail instead. I don't need a
fast KMail, I need a great e-mail client. Trojitá works well enough
for me.

Oh, and there's also a currently running fundraiser
for better address book integration in Trojitá. We are not asking for
$ 100k, we are asking for $ 199. Let's see how many people are willing
to put the money where their mouth is and actually do something to help
the PIM on a free desktop. Patches and donations are both equally welcome.
Actually, not really -- great patches are much more appreciated. Because Jos
is right -- it takes a lot of work to produce great software, and things get
better when there are more poeple working towards their common goal
together.

Update: it looks like my choice of kickstarter platform was rather
poor, catincan apparently doesn't accept PayPal :(. There's the possiblity of
direct donations over
SourceForge/PayPal -- please keep in mind that these will be charged even
if less donors pledge to the idea.

Hi all,
we are pleased to announce version 0.4 of Trojitá, a fast Qt IMAP e-mail
client. For this release, a lot of changes were made under the hood, but of course there are some changes that are
visible to the user as well.

Improvements:

Users are able to use multiple sessions, which means that it is possible to use Trojitá with multiple IMAP accounts
at the same time. It can be used by invoking Trojitá with the --profile something switch. For each profile,
a new instance of the application is started. Please note that this is not our final solution for the multi-accounts
problem; work on this is ongoing. For details, refer to the detailed
instructions.

In the Composer Window, users can now control whether the current message is a reply to some other message.
Hopefully, this will make it easier to reply to a ton of people while starting a new thread, not lumping the unrelated
conversations together.

Trojitá will now detect changes to the network connection state. So for example, when a user switches from a
wireless connection to a wired one, Trojitá will detect that and try to reconnect automatically.

Trojitá gained a setting to automatically use the system proxy settings.

SOCKS5 and HTTP proxies are supported.

Memory usage has been reduced and speed has been improved. Our benchmarks indicate being ten times faster when
syncing huge mailboxes, and using 38% less memory at the same time.

The Compose Window supports editing the "From" field with hand-picked addresses as per common user requests.

This release is dedicated to the people of all nations living in Ukraine. We are no fans of political messages in
software announcements, but we also cannot remain silent when unmarked Russian troops are marching over a free country.
The Trojitá project was founded in a republic formerly known as Czechoslovakia. We were "protected" by foreign
aggressors twice in the 20th century — first in 1938 by the Nazi Germany, and second time in 1968 by the occupation
forces of the USSR. Back in 1938, Adolf Hitler used the same rhetorics we hear today: that a national minority was
oppressed. In 1968, eight people who protested
against the occupation in Moscow were detained within a couple of minutes, convicted and sent to jail. In 2014,
Moscowians are protesting on a bigger scale, yet we all see the cops arresting them on Youtube — including those displaying blank
signs.

This is not about politics, this is about morality. What is happening today in Ukraine is a barbaric act, an
occupation of an innocent country which has done nothing but stopped being attracted to their more prominent eastern
neighbor. No matter what one thinks about the international politics and the Crimean independence, this is an act which
must be condemned and fiercely fought against. There isn't much what we could do, so we hope that at least this
symbolic act will let the Ukrainians know that the world's thoughts are with them in this dire moment. За вашу и нашу
свободу, indeed!

Finally, we would like to thank Jai Luthra, Danny Rim, Benjamin Kaiser and Yazeed Zoabi, our Google Code-In students,
and Stephan Platz, Karan Luthra, Tomasz Kalkosiński and Luigi Toscano, people who recently joined Trojitá, for their
code contributions.

Summary

An SSL stripping vulnerability was discovered in Trojitá, a fast Qt IMAP e-mail client.
User's credentials are never leaked, but if a user tries to send an
e-mail, the automatic saving into the "sent" or "draft" folders could happen
over a plaintext connection even if the user's preferences specify STARTTLS as
a requirement.

Background

The IMAP protocol defines the STARTTLS command which is used to
transparently upgrade a plaintext connection to an encrypted one using
SSL/TLS. The STARTTLS command can only be issued in an unauthenticated state
as per the IMAP's state machine.

RFC 3501 also allows for a possibility of the connection jumping
immediately into an authenticated state via the PREAUTH initial response.
However, as the STARTTLS command cannot be issued once in the authenticated
state, an attacker able to intercept and modify the network communication
might trick the client into a state where the connection cannot be encrypted
anymore.

Affected versions

All versions of Trojitá up to 0.4 are vulnerable. The fix is included in version 0.4.1.

Remedies

Connections which use the SSL/TLS form the very beginning (e.g. the
connections using port 993) are secure and not vulnerable.

Possible impact

The user's credentials will never be transmitted over a plaintext connection even in presence of this attack.

Because Trojitá proceeded to use the connection without STARTTLS in face of
PREAUTH, certain data might be leaked to the attacker. The only example which
we were able to identify is the full content of a message which the user
attempts to save to their "Sent" folder while trying to send a mail.

We don't believe that any other data could be leaked. Again, user's
credentials will not be leaked.

Acknowledgement

Thanks to Arnt Gulbrandsen on the imap-protocol ML for asking what happens
when we're configured to request STARTTLS and a PREAUTH is received, and to
Michael M Slusarz for starting that discussion.

One of the improvements which were mentioned in the recent announcement of Trojitá, a fast Qt e-mail client, were substantial memory savings and speed
improvements. In the rest of this post, I would like to explain what exactly we have done and how it matters. This is
going to be a technical post, so if you are not interested in C++ or software engineering, you might want to skip this
article.

Planting Trees

At the core of Trojitá's IMAP implementation is the TreeItem, an abstract class whose basic layout will
be familiar to anyone who has worked with a custom QAbstractItemModel reimplementation. In short, the
purpose of this class is to serve as a node in the tree of items which represent all the data stored on a remote IMAP
server.

The structure is tree-shaped because that's what fits both the QAbstractItemModel's and the IMAP way of
working. At the top, there's a list of mailboxes. Children of these mailboxes are either other, nested mailboxes, or
lists of messages. Below the lists of messages, one can find individual e-mails, and within these e-mails, individual
body parts as per the recursive nature of the MIME encapsulation. (This is what enables messages with pictures attached,
e-mail forwarding, and similar stuff. MIME is fun.) This tree of items is used by the QAbstractItemModel
for keeping track of what is where, and for issuing the QModelIndex instances which are used by the rest of
the application for accessing, requesting and manipulating the data.

When a QModelIndex is used and passed to the IMAP Model, what matters most is its
internalPointer(), a void * which, within Trojitá, always points to an instance of some
TreeItem subclass. Everything else, like the row() and column(), are actually not
important; the pointer itself is enough to determine everything about the index in question.

Each TreeItem has to store a couple of interesting properties. Besides the usual Qt-mandated stuff like
pointer to the parent item and a list of children, there are also application-specific items which enable the code to,
well, actually do useful things like printing e-mail subjects or downloading mail attachments. For a mailbox, this
crucial information might be the mailbox name. For a message, the UID of the message along with a pointer
to the mailbox is enough to uniquely identify everything which is needed.

Lazy Loading

Enter the lazy loading. Many people confirm that Trojitá is fast, and plenty of them are not afraid to say
that it is blazingly fast. This speed is enabled by the fact that Trojitá will only do the smallest amount of
work required to bring the data over the network (or from disk, for that matter). If you open a huge mailbox with half a
million messages, perhaps the GMail's "All messages" account, or one's LKML archive, Trojitá will not start
loading half a million of subjects. Instead, the in-memory TreeItem nodes are created in a special state
"no data has been requested yet". Trojitá still creates half a million items in memory, but these items are rather
lightweight and only contain the absolute minimum of data they need for proper operation.

Some of these "empty" nodes are, eventually, consulted and used for item display -- perhaps because a view is
attached to this model, and the view wants to show the recent mail to the user. In Qt, this usually happens via the
data() method of the QAbstractItemModel, but other methods like rowCount() have a
very similar effect. Whenever more data are needed, the state of the tree node changes from the initial "no data have
been requested" to "loading stuff", and an asynchronous request for these data is dispatched. An important part of the
tale is that the request is indeed completely asynchronous, so you won't see any blocking whatsoever in the GUI. The
QTreeView will show an animation while a subtree is expanded, the message viewer might display a spinner,
and the mail listing shows greyed-out "Loading..." placeholder instead of the usual message subjects.

After a short while, the data arrive and the tree node is updated with the extracted contents -- be it e-mail
subject, or perhaps the attached image of dancing pigs. As the requested data are now here, the status of the tree node
is updated from the previous "loading stuff" into "done". At the same time, an appropriate signal, like
dataChanged or rowsInserted, is emitted. Requesting the same data again via the classic MVC
API will not result in network requests, but everything will be accommodated from the local cache.

What we see now is that there is just a handful of item states, yet the typical layout of the TreeItem looks roughly
like this:

On a 64bit system, this translates to at least three 64bit words being used -- one for the painter to the parent
item, one (or much more) for storage of the list of children, and one more for storing the enum
FetchingStatus. That's a lot of space, given we have just created half a million of these items.

Tagged Pointers

An interesting property of a modern CPU is that the data structures must be aligned properly. A very common
rule is that e.g. a 32bit integer can only start at memory offset which is a multiple of four. In hex, this means that
an address, or a pointer value, could end with 0x0, or 0x4, or 0x8, or
0xc. The detailed rules are platform-specific and depend on the exact data structure which we are pointing
to, but the important message is that at least some of the low bits in the pointer address are always going to be zero.
Perhaps we could encode some information in there?

Turns out this is exactly what pointer tagging is about.
Instead of having two members, one TreeItem * and one FetchingStatus, these are squashed into
a single pointer-sized value. The CPU can no longer use the pointer value directly, all accesses have to go via an
inlined function which simply masks away the lowest bits which do bring a very minor performance hit, but the memory
conservation is real.

Using Memory Only When Needed

Back to our example of a mailbox with 500k messages. Surely a user is only going to see a small subset of them at
once, right?

That is indeed the case. We still have to at least reserve space for 500k items for technical reasons, but there is
certainly no need to reserve space for heavy stuff like subjects and other headers. Indeed, in Trojitá, we track the
From/To/Cc/Bcc headers, the subjects, various kinds of timestamps, other envelope items and similar stuff, and this
totals a couple hundred bytes per each message. A couple hundred bytes is not much (pun intended), but "a couple
hundred bytes" times "half a million" is a ton of memory.

This got implemented here. One
particular benchmark which tests how fast Trojitá resynchronizes a mailbox with 100k of messages showed immediate
reduction in memory usage from previous 45 MB to 25 MB. The change, again, does come with a cost; one now has
to follow one more pointer redirection, and one has to perform one more dynamic allocation for each message which is
actually visible. That, however, proves to be negligible during typical usage.

Measure, Don't Guess

As usual with optimizing, the real results might sometimes be surprising. A careful reader and an experienced Qt
programmer might have noticed the QList above and shuddered in horror. In fact, Trojitá now uses
QVector in its place, but when I was changing the code, using std::vector sounded like a
no-brainer. Who needs the copy-on-write semantics here anyway, so why should I pay its price in this context? These data
(list of children of an item) are not copied that often, and copying a contiguous list of pointers is pretty cheap
anyway (it surely is dwarfed by dynamic allocation overhead). So we should just stick with std::vector,
right?

Well, not really. It turned out that plenty of these lists are empty most of the time. If we are looking at
the list of messages in our huge mailbox, chances are that most of these messages were not loaded yet, and therefore the
list of children, i.e. something which represents their inner MIME structure, is likely empty. This is where the
QVector really shines. Instead of using three pointers per vector, like the GCC's std::vector
does, QVector is happy with a single pointer pointing to a shared null instance, something which is
empty.

Now, factor of three on an item which is used half a million times, this is something which is going to hurt.
That's why Trojitá eventually settled on
using QVector for the m_children member. The important lesson here is "don't assume,
measure".

Wrapping up

Thanks to these optimization (and a couple more, see the git log), one particular test case now runs ten times faster
while simultaneously using 38% less memory -- comparing the v0.4 with v0.3.96. Trojitá was pretty fast even before, but
now it really flies. The sources of memory diet were described in today's blog post; the explanation on how the time
was cut is something which will have to wait for another day.

Some of the recent releases of Trojitá, a fast Qt e-mail client, mentioned an ongoing work towards bringing the application to the Ubuntu Touch platform.
It turns out that this won't be happening.

The developers who were working on the Ubuntu Touch UI decided that they would prefer to end working with upstream and instead focus on a standalone long-term fork of Trojitá called Dekko.
The fork lives within the Launchpad ecosystem and we agreed that there's no point in keeping unmaintained and dead code in our repository anymore -- hence it's being removed.

It is that time of the year again, and people are applying for Google Summer of Code positions.
It's great to see a big crowd of newcomers.
This article explains what sort of students are welcome in GSoC from the point of view of Trojitá, a fast Qt IMAP e-mail client.
I suspect that many other projects within KDE share my views, but it's best to ask them.
Hopefully, this post will help students understand what we are looking for, and assist in deciding what project to work for.

Finding a motivation

As a mentor, my motivation in GSoC is pretty simple — I want to attract new contributors to the project I maintain.
This means that I value long-term sustainability above fancy features.
If you are going to apply with us, make sure that you actually want to stick around.
What happens when GSoC terminates?
What happens when GSoC terminates and the work you've been doing is not ready yet?
Do you see yourself continuing the work you've done so far?
Or is it going to become an abandonware, with some cash in your pocket being your only reward?
Who is going to maintain the code which you worked hard to create?

Selecting an area of work

This is probably the most important aspect of your GSoC involvement.
You're going to spend three months of full time activity on some project, a project you might have not heard about before.
Why are you doing this — is it only about the money, or do you already have a connection to the project you've selected?
Is the project trying to solve a problem that you find interesting?
Would you use the results of that project even without the GSoC?

My experience shows that it's best to find a project which fills a niche that you find interesting.
Do you have a digital camera, and do you think that a random photo editor's interface sucks?
Work on that, make the interface better.
Do you love listening to music?
Maybe your favorite music player has some annoying bug that you could fix.
Maybe you could add a feature to, say, synchronize the playlist with your cell phone (this is just an example, of course).
Do you like 3D printing?
Help improve an existing software for 3D printing, then.
Are you a database buff?
Is there something you find lacking in, e.g., PostgreSQL?

Either way, it is probably a good idea to select something which you need to use, or want to use for some reason.
It's of course fine to e.g. spend your GSoC term working on an astronomy tool even though you haven't used one before, but unless you really like astronomy, then you should probably choose something else.
In case of Trojitá, if you have been using GMail's web interface for the past five years and you think that it's the best thing since sliced bread, well, chances are that you won't enjoy working on a desktop e-mail client.

Pick something you like, something which you enjoy working with.

Making a proposal

An excellent idea is to make yourself known in advance.
This does not happen by joining the IRC channel and saying "I want to work on GSoC", or mailing us to let us know about this.
A much better way of getting involved is through showing your dedication.

Try to play with the application you are about to apply for.
Do you see some annoying bug?
Fix it!
Does it work well?
Use the application more; you will find bugs.
Look at the project's bug tracker, maybe there are some issues which people are hitting.
Do you think that you can fix it?
Diving into bug fixing is an excellent opportunity to get yourself familiar with the project's code base, and to make sure that our mentors know the style and pace of your work.

Now that you have some familiarity with the code, maybe you can already see opportunities for work besides what's already described on the GSoC ideas wiki page.
That's fine — the best proposals usually come from students who have found them on their own.
The list of ideas is just that, a list of ideas, not an exhaustive cookbook.
There's usually much more what can be done during the course of the GSoC.
What would be most interesting area for you?
How does it fit into the bigger picture?

After you've thought about the area to work on, now it's time to write your proposal.
Start early, and make sure that you talk about your ideas with your prospective mentors before you spend three hours preparing a detailed roadmap.
Define the goals that you want to achieve, and talk with your mentors about them.
Make sure that the work fits well with the length and style of the GSoC.

And finally, be sure that you stay open and honest with your mentoring team.
Remember, this is not a contest of writing a best project proposal.
For me, GSoC is all about finding people who are interested in working on, say, Trojitá.
What I'm looking for are honest, fair-behaving people who demonstrate willingness to learn new stuff.
On top of that, I like to accept people with whom I have already worked.
Hearing about you for the first time when I read your GSoC proposal is not a perfect way of introducing yourself.
Make yourself known in advance, and show us how you can help us make our project better.
Show us that you want to become a part of that "we".

Are you interested in cryptography, either as a user or as a developer?
Read on -- this blogpost talks about some of the UI choices we made, as well
as about the technical challenges of working with the existing crypto
libraries.

The next version of Trojitá, a
fast e-mail client, will support working with encrypted and signed messages.
Thanks to Stephan Platz for implementing this during the Google Summer of Code
project. If you are impatient, just install
the trojita-nightly package and check it out today.

Here's how a signed message looks like in a typical scenario:

Some other e-mail clients show a yellow semi-warning icon when showing a
message with an unknown or unrecognized key. In my opinion, that isn't a great
design choice. If I as an attacker wanted to get rid of the warning, I could
just as well sign a faked but unsigned e-mail message. This message is
signed by something, so we should probably not make this situation
appear less secure than as if the e-mail was not signed at all.

(Careful readers might start thinking about maintaining a peristant key
association database based on the observed traffic patterns. We are aware of
the upstream initiative within the GnuPG project, especially the TOFU, Trust
On First Use, trust model. It is a pretty fresh code not available in major
distributions yet, but it's definitely something to watch and evaluate in
future.)

Key management, assigning trust etc. is something which is outside of scope
for an e-mail client like Trojitá. We might add some buttons for key retrieval
and launching a key management application of your choice, such as Kleopatra,
but we are definitely not in the business of "real" key management,
cross-signatures, defining trust, etc. What we do instead is
working with your system's configuration and showing the results based on
whether GnuPG thinks that you trust this signature. That's when we are happy
to show a nice green padlock to you:

We are also making a bunch of sanity checks when it comes to signatures.
For example, it is important to verify that the sender of an e-mail which you
are reading has an e-mail which matches the identity of the key holder -- in
other words, is the guy who sent the e-mail and the one who made the
signature the same person?

If not, it would be possible for your co-worker (who you already trust) to
write an e-mail message to you with a faked From header pretending to be your
boss. The body of a message is signed by your colleague with his valid key, so
if you forget to check the e-mail addresses, you are screwed -- and that's why
Trojitá handles this for you:

In some environments, S/MIME signatures using traditional X.509
certificates are more common than the OpenPGP (aka PGP, aka GPG). Trojitá
supports them all just as easily. Here is what happens when we are curious and
decide to drill down to details about the certificate chain:

Encrypted messages are of course supported, too:

We had to start somewhere, so right now, Trojitá supports only read-only
operations such as signature verification and decrypting of messages. It is
not yet possible to sign and encrypt new messages; that's something which will
be implemented in near future (and patches are welcome for sure).

Technical details

Originally, we were planning to use the QCA2 library because it provides a
stand-alone Qt wrapper over a pluggable set of cryptography backends. The API
interface was very convenient for a Qt application such as Trojitá, with
native support for Qt's signals/slots and asynchronous operation implemented
in a background thread. However, it turned out that its support for GnuPG, a
free-software implementation of the OpenPGP protocol, leaves much to be
desired. It does not really support the concept of PGP's Web of Trust, and
therefore it doesn't report back how trustworthy the sender is. This means
that there woldn't be any green padlock with QCA. The library was also
really slow during certain operations -- including retrieval of a
single key from a keystore. It just isn't acceptable to wait 16 seconds when
verifying a signature, so we had to go looking for something else.

Compared to the QCA, the GpgME++ library lives on a lower level. Its Qt
integration is limited to working with QByteArray classes as buffers for
gpgme's operation. There is some support for integrating
with Qt's event loop, but we were warned not to use it because it's apparently
deprecated code which will be removed soon.

The gpgme library supports some level of asynchronous
operation, but it is a bit limited. Ultimately, someone has to do the work and
consume the CPU cycles for all the crypto operations and/or at least
communication to the GPG Agent in the background. These operations can take a
substantial amount of time, so we cannot do that in the GUI thread (unless we
wanted to reuse that discouraged event loop integration). We could use the
asynchronous operations along with a call to gpgme_wait in a
single background thread, but that would require maintaining our own
dedicated crypto thread and coming up with a way to dispatch the results of
each operation to the original requester. That is certainly doable, but in the
end, it was a bit more straightforward to look into the C++11's toolset, and
reuse the std::async infrastructure for launching background
tasks along with a std::future for synchronization. You can take
a look at the resulting code in the src/Cryptography/GpgMe++.cpp.
Who can dislike lines like
task.wait_for(std::chrono::duration_values::zero())
== std::future_status::timeout? :)

Finally, let me provide credit where credit is due. Stephan Platz worked on
this feature during his GSoC term, and he implemented the core infrastructure
around which the whole feature is built. That was the crucial point and his
initial design has survived into the current implementation despite the fact
that the crypto backend has changed and a lot of code was refactored.

Another big thank you goes to the GnuPG and GpgME developers who provide a
nice library which works not just with OpenPGP, but also with the traditional
X.509 (S/MIME) certificates. The same has to be said about the developers
behind the GpgME++ library which is a C++ wrapper around GpgME with roots in
the KDEPIM software stack, and also something which will one day probably move
to GpgME proper. The KDE ties are still visible, and Andre Heinecke was kind
enough to review our implementation for obvious screwups in how we use it.
Thanks!