What has been achieved

I prefer to set expectations, I haven’t worked hard on GtkSourceView and Tepl this time around, because the fundraising is not as successful as I would like. Since I’m paid less than one hour per week for that project, I don’t feel forced to work > 10 times more, I think it’s understandable.

But I still continue to maintain GtkSourceView and Tepl, and I’ve progressed a little for the file loading and saving. The tasks that I’ve done in November/December:

I’ve launched two months ago a fundraising for the GtkSourceView library. I intend to write a report every two months, so that you can follow what’s going on in that project, and at the same occasion I can explain in more details some facets of the fundraising.

Only one maintainer (me)

For most of the code in GtkSourceView, there is basically only one remaining maintainer: myself. Tobias Schönberg helps to maintain and review some *.lang files (for the support of syntax highlighting), especially in the area of web development. But that’s it.

So you can see that I’m the first contributor. All the other developers also contributed during their free time, and they don’t have enough free time anymore (they have an unrelated full-time job, etc).

So in some way, the future of GtkSourceView rests in my hands.

File loading and saving

These past weeks my focus was on file loading and saving in Tepl (the incubator for GtkSourceView).

There are several layers for the file loading and saving:

The backend/toolkit part, i.e. the low-level API, it’s what I’ve added to GtkSourceView in 2014 but needs improvements.

The high-level API taking care of the frontend, part of the Tepl framework.

Some features that are built on top of the high-level API, for example an action to save all documents, or the auto-save to automatically save a document periodically.

For the first layer, the backend/toolkit part, Tepl provides two new things: file metadata (for example to save the cursor position) and a new file loader based on uchardet, to improve the character encoding auto-detection. This past month I’ve improved the new file loader, it is now in a good enough shape for most applications (but still lacks some features compared to the old file loader, so some work is still needed to be able to deprecate the old implementation).

For the second layer, I’ve started to create the high-level API. Creating the API is not the most difficult, the bulk of the work will be to improve what the implementation does internally (creating infobars, handling errors, etc).

The third layer has not yet started.

File loading and saving was not the only thing that I did these past two months, a lot of other smaller things have been done, for more details see the NEWS files:

Conclusion

Even if GtkSourceView already provides a lot of features, it is far from sufficient to create even a basic text editor (to have an implementation of a good quality). To give a concrete example, the core of gedit – if we remove all the plugins – is currently made of 40.000 lines of code! It’s a lot of work for a developer who wants to create a specialized text editor or a new IDE.

So my goal with GtkSourceView and Tepl is to make more code re-usable.

I think it’s useful to have a list of projects fundraisers in GNOME or at least GNOME-related. Ideally it would be nice to have that list on the gnome.org website, it looks to me an obvious thing to do, but after a discussion on the GNOME foundation-list, it seems unlikely to happen anytime soon.

So I’ve created this wiki page in the meantime. It also explains the difference with donations made to the GNOME Foundation.

The list includes the GtkSourceView fundraiser that I launched last month. I plan to write regular updates on that front on this blog, for example every two months. Stay tuned, and thanks for your support :-)

Posted inUncategorized|Comments Off on List of GNOME-related projects fundraisers

If you don’t know what GtkSourceView is, it’s a widely used library for text editors and IDEs (or text editing in general). For example on Debian, more than 50 applications rely on GtkSourceView, including gedit and GNOME Builder.

What less people know about is that GtkSourceView has been almost entirely developed by volunteer work, without being paid, except a few Google Summer of Code. So with the fundraising it’ll hopefully change, to bring the library to the next level!

I wanted to share one recurrent API design that I’ve implemented several times and that I’ve found useful. I’ve coined it “attached class extension”. It is not a complete description like the design patterns documented in the Gang of Four book (I didn’t want to write 10 pages on the subject), it is more a draft. Also the most difficult is to come up with good names, so comments welcome ;)

Intent

Adding a GObject property or signal to an existing class, but modifying that class is not possible (because it is part of another module), and creating a subclass is not desirable.

Also Unknown As

Motivation

First example: in the gspell library, we would like to extend the GtkTextView class to add spell-checking. We need to create a boolean property to enable/disable the feature. Subclassing GtkTextView is not desirable because the GtkSourceView library already has a subclass (and it should be possible in an application to use both GtkSourceView and gspell at the same time1 ).

Before describing the “attached class extension” design pattern, another solution is described, to have some contrast and thus to better understand the design pattern.

Since subclassing is not desirable in our case, as always with Object-Oriented Programming: composition to the rescue! A possible solution is to create a direct subclass of GObject that takes by composition a GtkTextView reference (with a construct-only property). But this has a small disadvantage: the application needs to create and store an additional object. One example in the wild of such pattern is the GtkSourceSearchContext class which takes a GtkSourceBuffer reference to extend it with search capability. Note that there is a one-to-many relationship between GtkSourceBuffer and GtkSourceSearchContext, since it is possible to create several SearchContext instances for the same Buffer. And note that the application needs to store both the Buffer and the SearchContext objects. This pattern could be named “one-to-many class extension”, or “detached class extension”.

The solution with the “attached class extension” or “one-to-one class extension” design pattern also uses composition, but in the reverse direction: see the implementation of the gspell_text_view_get_from_gtk_text_view() function, it speaks for itself. It uses g_object_get_data() and g_object_set_data_full(), to store the GspellTextView object in the GtkTextView. So the GtkTextView object has a strong reference to the GspellTextView; when the GtkTextView is destroyed, so is the GspellTextView. The nice thing with this design pattern is that an application wanting to enable spell-checking doesn’t need to store any additional object, the application has normally already a reference to the GtkTextView, so, by extension, it has also access to the GspellTextView. With this implementation, GtkTextView can store only one GspellTextView, so it is a one-to-one relationship.

Other Examples

I’ve applied this design pattern several other times in gspell, Amtk and Tepl. To give a few other examples:

GspellEntry: adding spell-checking to GtkEntry. GspellEntry is not a subclass of GtkEntry because there is already GtkSearchEntry.

AmtkMenuShell that extends GtkMenuShell to add convenience signals. GtkMenuShell is the abstract class to derive the GtkMenu and GtkMenuBar subclasses. The convenience signals must work with any GtkMenuShell subclass.

AmtkApplicationWindow, an extension of GtkApplicationWindow to add a statusbar property. Subclassing GtkApplicationWindow in a library is not desirable, because several libraries might want to extend GtkApplicationWindow and an application needs to be able to use all those extensions at the same time (the same applies to GtkApplication).

Why not implementing spell-checking in GtkSourceView then? Because gspell also supports GtkEntry. [↩]

GtkUIManager has been deprecated without a good replacement for applications that want to keep a traditional UI (with a menubar, toolbar and statusbar). So I’ve written a new shared library called Amtk, currently developed inside the Tepl repository. It is a basic GtkUIManager replacement based on GAction. If you are interested, read the Amtk introduction (it explains the problems with what GTK+ currently provides and that Amtk solves) and the API reference.

Note that the library is not yet finished, factory functions are missing, release early release often etc. But I think that what remains to be done is not a lot of work (for my needs at least).

The name “Text editor product line” was inspired by this book: Feature-Oriented Software Product Lines. It’s an interesting book, I’m learning new things, like the difference between a white-box framework and a black-box framework, or cross-cutting concerns (and how to better handle them), etc.

It’s been a long time since the last chapter. I was busy with various programming projects as can be seen on this blog. But it’s important to share our knowledge. And a book scales much better than explaining again and again the same things to newcomers on IRC or mailing lists.

The guide follows a bottom-up approach. The last chapter was about writing semi-OOP classes in C. The new chapter is a small introduction to GObject. The next chapter that I’ve already started to write will finally be about GTK+, hopefully I’ll finish it soon.

The gspell bug tracker is perfect again, there are only feature requests (marked as enhancements).

I’ve fixed two bugs recently, the second one was not that easy to fix:

One crash (a failed assertion) probably due to a bug in an underlying library.

A responsiveness problem when editing long lines. It turned out that the spell-checking code for GtkTextView was very slow (200 ms to re-check the long line). So I’ve written a new implementation, which is 20x faster! So with 10 ms it’s now responsive.

And I regularly do other various maintenance tasks in gspell, as can be seen in the Git repository.

gspell is now used by at least 6 applications (see the list on the wiki page), and with both GtkTextView and GtkEntry support I’m sure a lot more applications will use it in the future.

If you like the work I’m doing, the gspell fundraising is still open. Your donations encourage me to continue to take care of gspell, to make it a rock-solid library and well-maintained in the long term. Thanks!

In this blog post I’ll explain in more details some aspects of Gtef: why a new library was needed, why calling it a framework, and one feature that I worked on during this cycle (a new file loader). There are more stuff already in the pipeline and will maybe be covered by future blog posts, stay tuned (and see the roadmap) ;)

Iterative API design + stability guarantees

In Gtef, I want to be able to break the API at any time. Because API design is hard, it needs an iterative process. Sometimes we see possible improvements several years later. But application developers want a stable API. So the solution is simple: bumping the major version each time an API break is desirable, every 6 months if needed! Gtef 1.0 and Gtef 2.0 are parallel-installable, so an application depending on Gtef 1.0 still compiles fine.

Gtef is a small library, so it’s not a problem if there are e.g. 5 different gtef *.so loaded in memory at the same time. For a library like GTK+, releasing a new major version every 6 months would be more problematic for memory consumption and application startup time.

A concrete benefit of being able to break the API at any time: a contributor (David Rabel) wanted to implement code folding. In GtkSourceView there are several old branches for code folding, but nothing was merged because it was incomplete. In Gtef it is not a problem to merge the first iteration of a class. So even if the code folding API is not finished, there has been at least some progress: two classes have been merged in Gtef. The code will be maintained instead of bit-rotting in a branch. Unfortunately David Rabel doesn’t have the time anymore to continue contributing, but in the future if someone wants to implement code folding, the first steps are already done!

Framework

Gtef is the acronym for “GTK+ Text Editor Framework”, but the framework part is not yet finished. The idea is to provide the main application architecture for text editors and IDEs: a GtkApplication on top, containing GtkApplicationWindow’s, containing a GtkNotebook, containing tabs (GtkGrid’s), with each tab containing a GtkSourceView widget. If you look at the current Gtef API, there is only one missing subclass: GtkNotebook. So the core of the framework is almost done, I hope to finish it for GNOME 3.26. I’ll probably make the GtkNotebook part optional (if a text editor prefers only one GtkSourceView per window) or replacable by something else (e.g. a GtkStack plus GtkStackSwitcher). Let’s see what I’ll come up with.

Of course once the core of the framework is finished, to be more useful it’ll need an implementation for common features: file loading and saving, search and replace, etc. With the framework in place, it’ll be possible to offer a much higher-level API for those features than what is currently available in GtkSourceView.

Also, it’s interesting to note that there is a (somewhat) clear boundary between GtkSourceView and Gtef: the top level object in GtkSourceView is the GtkSourceView widget, while the GtkSourceView widget is at the bottom of the containment hierarchy in Gtef. I said “somewhat” because there is also GtkSourceBuffer and GtefBuffer, and both libraries have other classes for peripheral, self-contained features.

New file loader based on uchardet

The file loading and saving API in GtkSourceView is quite low-level, it contains only the backend part. In case of error, the application needs to display the error (preferably in a GtkInfoBar) and for some errors provide actions like choosing another character encoding manually. One goal of Gtef will be to provide a simpler API, taking care of all kinds of errors, showing GtkInfoBars etc.

But how the backend works has an impact on the GUI. The file loading and saving classes in GtkSourceView come from gedit, and I’m not entirely happy with the gedit UI for file loading and saving. There are several problems, one of them is that GtkFileChooserNative cannot be used with the current gedit UI so it’s problematic to sandbox the application with Flatpak.

With gedit, when we open a file from a GtkFileChooserDialog, there is a combobox for the encoding: by default the encoding is auto-detected from a configurable list of encodings, and it is possible to choose manually an encoding from that same list. I want to get rid of that combobox, to always auto-detect the encoding (it’s simpler for the user), and to be able to use GtkFileChooserNative (because custom widgets like the combobox cannot be added to a GtkFileChooserNative).

The problem with the file loader implementation in GtkSourceView is that the encoding auto-detection is not that good, hence the need for the combobox in the GtkFileChooserDialog in gedit. But to detect the encoding, there is now a simple to use library called uchardet, maintained by Jehan Pagès, and based on the Mozilla universal charset detection code. Since the encoding auto-detection is much better with uchardet, it will be possible to remove the combobox and use GtkFileChooserNative!

Jehan started to modify GtkSourceFileLoader (or, more precisely, the internal class GtkSourceBufferOutputStream) to use uchardet, but as a comment in GtkSourceBufferOutputStream explains, that code is a big headache… And the encoding detection is based only on the first 8KB of the file, which results in bugs if for example the first 8KB are only ASCII characters and a strange character appears later. Changing that implementation to take into account the whole content of the file was not easily possible, so instead, I decided to write a new implementation from scratch, in Gtef, called GtefFileLoader. It was done in Gtef and not in GtkSourceView, to not break the GtkSourceView API, and to have the time in Gtef to write the implementation and API incrementally (trying to keep the API as close as possible to the GtkSourceView API).

The new GtefFileLoader takes a simpler approach, doing things sequentially instead of doing everything at the same time (the reason for the headache). 1) Loading the content in memory, 2) determining the encoding, 3) converting the content to UTF-8 and inserting the result into the GtkTextBuffer.

Note that for step 2, determining the encoding, it would have been entirely possible without uchardet, by counting the number of invalid characters and taking the first encoding for which there are no errors (or taking the one with the fewest errors, escaping the invalid characters). And when uchardet is used, that method can serve as a nice fallback. Since all the content is in memory, it should be fast enough even if it is done on the whole content (GtkTextView doesn’t support very big files anyway, 50MB is the default maximum in GtefFileLoader).

GtefFileLoader is usable and works well, but it is still missing quite a few features compared to GtkSourceFileLoader: escaping invalid characters, loading from a GInputStream (e.g. stdin) and gzip uncompression support. And I would like to add more features: refuse to load very long lines (it is not well supported by GtkTextView) and possibly ask to split the line, and detect binary files.

The higher-level API is not yet created, GtefFileLoader is still “just” the backend part.