Tag Archives: python

Recently, there was a dot story about Frameworks 5: Started in spring of 2011, the KDE software stack is undergoing a heavy split. The idea is to modularize the KDE libraries into lots of rather small units. Each unit has well-defined dependencies, depending on whether it’s in the tier 1, tier 2, or tier 3 layer, and depending on whether it provides plain functionality, integration, or a solution. If you haven’t yet, please read the article on the dot for a better understanding.

With this modularization, the question arises about what will happen to the KTextEditor interfaces and its implementation Kate Part – and of course all the applications using Kate Part through the KTextEditor interfaces, namely Kate, KWrite, KDevelop, Kile, RKWard, and all the others… The purpose of this blog is to give an answer to these questions, and to start a discussion to get feedback.

Since not everyone is familiar with “Kate & friends” and its sources, let’s have a quick look at its software architecture:

From this figure, we can see that the software stack has 3 layers: the application layer, the interfaces layer and the backend layer. KTextEditor provides an interface to all functions of an advanced editor component: Documents, Views, etc. (see also the API documentation). However, KTextEditor itself does not implement the functionality. So it’s just a thin layer that is guaranteed to stay compatible for a long time (for the KDE 4 line, KTextEditor is compatible since 2007, so as of now 7 years). This compatibility, or better yet interface continuity is required, since applications in the Application layer use the KTextEditor interfaces to embed a text editor component. The implementation of the text editor itself is completely hidden in Kate Part. Think of a backend library implementing all the KTextEditor interfaces. When an application queries KDE for a KTextEditor component, KDE loads a Kate Part behind the scenes and returns a pointer to the respective KTextEditor classes. This has the advantage, that Kate Part itself can be change without breaking any applications, as long as the interfaces stay the same. Now, with KDE Frameworks 5, this principle will not change. However, interfaces will undergo a huge cleanup, as will be explained now. As a consequence, all nodes that point to or from the KTextEditor node, namely Kate Part on the backend layer as well as applications, will need to adapt to these interfaces.

Milestone 1: KTextEditor and Kate Part on 5

KTextEditor will be a separate unit in the frameworks split. Therefore, the KTextEditor interfaces will not come bundled with one monolithic ‘kdelibs’ as it was the case for that last 10 years. Instead, the KTextEditor interfaces are developed and provided in a separate git repository. This is already now the case: The KTextEditor interfaces exist as a copy in Kate’s git repository, and relevant changes were merged into kdelibs/interfaces/ktexteditor in the KDE 4.x line. For “KTextEditor on 5,” the first milestone will be to get KTextEditor compile with the libraries and tools from the frameworks 5 branch. Along with this port, the KTextEditor interfaces have a lot of places that are annotated with “KDE 5 todos.” That is, the KTextEditor interfaces will undergo a huge cleanup, providing an even better API for developers than before.

Currently, the KTextEditor and therewith also its implementation Kate Part use the KParts component model. The KParts model allows to easily embed Kate Part in other applications along with Kate Part’s actions and menus. Further, Kate Part internally uses KIO to load and save files to support network transparent text editing. KParts itself and KIO are both Tier 3 solutions. This implies that KTextEditor along with its implementation Kate Part are a Tier 3 solution.

In other words, a straight port and cleanup of KTextEditor and Kate Part will depend on a lot of high level frameworks. This solution will provide all the features the KTextEditor interfaces provides right now in the KDE SC 4.x line.

Currently, we plan one major change in the KTextEditor on 5: We will remove KTextEditor plugins. Over the last 10 years, we got close to no contributions to the KTextEditor plugins. Existing KTextEditor plugins partly clash with code in Kate Part (for instance the Auto Brackets with the Autobrace plugin), and merging the plugin’s xml gui into the KTextEditor::Views always requires some hacks to avoid flickering and make it work correctly. Besides, if the KTextEditor plugins are removed, for instance the Kate config dialog only shows one “Plugins” item instead of two. This is much cleaner to the user. Existing functionality, like for instance the “Highlight Selected Text” plugin, will be included into Kate Part directly. The same holds true for the HTML export feature. This is a bold change. So if you want to discuss this, please write to our mailing list kwrite-devel@kde.org.

The time frame for the KTextEditor port & cleanup is rather short: We want to provide rather stable KTextEditor interfaces so that other applications can rely on it. Therefore, we will probably create a frameworks branch in the Kate git repository in December (current proposal on kwrite-devel). Binary and source incompatible changes will be allowed until other applications like KDevelop or Kile are ported to Frameworks 5. Then, the KTextEditor interfaces will again stay binary compatible for years.

Milestone 2: KWrite and Kate on 5

KWrite is just a thin wrapper around the KTextEditor interfaces and therewith Kate Part. Therefore, KWrite will mostly support just the same functionality as it provides now. The same holds true for Kate. However, Kate itself provides quite a lot of advanced features, for instance to have multiple main windows (View > New Window), or sessions, and a good plugin infrastructure. Of course, Kate itself will also undergo cleanups: i) cleanups due to changes in the KTextEditor interfaces, and ii) cleanups like for instance moving the Projects plugin Kate itself, making it more easily accessible to other plugins like the Search & Replace or Build plugin. We will also remove support for multiple main windows through “View > New Window.” This is due to the fact, that many Kate plugin developers were not aware of this feature, and therefore completely messing up their code by not separating the logic from the view, resulting in crashes or broken behavior when using multiple main windows. Removing the support for multiple main windows, we will loose this feature. However, we get simpler and more maintainable code.

There are other small details that will change. For instance, as it looks right now, the Python pate host plugin in Kate on 5 will only support Python 3 (current discussion on kwrite-devel). Python developers, you are welcome to contribute here, as always!

Milestone 3: More Modularization in the KTextEditor Interfaces?

Milestone 1 & milestone 2 will happen rather sooner than later (fixed dates will follow once we’re sure we can satisfy them). Since the transition to Frameworks 5 allows us to change KTextEditor interfaces, it is the right time to think how we can improve the KTextEditor interfaces and its implementation Kate Part even further. For instance, on the mailing list, the idea was raised to make the KParts model optional. This could be achieved for instance by deriving KTextEditor::Document from QObject directly, and create a thin KParts wrapper, say KTextEditor::DocumentPart that wraps KTextEditor::Document. This would be a major change, though, and possibly require a lot of changes in applications using the KTextEditor interfaces. As of now, it is unclear whether such a solution is feasible.

Another idea was raised at this year’s Akademy in Bilbao: Split Kate Part’s highlighting into a separate library. This way, other applications could use the Kate Part’s highlighting system. Think of a command line tool to create highlighted html pages, or a syntax highlighter for QTextEdits. The highlighting engine right now is mostly internal to Kate Part, so such a split could happen also later after the initial release of KTextEditor on 5.

Join Us!

The Kate text editor only exists thanks to all its contributors. Moving to frameworks, it is the perfect time to follow and contribute to the development of Kate. In fact, you can learn a lot (!) in contributing. In case you are interested, have ideas or want to discuss with us, please join our mailing list kwrite-devel@kde.org.

Another release cycle gone, and the KDE Software Compilation 4.11 is out in the open (well, for quite some time already), and with that it is time to talk about what changed in Kate the last half year since the 4.10 release. Besides the usual bug fixing (~50 bugs since 4.10), the following sections present some major improvements and features of Kate in 4.11.

Python Plugins

Since KDE 4.10, Kate features the Python »Pâté« plugin. This plugin basically wraps the API of all Kate application interfaces. Therewith, together with the Python KDE (PyKDE) bindings you can develop full-featured Kate plugins in Python. Being relatively new, there are quite some Python plugins available already now (see this blog, this blog, or this blog).

@Python community: You can interpret this as an invitation for writing lots of cool addons for the Kate text editor in python. All contributions are usually very much welcome!

Vi Input Mode

Enabling the vi input mode turns Kate into a full-fleged vim compatible editor: modal editing exactly like in vim. This mode is especially suited for vim users who want a fully KDE integrated text editor with all the beloved vim features. Started as GSoC project in 2010, the vi input mode evolved over the years to become a very mature alternative to vim itself. However, there is always room for improvements. The good news is, that Kate Part’s vi input mode gained a lot of attention thanks to Simon, mostly in the form of small “papercuts” that fixed small bugs and annoyances and made behaviour more compatible with Vim, including:

Yank highlighting, which helps you to see what text you just yanked;

“Yank to clipboard” (courtesy of Fabian Kosmale!);

The “last edit” markers, “.”, “[” and “]”;

Recursive vs non-recursive mappings, for if you want to e.g. map “j” to “gj” …

… plus plenty of fixes to gj and gk;

and countless other small tweaks to things like the “inner block” text object; cursor position after paste; fixes to ctrl-a/x (add to/ subtract from number under cursor); making more motions available in Visual Mode and more motions counted, etc!

One fairly major change is the introduction of an experimental Emulated Vim Command Bar. This, as you will probably have guessed, is a replacement for the current Search dialog that behaves more like Vim’s, allowing one to e.g. insert the contents of registers via ctrl-r; dismiss via ctrl-c/ ctrl-[ as well as ESC; work properly with both forward (“/”) and backward (“?”) incremental searches; use smart case for case-sensitivity; allow the use of Vim-style regex’s etc. As an extension to Vim, I’ve also added the ability to auto-complete words from the document via ctrl-space: not that useful for searching, but very useful for when you want to do a search+replace command (command mode (“:”) did not make it into 4.11, alas).

In 4.11, it is disabled by default as it is not yet ready for primetime, but may be enabled by setting the hidden config option “Vi Input Mode Emulate Command Bar” to “true”. It is much more powerful and featureful in current master (see the blog here!) and will likely be the default in 4.12.

New Text Folding

Back in KDE SC 4.8, Kate got a new code folding implementation as part of the yearly GSoC project. Compared to the previous implementation, the amount of bugs indeed reduced, but during the 4.9 and 4.10 release it became apparent, that the code folding was still not where it should be. Maybe writing a good code folding implementation was also a bit too much in terms of a newby developer in a GSoC project.

So Christoph sat down and wrote it completely new from scratch, this time very clean and simple. And as a new feature, the text folding is now per view, so the folded parts can be different in each view of a document. And as another feature for those who do not use text folding at all: Not using text folding means you have zero overhead, no matter how large the file. Besides that, code folding in Python works now better than ever before, in fact, it should be bug-free.

Passive Notifications & KTextEditor Interfaces

Passive notifications are meant to provide a non-intrusive way showing notifications (replacing KPassivePopup). Used e.g. by the data recovery, these notifications were also introduced for the search & replace feature (most of this is already available in KDE SC 4.10.3). In KDE SC 4.11, the interface for showing these passive notifications got officially included into the KTextEditor interfaces, meaning that applications like Kile or KDevelop can show notifications as well. (Issues in KDE 4.11.0 should be fixed for 4.11.1).

Saving Files

When saving files, Kate uses the class KSaveFile (in Qt5 available as QSaveFile thanks to David Faure). In short, to avoid data loss, KSaveFile saves the file to a temporary file in the same directory as the target file, and on successful write finally moves the temporary file to the target filename. If the folder of the file does not allow creating new files, KSaveFile automatically falls back to writing the file directly. In this case, data loss could happen if e.g. the system crashes in this moment, but saving the file would otherwise be not possible at all.

What Comes Next?

Announced just a few days ago, the KDE community prepares for KDE Frameworks 5 and much more: The KDE libraries and the KDE Plasma Workspace are now in long term support mode, meaning that bugs will get fixed for another two years. That means that the KTextEditor interfaces are frozen as well, there will not be any feature updates at all in the KTextEditor interfaces itself. Since Kate Part and the Kate Application are developed outside of kdelibs, Kate Part and Kate will most certainly see another feature release. Especially, since there are already a lot of improvements for the vi input mode in the pipe for Kate in KDE SC 4.12.

After that, we plan to port Kate to Qt5 and the new frameworks 5 libraries. We already added a lot of “KDE 5″ todo-markers to the Kate source code, meaning that we will probably work on a 5 port rather sooner than later. We will address this schedule in a separate post later this year, so stay tuned!

I have done some improvements in the plugins: python_console_ipython, python_autocomplete, python_utils, js_utils, xml_pretty and django_utils. These plugins I added a month and a half ago (except python_console_ipython) to the kate repository. I have done two improvements and a new feature:

Now they work with Python2 and Python3 (except python_autocomplete, this only works with Python2, due pysmell dependence)

Now they have a configuration page (except python_autocomplete, this should not have configuration parameters)

Page Config Plugins

The new feature is the integration of the python_autocomplete and python_console_ipython plugins with the project plugin. The behaviour of these plugins depends of the loaded project. That is to say, the python_autocomplete plugin autocompletes with our project modules. Currently the kate community is working to add a new python autocomplete using jedi (this will work with Python2 and Python3).

Python Autocomplete (with our modules)

And the python_console_ipython plugin has something that we can name the project console. A ipython shell with the python path of our project and with the environments variables that the project plugin needs.

IPython Console (converted in a django shell)

To use this we need to add in the project file (.kateproject) a new attribute named “python”, with this structure:

I am a django developer, as we say in Django we can come to have a django shell (python manage.py shell) integrated in our editor. But this feature is a generic feature not tied to Django. This feature can be used by other Python developers. I have added only a specific feature to the django developers. If we set in the project file (.kateproject) the attribute projectType with the value Django, the ipython shell automatically loads the models and the settings like a shell_plus does.

The project plugin in kate.git master now has four new more plugins, with many features in each one: Python (autocomplete, smart snippets, parse checker, pep8 checker and pyflakes checker), Javascript (autocompletes, jquery snippet, pretty JSON, and jslint checker), Django (smart snippets and utils to Django template) and XML (pretty xml). There are also many generic functions and generic classes in these that will be useful to the new plugins developers.