Tuesday, June 22, 2010

fixed an issue with the new data from one view sometimes not being saved in the final epub when switching to the other view

worked around a Qt focus issue causing current tab data to sometimes not be saved; this was uncovered by fixing the hang-on-save issue, which was caused by the same underlying problem (issue #466)

The work in 0.2.1 has caused some unfortunate synchronization issues.

Problem #1: You work in Code View, make some changes and switch to Book View. You see the changes transfer to BV, then save the epub. Opening the epub, the changes are not there.

This was my fault and should now be fixed.

Problem #2: You work in Book View (or Code View), make changes and save. Sometimes the changes do save, sometimes they don’t. Same thing goes for the TOC editor sometimes not seeing your changes, or the Find/Replace dialog messing things up and losing data.

This is caused by a “lost” focus event. I’ve observed this only once on Windows 7, but it’s reportedly much more frequent on Win XP machines. Linux and Mac machines seem to be immune (although similar issues caused by problem #1 can masquerade as this problem). This seems to be a Qt issue, and I’ve now worked around it. The same underlying “missing focus event” issue was causing the previous hang problems.

There was a third problem reported: you make changes in CV, then switch directly to BV. Any changes are now gone. I’ve only had one user report this, all other reports appear to be caused by problems #1 and #2. I’m still unable to reproduce this, no matter what I do or what machine I use. If you have this issue with the new 0.2.3, please report it ASAP. Use the issue tracker.

I’ve been stealing away hours from my other obligations to work on Sigil since these are major problems, but man, doing this has already started to haunt me…

Sunday, June 20, 2010

simplified the resource locking mechanism; should eliminate the hang-on-save issue

fixed a problem with Book View chapter splitting sometimes not being registered on save, causing duplicate content (issue #450)

I’m trying to leave a rather stable version of Sigil before going on my coding break, so this release is important. Users have reported a higher chance of encountering the hang-on-save problem, and now I've completely changed a fundamental aspect of the resource locking architecture and also minimized all the critical sections so this should really be fixed now. If it’s not… then it’s caused by something completely unrelated. Which is possible, seeing as how I can’t reproduce the issue and thus get an exact fix on the culprit. The bug is not deterministic, which clearly points to the threading-enabled code.

The second major bug was caused by a line a code I forgot to add after some recent refactorings.

If you see the hang-on-save issue with this new release (or any other kind of hang bug, or any other kind of bug at all :) ), please report it ASAP with as much detail as possible.

fixed a regression that made it impossible to add removed headings back into the TOC (issue #439)

fixed a problem with some file-wide replacements reverting

fixed a problem with the opened tabs not being updated until the user gave them keyboard focus when a file-wide replace was performed (issue 408)

fixed a problem with Book View not reflecting changes done in Code View when the Code View was used for editing, and then the tab closed

use of custom synchronization primitives should resolve most infrequent hang bugs

fixed a problem with the search not progressing in Book View find&replace when using recursive replacements

fixed an issue with inserting images that have apostrophes in the filename (issue #391)

TOC text now has leading and trailing whitespace trimmed, and inner whitespace condensed (issue #422)

an empty ALT attribute is now added to "img" elements that don't have them (issue #406)

added the build time to the About dialog, showing date and time in UTC

Remember when I said that 0.2.0 was surprisingly free of major bugs? I spoke too soon. :)

The major issue was that replacements performed with the Find&Replace dialog could sometimes revert back. Horrible, I know. This should now be fixed.

The next major issue actually goes way back to the start of 0.2.0 betas, I just couldn’t track it down: Sigil could sometimes completely hang. This usually happened during saves, and should also be fixed now. I’ve written some custom synchronization primitives[1] that should alleviate these problems. This also involved rewriting the way tabs release their locks etc. Lot’s of nice things. Should work now.

And the last major problem was a regression that made it impossible to add removed TOC items back in.

Footnotes

[1] Nothing too fancy, I needed a ReadWriteLock that was shallow and non-recursive. Qt provides one that is non-recursive by default, but not shallow. By “shallow”, I mean a lock that allows only one level of locking by the same thread, thus silently letting multiple lock calls succeed, even though only the first one did. Same thing goes for unlock: the first call unlocks, the others just skip. The lock and unlock calls are appropriately called LockIfNeeded() and UnlockIfNeeded().

Admittedly, a shallow lock is rarely needed, and if you’re using one, you damn well better know what you’re doing.

Monday, June 14, 2010

Well it’s been quite a while since the last post. I’ve been busy with university work, and while I have one more paper to submit in a few days and finals in about ten, I have a bit of free time now.

I’ll be working through the bug reports I received since 0.2.0 final went live. I must say I’m pleasantly surprised that nothing major was reported. I’d like to thank all the people who have written thorough bug reports. I haven’t had time to go through them all and respond appropriately, but I’m getting there.

Sigil 0.2.1 will probably come sometime in late July/early August. While I have a few days free this week, I’ll be studying the following three weeks and immediately after that I’m going on a two-week vacation.

There’s an ever-so-slight chance that I’ll push a very minor bugfix release in about a week and postpone all the work I wanted to do for 0.2.1 into 0.2.2.

I’ve also started work on replacing all the uses of QDom with Xerces-C++ in a separate repo, but that’s going to take quite a bit of time.

What I’ve really spent a lot of time with over the last several days is QtWebKit. There was a bug on the WebKit Bugzilla for the general QtWebKit performance problems… When the bug was reported, my testcase on one machine took 55 seconds to load and render. For Qt 4.7beta1, Nokia got that down to 21 seconds. With the trunk version of Qt 4.7 from a couple of days ago, it’s down to 13.3 seconds. Bottom line, they’ve done some good work speeding it up, and some more work still remains. It’s not native-WebKit or Firefox fast, where the testcase renders instantaneously, but it’s improving.

I downloaded the Qt trunk source a few days ago and extensively profiled QtWebKit rendering. 95% of the time is spent shaping glyphs in the call tree of QFontMetrics::width(), and man does a lot of code get run just to get the width of a text string. The calls drop into Harfbuzz, and all sorts of functions end up eating a bit of wall time here and there, and this all adds up. There’s no single point that does something very stupid that could then be optimized away, at least not in this module. There’s even an internal “simplified” codepath for getting the width of a text string, but I’m not sure why the Nokia guys are not using it in QtWebKit. I’m probably missing a piece of the puzzle. Either way, there was no low-hanging fruit to remove.

The most fruitful optimization work will probably come from higher up the call chain. For instance, there are superfluous layout calls for page rendering, which (if I’m reading the bug comments right) make the whole thing calculate the render tree three times. Ouch. But they’re working on it.

The point to take home is that when Qt 4.7 lands, Sigil should see a very nice performance boost for loading books and switching to the Book View. Probably for general WYSIWYG actions in the Book View too. Hurrah for that.