Bug Description

In ui.py do_convert() I here observe a race between the TagReader and the real start of the the conversion. There 's an idle callback for tags_read() which adds the sound_file to the queue, but converter.start() is called prior than that and may skip files or process an empty queue even. If the idle callback adds a file to the queue later, that one will be available only for subsequent usage of the "Convert" button and leads to weird errors as two tasks in the queue want to work on the same file.

What's problematic here is that the tag reading can trigger the gstreamer plug-in search. If a missing plug-in is not found or not installed, the tag reader doesn't continue. Dunno yet what happens to the converter task queue in that case.

For comparison, Fedora 18 is at glib2-2.34.2, gtk2-2.24.13, pygtk2-2.24.0.

One can observe that with Fedora 18, the tag reading is not race-free either. The last pending idle callbacks may add tasks to the converter queue after it has been started already. With regard to the task queue implementation, that's okay because it's an array "append", and the queue uses "pop" to retrieve waiting tasks. But if the tasks are added too late via idle calls, processing of the queue would have stopped already. That's easy to reproduce with Fedora 19 when converting a single audio file.

Other symptoms: Where with Fedora 18, any of the modal dialogs (such as "file exists - what to do?" or "missing plugin search") seem to block the main loop -- so if I close the first dialog, the next one is displayed -- with Fedora 19 only the first dialog is displayed and all other ones are ignored.

If this is not just temporary, it might be that something has changed to the gobject idle call behaviour and hence affects SoundConverter, where even the background tasks use idle calls.

summary:

- Race between TagReader and start of conversion+ Idle call race between TagReader and start of conversion

in tags_read() callback: self.tag_reader_tasks.remove(tagreader) # maintain list of TagReader tasks
# only add files where TagReader was successful
if sound_file.tags_read: # or if not True, alternatively store an error to be found by do_convert() before converter.start self.converter.add(sound_file)

Yeah, all the current complex flow is stupid and prone to errors.
I don't remember why we are trying to read tags and convert at the same time, but this is way too complex.
I think this was used to remove tags from memory while we were converting, to decrease memory usage. But it's not worth it.

I wanted to fix all this when rewriting soundconverter 3.0 from scratch, but it seems not so hard to fix in fact.
Just read tags if needed, and them convert.

This is exactly what you are proposing BTW. I still have to figure what happens when we have a codec installation...

(1) gnome-packagekit has been broken for several releases of Fedora including current Fedora 18 ( https://bugzilla.redhat.com/756208 ). It crashes, which makes Soundconverter treat that as unsuccessful installation immediately. Only very rarely one can install a missing plugin that way.

(2) installation of the plugins is asynchronous on top of TagReader being based on BackgroundTask gobject idle callbacks. Currently, and especially with Fedora 19 development, I cannot be sure that TagReaders, which encounter a missing codec, are processed properly at all. One of the running tag readers results in the async dialog for codec installation, but the others don't receive the install_plugin_cb callback (and possibly not even a GST ASYNC_DONE message).

It still contains the forwarded exceptions work-around. And it waits for the tag readers to complete before starting the actual conversion. Because that is done, I forward another flag from inside the gstreamer Pipeline that tells the main UI whether a gstreamer plugin has been installed. If that happens during tag reading, I simply restart the entire do_convert step (= the tag reading!), and that may prompt for further missing plugins. Great! So far that even works with Fedora 19 development.

Here it works so far that the actual conversion won't be started anymore, but the loop that starts a tag reader for each file cannot be interrupted (because it iterates over a fixed list of files with no implementation that could break out of the loop). And if one of the tag readers has triggered async installation of a missing codec, the pending dialog will appear, too. The GStreamer exceptions are forwarded, however, so such errors would be handled.

Pausing the tag reading, oh well, that has not been implemented before. The existing code only tries to pause the converter queue.

I've little free time these days, and the problem is complex, so it's going to be hard...

In fact, we are basically screwed here.
If we wait for the tag readers to finish, we have to implement pause/cancel, and it's very slow.
If we rename files after conversion, all overwrite/error stuff is broken. And what about network vfs?

And we want a fix ASAP, to release a new version.

The best and easiest solution may be to use one queue for both tags and conversion.
And be sure to add the following converter before "finishing" the tagreader.

The biggest challenge being that ConverterQueue must be replaced.

Another solution could be to add the tag reading in Converter or ConverterQueue, but it's not a great idea from a design point.