Main Menu

News Feed

Welcome to the fre:ac developer blog. I will post status updates and other information about fre:ac development here.

fre:ac development status update 08/2019

Written by Robert

Saturday, 07 September 2019 12:16

Hi folks, here's the latest fre:ac development status update covering the months of June to August of 2019.

Yes, it's again been three months already since the last update. While my goal is to write monthly reports, my free time is limited and especially in summer, working on fre:ac is competing with other activities like cycling and kayaking, so needless to say it's suffering a bit. The upcoming rainy autumn and cold winter should lend themselves better to spending time on open source projects though. ;)

As already teasered in the previous report, I have some exciting news today and it's about fre:ac's I/O performance.

I/O performance improvements

Doing most of my testing on a relatively fast SSD, I usually found fre:ac's I/O performance to be just fine. It turned out, however, that when putting the output folder on an HDD drive, it can be horribly slow.

This is especially true when doing parallel conversions with lots of threads and writing to an internal HDD. A user with a 16 core / 32 thread Threadripper system approached me, claiming the conversion performance would not scale beyond 8 threads and asked me to have a look at that.

Well, I did and found that while performance was OK when the output folder was on an SSD, it could be more than 20 times slower when writing to an HDD. Looking at the I/O performance in Task Manager revealed that fre:ac was writing a mere 6 MB/s when the disk should be capable of much higher throughput.

The reason turned out to be an oversight of mine when implementing the audio file I/O layer. While fre:ac's I/O system uses buffered I/O in most cases - data is collected in a buffer first and then written in larger chunks - this was not true for most audio data writing (due to a lower - closer to metal - layer being used there). For the low level calls used for writing audio data, buffering was not enabled, leading to audio data being written in many small chunks and congesting the operating system's I/O queue.

Fixing this by enabling buffering on the low level I/O calls instantly improved performance by a huge margin. Here are some numbers for before/after performance with different target formats:

Converting 2586 mixed songs to MP3

Converting 2586 mixed songs to FLAC

So we're looking at a 4x performance increase when writing small (MP3) files to an HDD and up to 20x speed-up with large (FLAC) files. And even when the output location is on an SSD we are seeing more than 2x improvement in some cases. I did see even higher performance gains when converting a smaller number of files, as disk and OS write caches have a larger effect on the result in that case.

If you are on Linux, you can already try these improvements with the continuous AppImage builds or the edge channel Snap.

Multi-threaded adding to joblist

Sometimes, thinking again about a feature previously thought to be difficult to implement will bring up new ideas for a simpler approach. In June, a user brought up the topic of using multiple threads when adding files to the joblist. While I thought about that before, I deemed it too complex to realize in the fre:ac 1.1 development cycle - it was thus on my list for possible 1.2 features.

Thinking about this again, however, I quickly came up with an approach much simpler than my previous ideas. In short, the algorithm assigns tasks to threads in a round-robin manner while the actual adding to the joblist is still done in a single management thread. This is a bit less efficient than an ideal solution, but is so simple in fact, that it could be implemented in just a few hours and without much risk to break anything else in the program.

So this is now implemented and in combination with the aforementioned I/O improvements and some other optimizations, it can greatly speed up the time needed to add files to the joblist. Adding my test library of about 2600 songs in different formats now takes roughly 10 seconds with these changes compared to about 2 minutes with the 20190423 release:

I included the 1.0.32 release in this comparison as it was actually faster than the 20190423 snapshot. This is due to the development releases reading a lot more information (like cover art) from the files. Also, adding the tracks to the tag editor in addition to the joblist takes some time, so it's good to see the new code outperforming any previous version of fre:ac.

As with the I/O improvements, Linux user can already try these changes with the continuous AppImage builds or the edge channel Snap.

Dark mode on macOS

In July and August, I was mostly working on implementing dark mode support for the macOS version of fre:ac.

There actually are two aspects to this. The first being actual work on support for dark color schemes - some elements had colors hard-coded to a value fit for a light theme only. The second - and vastly more difficult one - is reworking the drawing code architecture of the macOS version. The old drawing code used some deprecated patterns no longer allowed in modern macOS. Applications built using an older SDK and supporting only light mode can run in a compatibility mode, however, which is what fre:ac is still doing in the latest release. As soon as you declare dark mode support though, your application will no longer run in compatibility mode and will have to implement the latest drawing paradigms.

So this took a while to implement and is not quite finished yet, but should be fully integrated within the next one or two weeks.

Smaller improvements

Multi-channel Monkey's AudioThe Monkey's Audio codec got support for multi-channel audio in a recent release and fre:ac now supports that.

Tagging improvementsSome tag fields previously only available in ID3v2 tags are now supported in a wider range of tagging formats. This mostly effects the Mix artist, Grouping, Disc subtitle, Copyright and Catalog number fields.

Bug report and feature request templatesThe fre:ac issue tracker on GitHub now has templates for bug reports and feature requests which should help improve the quality of issue descriptions and ultimately make it easier and quicker for me to help.

Besides these improvements, several bugs have been fixed in the past three months. I'm working towards publishing a new release, but cannot give a definite time frame for it yet.

That's it for this issue. Make sure to come back here for the next one (hopefully in one month ;)).

fre:ac development status update 05/2019

Written by Robert

Thursday, 06 June 2019 21:00

Hi all, this is the fre:ac development status update covering April and May 2019.

A lot of interesting things have been worked on in the past two months including a new alpha release, continuous builds for Linux and several improvements and fixes.

New alpha release

The 20190423 alpha release now allows ripping with more CD drives than your system has CPU cores available. This was requested by a user who planned to connect 10 drives to his quad core system for ripping his music collection of several thousand CDs. As the CD drive's reading speed is usually the limiting factor when ripping with the CPU not fully utilized, most modern systems can easily handle multiple ripping threads per CPU core. Previous versions were limited to one thread per core, so the new release allows for great speed ups of such large scale conversion jobs.

Other than that, the new release ships mostly bug fixes. Everything already mentioned in the last report plus some last minute changes:

Added support for cue sheets referencing multiple multi-track files

Fixed decoding of very short Ogg files (Vorbis, Opus and Speex)

Fixed more thread synchronization issues to improve stability

Continuous AppImage builds

The Travis CI system now builds AppImage packages for the four major architectures (i686, x86-64, ARM and ARM64) for every commit to the code repository.

This means that if you are on Linux, you can now try out the latest changes almost immediately. You can find the continuous builds on the downloads page and on GitHub.

iTunes store app support

For users of Windows 10, iTunes is now offered as an app download from the Microsoft store. Unfortunately, this means that the current integration method for the Core Audio encoder with fre:ac no longer works. You currently need to explicitly select and install the non store version of iTunes in order to continue using this high quality AAC codec in fre:ac.

This will be fixed by using an adapted integration method starting with the next fre:ac release. The new version will detect an installed iTunes app and load the Core Audio codec that comes with it.

Other fixes and updates

Several minor updates and fixes have been implemented in the past few weeks and will be included in the next release:

Fixed invalid file time for CD rips when using "Keep time stamps" option

Fixed ID3v2 implementation being not completely standards compliant

That's it for this issue. Be sure to stay tuned for the next one which will have some exciting news!

fre:ac development status update 03/2019

Written by Robert

Wednesday, 27 March 2019 19:35

Hi all, here's a new fre:ac development status update. I missed the January and February updates and thus decided to do this one a bit earlier. Also, I'll be on vacation the next two weeks, so there's not much more development going to happen in March anyway. As this report is covering almost three months, it got much longer than usual.

Snap and AppImage packages

After introducing Flatpak packages for fre:ac in December, I worked on getting Snap and AppImage packages up and running in January. As a result, fre:ac is now available in the Snap store and AppImages have been added the the downloads section.

Also, since just a few days ago, continuous builds of fre:ac are now available as Snaps in the edge channel of the Snap store (with official releases now listed in the beta channel). I also sorted out an issue with the build process on the ppc64el architecture, so if you have such a machine, you can now enjoy fre:ac as well.

Following this, continuously built AppImages will hopefully be available soon too.

PulseAudio output component

The Linux versions of fre:ac will make use of the PulseAudio sound architecture starting with the next alpha release. This replaces the previously used ALSA output component and enables you to control fre:ac's volume independent of other applications in your desktop's mixer. It also simplifies the process of packaging fre:ac as a Snap and helped making continuous builds available.

Media Foundation WMA decoder

On the Windows platform, the WMA decoder based on Microsoft's Windows Media Format SDK has been replaced by one building on the newer Media Foundation APIs. The new decoder is used on all systems that support Media Foundation and works around a WMA Lossless decoding bug on recent versions of Windows 10.

Improved playlist files' interoperability

A user noted that M3U playlist files generated by VLC could not be opened in fre:ac (via the File->Load joblist feature). Turned out that VLC is putting file: URIs in its M3U files and fre:ac was not ready for that. This is now fixed so the next release will be able to handle such files.

A similar issue existed with XSPF playlists. With this format though, in addition to fre:ac not being able to handle VLC's files, the latter also could not open files generated by fre:ac. The next release fixes this and will provide complete XSPF interoperability between the two programs.

Bug fixes and minor improvements

In addition to these bigger changes, a lot of bug fixes and smaller improvements have been implemented since the beginning of the year:

Fixed conversion failing for MP4 and WMA when input and output files are the same

Show only one error dialog when adding multiple folders by drag & drop

Performance and compatibility updates for Monkey's Audio

Not directly related to fre:ac, but still noteworthy, I recently sent some more patches to the author of the Monkey's Audio codec. These provide better compatibility with Linux and Unix-like systems and speed up decoding by approximately 5% by using a faster CRC calculation algorithm previously only used for encoding plus some other optimizations.

That's it for this issue. I will be on vacation now and hope to be able to cut a new alpha release soon after I'm back. Watch for it around mid to end of April!

fre:ac development status update 12/2018

Written by Robert

Saturday, 05 January 2019 23:18

Hi and welcome to the latest fre:ac development status update! I did not write a report for November as I was busy preparing the latest alpha release around the end of the month, so this report covers both, November and December of 2018.

BeGeistert 2018

On the 3rd and 4th of November I attended the BeGeistert 2018 Haiku developers conference which was held in my hometown of Hamburg, Germany. I met great people from the Haiku project, got to work on the fre:ac port for Haiku and ported Monkey's Audio over.

It was a great event and a nice experience and I'm already looking forward to attending future BeGeistert meetings in the years to come.

Flatpak (and work on Snap)

The big news for December certainly is fre:ac finally being available as a Flatpak. This makes it easier than ever for Linux users to discover and try out fre:ac.

The fre:ac Flatpak is available from Flathub, the largest application repository for Flatpak distribution.

Now that the Flatpak is ready, I'm shifting my attention towards building a Snap package as well. There already is some progress, but I did not get to work on the project a lot during the holiday season. Thus I now expect the Snap to be ready around mid January.

Enhancements

The December release of fre:ac added a notification component that can play a customizable sound or display a message box upon finished conversions. This was requested by several users doing very large conversions that might take anywhere from several minutes to hours. They often leave fre:ac running unattended, so an audible notification upon job completion can be useful.

Additionally, the December release now supports accent colors on macOS Mojave and improves scaling on HiDPI displays, especially with radio buttons and with edit fields where text was sometimes cut off in scaled mode.

Minor enhancements added in November and December include an option to switch stereo channels that was added to the channel converter DSP component and support for drag & drop in the tag editor. The latter did not make it into the December release though and will debut in the next alpha or beta.

Various fixes

In addition to the above changes, many bugs have been fixed in the past two months:

Fixed MP4 files not being created when <directory> placeholder is used in output filename pattern

Fixed tag editor being unable to open some file types on case-sensitive file systems

Fixed freaccmd issues when input and output files are the same

Fixed crash when converting mono to stereo

Fixed encoding mono MP3s

That's all for this issue. The report for January should be out in early February, so make sure to check the site for updates.

fre:ac development status update 10/2018

Written by Robert

Saturday, 03 November 2018 09:35

Hi all, this is the fre:ac development status update for October 2018. I have a lot of interesting things to share.

Haiku packages and fixes

First and foremost, I created fre:ac packages for the Haiku operating system. The Haiku project just released their first beta in the end of September and fre:ac is now available via their HaikuDepot package manager. Just search for it and you are one click away from installing fre:ac.

I also fixed the Fraunhofer AAC encoder and updated the LAME MP3 encoder package for Haiku while at it. I plan to also port Monkey's Audio to this interesting OS.

But while it's great that fre:ac now runs on Haiku, the 20180913 release is still far from perfect. I implemented a lot of fixes in October to make the next alpha run much better:

Fixed window titles becoming inactive when a menu, dropdown or tooltip is shown

Fixed scrollbars not working when moving the cursor outside the window

Improved image downscaling using a weighted average box filter

Added support for font scaling (limited HiDPI mode)

Fixed application signature to avoid warning on startup

Fixed resource compilation to simplify package script

A new alpha with these fixes and more will probably be out in mid-November.

Notarization of macOS version

Apple recently introduced notarization for apps distributed outside the app store. It's basically an automated malware and best-practices check for applications that when passing will display a "this app was checked by Apple" message on macOS Mojave.

It took me a weekend to adapt the fre:ac package for this new security feature, but now it's functioning very well. Starting with the next release, fre:ac for macOS will be notarized by Apple.

Other fixes

Some other fixes have been implemented in October:

Linux HiDPI fixesThe Linux version of fre:ac will now evaluate the GDK_SCALE environment variable in order to automatically scale the font and UI element size on HiDPI displays.

freaccmd fixes for non-Windows systemsThe fre:ac command line interface had some issues with spaces in file names. Namely, the current version does not work with the standard Unix shell space escaping (which is putting a backslash in front of each space character). This will be fixed in the next release.

Fixed issue with enabling "Write to input folder" and "Delete original files after encoding" optionsWhen enabling the "Write to input folder" and "Delete original files after encoding" options at the same time and the output filename equals the input filename, the current version of fre:ac can delete both, the existing input and the new output file after conversion, leaving you with none of the two. The next release will fix this and will be brought forward one or two weeks for this.

Work place changes

Finally, after more than 15 years at the previous company, I changed my job on 1st of October. I'm now working for PreSonus Software Ltd. in Hamburg, Germany on tasks much closer to what I'm already doing in my free time with fre:ac.

I will mainly work on the Notion music notation software, but also look into improving Studio One's audio file export and conversion capabilities and much more.

I'm happy to be part of the PreSonus team and really looking forward to great things coming in the future.

That's all for this month. Be sure to come back next month for another update.

fre:ac development status update 09/2018

Written by Robert

Sunday, 07 October 2018 17:47

Hi all, this is the fre:ac development status update for September 2018.

New alpha release

I published a new alpha last month, integrating the SuperFast encoding mode and freaccmd's dynamic arguments support that I wrote about in last month's issue. With these features added, the new alpha is now almost feature complete with respect to what is planned for fre:ac 1.1 beta.

Improved tags interoperability

Several changes have been implemented to improve compatibility with tags written by other applications (most notably foobar2000). These changes enable fre:ac to extract more information from files containing such tags than previously possible. The individual improvements are:

Support reading and writing ID3v2 tags from/to .wav files

Support reading cue sheets from ID3v2 tags

Use album artist for artist field if only the former is set

One example where these will be useful is when ripping a CD to a single .wav file with foobar2000. Upcoming fre:ac releases will be able to list the individual tracks contained in the .wav file even if no separate .cue file is available.

Improved error handling

Despite fre:ac 1.1 alpha releases having supported parallel conversions for some time, error handling is still designed for single threaded operation. When a conversion fails, the affected thread shows an error message and the whole conversion process stops at that point.

This will be much improved in the next release. An error in one conversion will not stop the whole process any longer. Instead, error messages will be collected and displayed all in one dialog at the end of the conversion process.

In other words, when the 100th file in a 1000 files conversion job causes an error, fre:ac will continue to process the other 900 files instead of stopping the whole job. No more worrying that large unattended conversion jobs stop early without you noticing.

HiDPI improvements

Last but not least, I've also implemented some improvements for the placement of tool windows and dialogs in HiDPI mode. Dropdown lists, popup menus, tooltips and dialogs should now be displayed where you would expect them when running in scaled mode.

This concludes this month's issue. Be sure to come back in a month for the next report.