Tuesday, 15 December 2015

Finding the mood of a song

Playlists let us group songs in a way that is meaningful to use instead of the traditional approach of listening to songs album by album as directed by the artist. So many of us have large music collections these days that playlists are
becoming increasingly more important way of categorising our music.

One way of categorising music is by considering the ambience or mood of the song. Clearly you would aim for a different ambience for a romantic evening at home then for a house party!

Manual Mood Identification

The original way to do this would be look at your songs and manually select them based on their mood. But its difficult to do this for most songs without actually listening to them, and most of us don't have time for this anymore.

Automatic Mood Identification

Its now possible to use computers to listen to the songs and extract various acoustic attributes, then by plotting multiple attributes against each other a mood can be worked out.One way of defined mood is by considering two acoustic attributes: Energy and Valence.

Energy

Represents a measure of intensity and powerful
activity released throughout the track. Typical energetic tracks feel
fast, loud, and noisy. For example, death metal has high energy, while a
Bach prelude scores low on the scale.

Combining these two on a graph is a strong indicator
of acoustic mood,for example a song with high arousal and high valence could be defined as delighted, whereas one with high arousal and low valence could be defined as angry. Low arousal and low valence may be bored, but low arousal and and higher valence could be content. This methodology is used by companies such as Echonest for the Spotify site.

SongKongs way of doing things

SongKong Pro also uses the Energy/Valence algorithm, built on top of the data provided by AcousticBrainz , these are the same guys who already provide BPM data. When a MusicBrainz song is identified by SongKong we can then lookup the acoustic attributes and calculate the mood. The mood is then stored in the standard mood metadata field so it is available to other applications.

Because the acoustic attributes have already been calculated for millions of songs your computer doesn't have to actually analyse the song itself. This computationally hard work has already been done

Because the AcousticBrainz is already stored in the JThinkMusicServer we can get this information at the same as we get everything else,
meaning no additional time is spent getting these new acoustic
attributes.

We also add some other acoustic attributes that we will talk about more in another post

Friday, 20 November 2015

Lightroom versus Photoshop

I have used Adobe Photoshop on and off for many years for photo editing. It is an incredibly powerful tool for photo-editing that offers limitless opportunities, but that is part of its problem. You can easily spend an afternoon working on a single photograph to get it perfect. And because I did not use it regularly often I did not use it in the most efficient cleverest way.

I also used Apple Aperture for managing my photographs but found it awkward to use not least because of the opaque library it uses in a similar way to how iTunes manages songs. But then I discovered Adobe Lightroom - at first it replaced Aperture offering a much clearer way of managing my photographs and making simple adjustments such as contrast and exposure and white balance. But then as I delved into it I found almost everything I used Photoshop for could be done in Lightroom , and it was more obvious what to do and the results were often better.

One particular aspect of Lightroom that I love is that instead of having to use layers to ensure I dont modify the original photo I do not even have to think about it, Lightroom never modifies the original photograph instead all changes are written to its catalog and can be undone.

And I never have to think about working colour profiles either, I only have to worry about colour profiles when I export the photograph for use in another application or print.

I still use Photoshop because it can do things that Lightroom cannot, but most of the time I just Lightroom.

Secondly, because the analogy between Lightoom and Photoshop is very similar to how I feel about SongKong and Jaikoz.

SongKong versus Jaikoz

Jaikoz development started in 2006 which coincidentally seems to be when Lightroom development began. But in this story Jaikoz is more analogous to Photoshop, from the start it has added manual editing and automatic matching . There are more than one way way to do some tasks and I have also been receptive to user enhancements requests to try and solve every possible problem. I think Jaikoz offers more comprehensive tagging than any other music tagger on the market.

But this complexity comes at the cost, although Jaikoz can be used simply this is not immediately apparent to users

SongKong is only two years old, the aim was to provide powerful automated matching with a clear workflow for the user. SongKong does not offer manual editing and never will, there are now many tools to do that. SongKong does not try to emulate all of the Jaikoz automated matching functionality just the most useful stuff.

For example MP3s support three versions of the ID3 tagging standard v22, v23 and v24, and Jaikoz supports saving with all three versions. But nobody uses v22 anymore, everything supports v23 and many applications now support v23. So SongKong just supports saving v23 and v24 we have lost a tiny bit of functionality for more simplicity, nobody has complained about this so far.

Now lets try to stretch the analogy to Photoshop layers . Unlike many other tag editor tools Jaikoz doesn't save changes as they happen instead everything is done in memory and can be reviewed before the modifications are saved. This is much safer but it does put the onus on the user to check the modifications, and can mean a delay if many file need to be saved at the end of an editing session.

With SongKong I wanted to remove this burden from the user, but just having SongKong making changes without the user having an opportunity to reverse the changes was dangerous. So instead all modifications are written to SongKongs database similar to the way edits are written to Lightrooms catalog. Unlike Lightroom the files are modified as well but because the changes are written to the database they can be undone at a later date, no problem.

These days my first point of call for cataloging my music is SongKong, I then use Jaikoz for songs that SongKong fails to match plus for special tasks such as exporting metadata to a spreadsheet.

Summary

For my photographs I need both Lightroom and Photoshop but I get much more use out of Lightroom. For my own music I need both SongKong and Jaikoz but songs always go through SongKong first.

Wednesday, 4 November 2015

What's the best lossless format?

Most of us know that generally lossless is better than lossy because it retains all of the audio and therefore sound better, and you can also convert form one lossless format to another without losing any sound quality. Lossless does require significantly more disk space but his doesn’t mean you have to reduce the amount of music stored on your iPod,
iPhone or mobile device. For example iTunes allows users to convert higher data
rate music files to 128kbps AAC on the fly as the music is synced to the
mobile device in question.

But what Lossless format should you choose ?

Currently there are four widely available choices:

Aiff

Wav

Apple Lossless

Flac

Wav

WAV is a music file format developed by Microsoft and IBM capable of storing Linear PCM audio (the
digital encoding format used on compact discs) in completely
uncompressed form. Ripping a CD and storing it as an uncompressed WAV
results in “bit perfect” storage; the ripped music file is identical to
the original CD data package. WAV files can also store high-resolution
music files at greater bit depths and sampling rates than CD’s
16-bit/44.1kHz resolution. Uncompressed WAV files can be ripped and
played back in iTunes and are very high quality. However, they do take
up more hard drive storage space then lossless compressed formats such as Flac or Apple Lossless

WAV
files originally did not support metadata, there is now defacto support using ID3v2 - the same format that is used by Mp3s. However not all players and tools currently recognize ID3v2 metadata in Wavs.

Aiff

AIFF was developed by Apple and is very similar to WAV. Again it is capable of storing
uncompressed Linear PCM audio. Ripping a CD and storing it as
uncompressed AIFF results in “bit perfect” storage with the ripped music
file identical to the original data on the CD. Like WAV files, AIFF
files can also store high-resolution music files at high bit depths and
sampling rates.

AIFF uses ID3v2 like Wavs and Mp3s. ID3v2 metadata is within Aiff is much more widely recognized - notably it is properly supported in iTunes.

Apple Lossless

This is a newer Apple file format option that employs lossless compression, which reduces the stored
data to as little as half of the original music file’s size but restores
bit-for-bit identical to the original music file on playback. The
process is not unlike a zip file in which a large amount of data is
“zipped” down to a smaller file size for storage and “unzipped” to its
full size when opened.

There is discussion about whether uncompressed music files such as WAV or AIFF can
sound better than lossless compression formats like Apple Lossless or
FLAC because they don’t require the additional step
of being “unzipped” and restored to their original PCM data
package during real-time during playback, but this difference should only be noticeable in older and slower hardware.

Apple Lossless
offers full metadata support using its own metadata format, this metadata format is also used by Apples lossy Advanced Audio Coding format. Both formats can be found using the .mp4 or or .m4a format which can be confusing

FLAC (Free Lossless Audio Codec)

FLAC is an open source lossless compression format similar to Apple Lossless Compression, this
reduces the stored music file’s size, but then restores the data package
bit-for-bit identical to the original music file on playback. It
supports high-resolution audio with greater bit depths and sample rates
and also supports metadata tagging using Vorbis Comments.

The main problem with FLAC is that although it is an extremely
common and accepted format, it is not supported by iTunes and Apple hardware such as the iPod.

Conclusions

I would love to recommend the open source FLAC format but because of its poor support in the Apple ecosphere I don't think it is currently the best choice, particularly since Apple punches above its weight when it comes to digital music.

Although Wav audio format is well supported, support for its metadata is poor so again I would not recommend Wav.

So we are left with two Apple formats. With Apple Lossless we get a smaller filesize and good metadata support, with Aiff we get a larger filesize but even better metadata. Both are well supported on Apple and non Apple hardware and software. The ID3v2 format used by Aiff format has been around a long time (because used by Mp3 as well) and although it is a little unwieldy it is well understood and very powerful, the metadata format used by Apple Lossless is not as well understood or flexible.

Everybody circumstances differ but for me the current best choice is Aiff

Aiff and Wav Support

Aiff and Wav formats each provide an uncompressed lossless format for your audio and were created in 1988 and 1991. However
until more recently there was no agreed way to store anything but the bare
minimum of metadata, this is why we originally didn't add support within Jaikoz.
The other reason was these formats were expected to die out and be
replaced with newer formats but although they are less prevalent then
before this has not happened.

However we revisited these formats and discovered that defacto support for Aiff tagging and Wav tagging now existed using an ID3v2 chunk, just like MP3 files. Because they use ID3v2 they can now store all the metadata that Jaikoz finds such as artwork, composers, bpm and MusicBrainz and Discogs
identifiers - not just the basic metadata such as artist and album.

This also means you can edit your Aiff and Wav metadata not just with the Edit tab but also using the ID3 Edit tab. The ID3 Edit tab shows hows exactly how the fields are stored and also gives access to any additonal ID3 fields that are not mapped to a generic field in the Edit tab.

You can also decide on the ID3v2 format to use, by default they will use the value of Preferences:Save:ID3Tag V2:Write V2 Tag Version

But you can override this for any individual file by selecting a different value from theVersion column.

Single Song Matching

We also make some improvements to single song matching. Single song matching occurs when a song has been not been grouped with any others songs or no match was found for the group of songs and user preferences allow Jaikoz to try and match each song individually. Because only one song is being matched any album containing the song is a potential good match but we have made some changes to find a match for an original album rather than a compilation whenever possible.

Full list of fixes is as follows:

Bug

[JAIKOZ-1070] - Should not bother trying to do release metadata match unless have artist and release

[JAIKOZ-1079] - Saving Acoustids gets confused if filename has been modified but not saved

Tuesday, 13 October 2015

SongKong now supports reading and writing metadata to Aiff and Wav formats !

AIFF and WAV formats each provide an uncompressed lossless format for your audio, whereas Flac and Apple Lossless provide a compressed lossless format. Both types can encode your audio without losing any of the original data. An uncompressed format does not need to be uncompressed during playback, which can potentially result in smoother playback but at the expense of larger file sizes, however with most hardware it is unlikely you could detect any difference in playback . To confuse matters a little it is now possible to store compressed lossless in Aiff and Wav files as well - but this is rarer and not part of the original specification.

Audio Interchange File Format (AIFF) was developed by Apple Inc. in 1988 and is most commonly used on Apple Mac systems. Waveform Audio File Format (WAV) is a Microsoft and IBM audio file format standard for storing an audio bitstream on PCs developed in 1991. Both are version of the Interchange File Format (IFF) format developed by Electronic Arts back in 1985.

All formats provide a way of storing data in chunks - however until recently there was no agreed way to store anything but the bare minimum of metadata, this is why we never added support within Jaikoz. The other reason was these formats were expected to die out and be replaced with newer formats but although they are less prevalent then before this has not happened.

However we revisited this and discovered that defacto support for aiff tagging and wav tagging now existed using an ID3v2 chunk, just like MP3 files. Because they use ID3v2 they can now store all the metadata that SongKong finds such as artwork, composers, bpm and MusicBrainz and Discogs identifiers - not just the basic metadata such as artist and album. Wavs can store metadata in an ID3v2 chunk and also some basic metadata in an INFO chunk: for maximum compatibility with older applications SongKong adds both an ID3v2 and a INFO chunk to files when they are saved.

After a period of development and testing support has now been added to SongKong 3.18 and this is available from today, plus as usual we have some matching improvements and some additional fixes.

Support for these new formats will be added to Jaikoz within a few weeks.

But there was still some songs that were in the MusicBrainz database and a few songs that were being misidentified. It got me thinking is it possible correctly identify every single song if its in the database, and not misidentify any.

It is a difficult tightrope to walk trying to identify more songs where there is not a clear answer, ensuring we do not misidentify songs,
because this is worse than not identifying the song at all.

It has taken a few iterations but now with Jaikoz 8.3.3 for these particular tests we now have 100% correct matches and no mismatches for a series of essentially random folders.

I realize not all our customers will have such success but I think
most of you will be very happy with the improvements we have made, now I need to incorporate some of these improvements into the next version of SongKong.

The improvements we made in this release are as follows:

[JAIKOZ-1043] - isGoodArtistAndTitleMatch failing artist which just has additional 'the' in name

[JAIKOZ-1044] - Single song match doesnt do release match if metadata has song title

Monday, 29 June 2015

Jaikoz 8.2.4 released June 29th 2015

Jaikoz relaxes its matching criteria if it detects a folder that contains songs not limited to one album and artist. Whilst you may have some of your songs organized one folder per album its not uncommon to have folders containing songs from all kind of sources in no discernible order, fixing these in Jaikoz is why you bought it in the first place, right ?

If Jaikoz identifies such a folder it relaxes its matching criteria so even if Only Match all songs to one album if all songs in grouping were matched is enabled it is ignored allowing these songs to be matched without breaking up folders that have already been organized

But there was a bug preventing it properly matching songs in such a folder even if the option was disabled. Now with this fix it does its matching correctly for these types of folders, even if the option is enabled. So you should now get much better results when trying to match such folders.

iTunes integration

Error Handling

Usually SongKong can be run on any number of files without memory issues, but badly behaved images can cause a memory issue. SongKong was not recovering from this error and if the memory issue did reoccur when loading songs SongKong was not reporting the error correctly, this is now resolved.

Report Improvements

We have added a new option for reports. This option can be used to prevent the report being opened automatically, the report is still created and can be opened immediately after processing from the summary panel or from the Reports menu.

Within MusicBrainz it is possible to link an artist to the equivalent Discogs artist, in fact there are already some 300,000 artists linked. However there are nearly a million artists within MusicBrainz leaving nearly 700,000 artists unlinked, the majority of these already existing in Discogs.

Linking both artists together allow information between the two databases to be shared, for example the Discogs artist may contain the real name of the artist and that might be missing from the equivalent MusicBrainz artist aliases. Alternatively MusicBrainz may have various translations of an artists names for different languages, and these variations may not be available in Discogs.

Albunack now contains a number of reports that attempt to find potential matches between the two together with a link for each match to make it easy to add a link from MusicBrainz artist to the Discogs artist, all listed here

Simplified MusicBrainz artist is unique within MusicBrainz list of artists.

Smplified Discogs artist name is unique within Discogs link of artists.

Neither name is already matched

Both the MB artist and the Discogs artist have released an album with the same name

Because of these conditions we are confident there are very few (if any at all) bad links.

The simplified artist names remove case, accents and some punctuation in order to allow matches between artist names that are not identical. Because we only include artists whose simplified name is unique within the databases it is unlikely that the MusicBrainz artist has linked to the wrong Discogs artist. And this is made very unlikely with the last condition that requires both artists to have a release with the same name.

Some of the links may already have already been created because the underlying database is not updated in real time, but it is regularly updated - usually every week.

Links are listed so that artists with the most releases are listed first, the basis being that these are likely to be of most interest.

But with 40,000 artists to link the job to complete this task could take some time, so if you have 10 minutes please consider double-checking and then creating some of these links within MusicBrainz.

Friday, 12 June 2015

Linking a Discogs release to a MusicBrainz Release

In the past
there have been two significant problems with trying to link Discogs releases to
MusicBrainz release.

Finding a release

Firstly, it was difficult to Discogs release that
can be linked to MusicBrainz but aren't already. You could browse an artist releases on Discogs but there is no indication on the Discogs site as to which ones are already linked to a MusicBrainz release. You can check whether an individual release on MusicBrainz is linked to the Discogs but you cannot filter a list of releases by whether or not they are already linked.

And when popular artists can have hundreds of Discogs releases linked to a single master, and even moderately popular artists can often have more than thirty it is difficult to find the correct release to link to if any. Working out the best match is difficult when having to consider and compare differences and matches in metadata such as track lists, catalogue nos, barcodes, country of release and record labels.

Linking the Release

Secondly you have to
actually add the link.

Linking the release is not so difficult but when the Discogs release contains extra information that the MusicBrainz release does not have, wouldn't it be good to get that data in as well, but adding that extra data for more than a few releases at a time is time-consuming.

Linking a Discogs release to a MusicBrainz Release with Albunack

Today I am going to show you how to
easily link Discogs releases to MusicBrainz releases with Albunacks artist-centric approach solves both
of these problem

The JThink Music Server is a dedicated server that contains MusicBrainz and Discogs data, and is what Jaikoz and SongKong use for their automatic matching. Albunack is the public face of the music server and can be used to browse combined MusicBrainz/Discogs discographies for almost any artist.
If SongKong is not finding a match when you think it should, it is probably easier to check on Albunack instead of MusicBrainz or Discogs. Although note Albunack doesn't currently show collaborations or Various Artist releases (although they are in the JThink Music Server).

Two months ago we released Albunack and now we have now added a Youtube channel , and this includes an introduction to Albunack

this gives a nice overview on how to navigate around an Albunack discography.

Thursday, 4 June 2015

SongKong 3.15 Released

Firstly this includes modifications required to work with cover art newly added to Discogs in the last two months, this is required due to a change in how Discogs provides their cover art data.

Then we have fixed a problem matching song title and track total when matching multi-disc Discogs releases.

We recommend you update to this latest version of SongKong and then run Fix Songs over your collection with the For Songs Already Matched to MusicBrainz releases option set to Update Metadata and Filename Only. This will fix any data that may have been incorrectly added in earlier releases
and will also updated your previously matched songs the very latest data added to MusicBrainz and
Discogs.

New version of JThink Music Server

Yesterday we released a new version of the JThink Music Server - this is what powers Jaikoz, SongKong and Albunack . This latest release fixes an issue with incorrect values for the Sort Artist field. It also add supports for the latest Discogs cover art, required because the way cover art has been provided by Discogs for releases added over the last couple of months has changed.

New version of Jaikoz Music Tagger 8.2.3

Today we release a new version of Jaikoz.

Firstly this includes modifications required to work with cover art added to Discogs in the last two months.

Then we have fixed two problems related to song titles and track total when matching multi-disc Discogs releases.

Then we have fixed a regression whereby the Only Update Year if Earlier option for the Year field was ignored so this field could be overwritten with a later date.

We recommend you update to this latest version of Jaikoz and then run Update Metadata from MusicBrainz and Update Metadata from Discogs to fix any data that may have been incorrectly added in earlier releases and to benefit from the very latest data added to MusicBrainz and Discogs.

Only Allow Match if All Songs In Grouping Match to One Album is defaulting to false on new install it should default to true otherwise existing songs grouped by album could be split if Jaikoz can match the songs but not all to the same album.

Tuesday, 28 April 2015

MusicBrainz is the defacto standard for Music, MusicBrainz Ids are used by many of the major players in online Music services such as Google the BBC and Spotify, and the MusicBrainz schema is a thing of beauty. But it has always been difficult getting the latest releases into MusicBrainz because there are no automated feeds from record companies and it mainly relies on the hard work of the many contributors.

Discogs is another online database, its not as well designed as MusicBrainz and its codebase is not open, but it has more artists and more releases and does make its database available monthly under a public domain license. Discogs has a vibrant marketplace, but to sell a record it has to be in the database, this is a good reason for users to contribute releases, and because buyers need to know exactly what they are purchasing sellers need to enter the details accurately.

For years I have wanted to something to make it easier for editors to get data into MusicBrainz from Discogs in some form or other but never seemed to have the time, but over the last month I have been working on a solution and now I have beta available as:
http://albunack.net

The idea is you look up an artist and it shows you a combined MusicBrainz + Discogs discography, showing releasegroups/release and tracks on a single page.

Here is an example for the Pale Saints, a greatly under appreciated UK band from the early 90's

This makes it easier to see gaps in the MusicBrainz which could be filled by data from Discogs, and makes it easier to add that data:

It shows any existing links between MusicBrainz and Discogs matches.

It also matches up Musicbrainz releases without a Discogs link to potential Discogs releases and lists these matches together with a Link button to seed a MusicBrainz Edit release.

It provides an Import button to seed a MusicBrainz Add Release from other Discogs releases not linked to Musicbrainz

I have been testing it myself over the last few weeks and it is so much quicker and so much safer then previous ways I have used of adding data. Ive also found it is very good at showing when there is incorrect data in either MusicBrainz or Discogs.

You can also use it to check automated tagging done by SongKong or Jaikoz since they use the same underlying database.

Currently it only shows releases where the artist is the only album artist credit (i.e we don't list collaborations, nor Various Artist releases)
Of course, the user still has to make decisions about the data and check the data before submitting anything to MusicBrainz, and everything still goes through the edit system.
You can use the bug tracker for raising any issues.