Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I would like to make 2 feature requests with regards to the lastfm integration. Currently in these cases lastfm plays are stuck on 0 in my library.

The first one is for custom field mappings. For example, I use '$if2([%conductor%],[%artist%])' as artist field in foo_scrobble. Obviously the artist that foo_enhanced_playcount retrieves from lastfm does not match up anymore with the artist tag in the song. Ideally your component copies the field mappings from foo_scrobble or at least allows to set them manually.

Second request is to take into account lastfm autocorrected tags. There are many cases where lastfm incorrectly autocorrects to a different title/artist, when foo_enhanced_playcount retrieves those scrobbles back they do not match up with the tags in my library and thus remain on a playcount of 0.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I really like the first idea. I'll take a look at implementing that in the near future.

As for the 2nd, I don't think there's a way to know what the originally played artist/title was to track them back. If you can give me an example from your own getRecentTracks call that you can point me to of a track that was corrected I can verify this. I've just looked through about 2000 tracks looking for something that was designated as autocorrected and couldn't find anything.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

v4.1.1 - Option to defeat auto pull in configuration - Autopull pulls 1000 songs at a time again - Config shows when the next historical pull will start from, and allows you to restart from most recent scrobble - It was possible in some weird confluence of events for you be to pulling historical scrobbles from 2019 and the next pull could be years previously (despite thousands of songs in between). This should be impossible now - Once we reach the end of the scrobble list, start over from most recent date - Better prevention of querying last.fm for the same song twice - Prevent unnecessary last.fm queries for songs that were pulled back in versions 1.5 to 2.0.2

This is a soft release and if nobody reports any issues, I'll update it in the official registry.

You can see the track parameter in the query is "Guns & Synths", which is the title in my library. However the query returns 3 scrobbles for "Guns & Synths (Prod. Bok Bok)". These scrobbles are not recognized and applied atm.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

v4.1.2 is up in the component registry. Fixes two issues: - Songs with an apostrophe in the album or title were not having scrobbles automatically checked - If you started up foobar and there were no songs scrobbled since the last check, the historical scrobble check would reset

You can see the track parameter in the query is "Guns & Synths", which is the title in my library. However the query returns 3 scrobbles for "Guns & Synths (Prod. Bok Bok)". These scrobbles are not recognized and applied atm.

I think the solution here is to stop checking to see if the song title matches. This was required when we had to do artist queries, but now that the plugin does getTrackScrobbles instead it *should* be okay to skip this comparison. i'm gonna do a little more testing and then roll this out if I can't find any conflicts.

In this case I changed "+" with the uri encoded version "%2B". I'm not aware of any more cases like this with other characters, but it may be worth investigating.

Thanks!

Fixed in 4.1.3 which is available now.

BTW: while doing sanity checks of other "special" characters, I found 3 songs that last.fm refuses to allow to be scrobbled. Check out Song #7 here: https://www.last.fm/music/Psychostick/DoI thought things were broken on my end until I checked the # of scrobbles on that page. Pretty sure they're ignoring them as they think that whatever last.fm client being used is sending bogus data (i.e. a filename).

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

The first one is for custom field mappings. For example, I use '$if2([%conductor%],[%artist%])' as artist field in foo_scrobble. Obviously the artist that foo_enhanced_playcount retrieves from lastfm does not match up anymore with the artist tag in the song. Ideally your component copies the field mappings from foo_scrobble or at least allows to set them manually.

I'd prefer the at least allows to set them manually part to have maximum freedom :-)

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

BTW: while doing sanity checks of other "special" characters, I found 3 songs that last.fm refuses to allow to be scrobbled. Check out Song #7 here: https://www.last.fm/music/Psychostick/DoI thought things were broken on my end until I checked the # of scrobbles on that page. Pretty sure they're ignoring them as they think that whatever last.fm client being used is sending bogus data (i.e. a filename).

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

@fuffi That's a very interesting issue. Is there any possibility that none of the 1000 songs in your 5 getRecentTracks calls still exist in your library?

Here's the way the logic flow works:* we query for the first 1000 songs based on the last checked timestamp.* We take that list of 1000 songs and remove duplicates* We iterate through that list and search for each song in our library and retrieve the record of last.fm values for it* If we don't have a record, or the timestamp from the play in last.fm is after the last recorded timestamp, we add that song to a new list of songs we need to pull scrobbles for laterHOWEVER, if we do have a record of last.fm scrobbles and the latest scrobble is equal or newer than the scrobble in the list that last.fm gave us, then we update the most recent timestamp value to the one that last.fm reported for us.

The reason for this it's much safer in case last.fm goes down. We won't mark a time stamp as checked unless we know it's already covered. It's still possible to miss scrobbles this way, but it's much more fault tolerant. Now, if none of the 1000 scrobbles that were pulled existed in your library then it would just bomb out on step 3.

If that's not the case (and it seems unlikely) then something else kind of insidious seems to be happening. I'd probably have to provide you with a debug version of the component to debug it further.

Please, if you provide a version with detailed debug information, make it save those infos to disk and not only log to console. I have too much other components writing to the console and its hard to separate the important lines from the useless ones ;-}

The last.fm screenshot contains music by daft punk, which was played simultaneously from iTunes, you can ignore it. All other songs are played via f2k like the playlist I screenshotted in my post above.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

Do you ever plan to implement manual importing (and exporting to xml too) of statistics?

I hadn't considered it before. If I were to ever do something like this, I'd probably export to JSON because it's easier to work with IMO.

Ten months later, have you reconsidered maybe?

It would be so much more convenient if I could have control over my own listening statistics. Which I currently can't have either with regular statistics (first/last play only), last.fm (third party & commercial, extremely limited editing/correcting), listenbrainz (still in early beta, sparse functionality) or your plugin (no manual mass import).

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I actually really like the idea, but I'm not 100% sure how it could work. Are you looking for just changing playcounts, or also timestamps as well? Does it need to work with another tool/export format? Do you want to just change a few songs? Is this for backups or mass editing? Also the exports would be pretty huge for any reasonably large play history (100k plays).

Unfortunately I just haven't had a lot of time or motivation to work on this recently, and have been focusing on some other side projects. Also this isn't really a feature I personally need which means it's not top of my priority list.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I know there's the last.fm import option, but it doesn't really do much for me as my early years on last.fm are a mess. A couple of years ago I downloaded all my last.fm data and with some help of excel I managed to weed out outliers and finally adopt it for original playback statistics component. But it took time and manual tinkering and I'm 100% sure no automation can manage to do that without my own input. Also, you know, maybe one day sites like YouTube or Soundcloud will allow downloading user history through API - another case where manual import would prove useful.

Basically even importing from file tags would do. It doesn't have to be an xml importer like in foo_playcount. After all I can make a dummy copy of my library (3% length 6kbps opus previews) and copy the data to properly formatted foo_enhanced_playcount file tags (%played_times%, %played_times_js% or %played_times_raw%, doesn't matter). It's just the last step to get this data into foo_enhanced_playcount db that's missing.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I am encountering cases where the value of %lastfm_play_count% are higher than what is returned by the lastfm query. Even after clearing saved plays and retrieving them again. It seems to happen when I pause playback mid-track and continue after a relatively long time (10+ mins). I can try to find more specific steps if you can't reproduce it.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

I am encountering cases where the value of %lastfm_play_count% are higher than what is returned by the lastfm query. Even after clearing saved plays and retrieving them again. It seems to happen when I pause playback mid-track and continue after a relatively long time (10+ mins). I can try to find more specific steps if you can't reproduce it.

I'm very curious if you can come up with more solid steps to reproduce this.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

Testing out version 4.2.0 now. Please give it a try and let me know if you encounter any issues.

Changes:

Field mappings are finally present. You can supply Artist, Album, and Title titleformatting strings. I don't know why you'd ever change anything other than Artist, but you can!

Discovered a bug in how I was returning lastfm and playtimes values in "_JS" format. Essentially I was converting timestamps to locally display the UTC timestamp, so all values had the timezone offset built in. This was dumb and has been fixed. If you're using my Georgia theme, the timeline will look off until the latest version has been posted.

Re: foo_enhanced_playcount - Record all song plays and Last.fm scrobbles

Field mappings are finally present. You can supply Artist, Album, and Title titleformatting strings. I don't know why you'd ever change anything other than Artist, but you can!

Just to give a valid reason:

I used to store remix or featuring information in other tags and stripped them from the artists and/or track tags to have a working "search for similar" (foo_quicksearch) option.

searching for similar track-titles only would work, if the title is exactly the same, but a title "good sound" and "good sound feat. dj bobo" or "good sound (dj bobo remix)" wouldn't not be similar, so I stripped "feat. dj bobo" or "(dj bobo remix)" from the tags and stored them elsewhere.

The drawback is, when scrobbling via last.fm component, only the stripped tags were sent to last.fm, so, for last.fm, I never listened to a remix of "good sound", but only to "good sound". This is especially bad, if you listen to a lot of electronic music, where there are a lot of remixes of a single track.

Glady, the last.fm component has a field mapping, like this wonderful component has now too! ;-D