06 February, 2005, 12:15:40 PM

Hotness is an algorithm that generates a score on a scale of 100 for every song in your library. It weighs frequency of play, rating, and recentness of last play against each other to determine how "hot" a song is. Because of its complex nature, its use of an absolute scale, and its awareness of time, it offers a unique alternative to other approaches to meta-rating (such as playcount × rating).

[!--sizeo:3--][span style=\"font-size:12pt;line-height:100%\"][!--/sizeo--]Update: 18Aug2007 1.7.c[/size]Lines 14-18 allow for easy alteration of which playback statistics tags to use. Default baselines have been greatly increased too, just because I find they work best for me. As always, experiment with them.

Update 18Feb05-"begin decay immediately" is now optional-uses new LAST_PLAYED standard-if an %added% tag is absent, it assumes the baseline frequency for the song-configurable "default rating" (assumes default rating when there is none)-plan to employ FIRST_PLAYED standard once it's implemented

Code updated 10Feb05-decay now begins immediately rather than the day after last play

Before using this code, remember that three tags are NECESSARY for it to work: %first_played%, %last_played%, and %play_counter%, provided by foo_playcount 1.9.2

%rating% will also affect things, but isn't essential.

So I had this idea that there are a number of statistics contained within each one of my tracks that are related to the overall "hotness" (or "popularity" or what have you) of a song: %play_counter%, %rating%, %last_played%, and %first_played%. But no algorithm had been designed yet that would weigh all of those factors in the calculation of one single score that measures what I'm calling "hotness."

Rating is all fine and dandy, but it's so cut and dry. Same goes for last_played and play_counter. They're such dead, linear statistics that aren't very interesting to me to watch at all. And play_counter is rendered meaningless unless it's compared to how long a song's been in your library. Complexity was the goal of this idea, the dynamic generation of a song's overall...SOMETHINGness that I couldn't easily predict. It does no good to tell me that I rated a song 5 stars. I know I did, I was there. And keep in mind, this isn't about determining a song's quality or "goodness"; goodness is given by rating. Hotness is something much more interesting, and I hope I'm on my way to a satisfactory representation of it.

To begin, I believe the importance of these factors in decreasing order is:-recentness of last play-overall frequency of play (play_counter ÷ days since added ("age"))-rating

When a song is played, its hotness should be given a boost (the amount of this boost is detailed below). As time passes since the last time a song was played, that song's hotness should decrease. So now we've got to define a decay period: the time it takes before a song's hotness dwindles to 0 again. The two factors that determine the length of this decay period are rating and play frequency.

The definition of this decay period depends on two lengths of time: baseline FREQUENCY and baseline DECAY. In the code below, baselinefrequency is set at 14 days and baselinedecay at 6 days. But, depending on your listening habits, you may need to alter these settings. The effect the settings have isn't immediately intuitive, but a good rule of thumb is:

-the more often you listen to music, the LOWER baselinedecay should be-the more variety of music you listen to, the HIGHER baselinefrequency should be

Setting baselinefrequency to 14 and baselinedecay to 6 means that a song played on average once every 14 days will decay to 0% hotness 6 days after it's played. If a song is played TWICE every 14 days, its decay period will DOUBLE (12 days). If a song is played once every 28 days, its decay period will halve (3 days).

Likewise, rating will affect a song's decay period. The absence of a rating will assume a default rating of 3. A rating of 5 will multiply the decay period by 5/3, and a rating of 1 will multiply the decay period by 1/5.

The initial boost of playing a song used to be 100% hotness. This made for boring playlists when looking at songs that were played "today," so in the current code, the immediate hotness is what the hotness would be TOMORROW if the hotness were to go up to 100% today. This makes for some interesting effects; for instance, my settings make it so that a song added today and played once today begins at a hotness of 83%. If I play it again today, I then double its play frequency, thus doubling its decay period, thus boosting its hotness.

// calculate "age": the number of days the song has been in the library$if(%added%,$puts(age,$sub($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$ifequal($mod(%_system_year%,4),0,$ifgreater(%_system_month%,2,1,0),0),%_system_day%),$add($mul($substr(%added%,3,4),365),$select($substr(%added%,5,6),0,31,59,90,120,151,181,212,243,273,304,334),$ifequal($mod($substr(%added%,1,4),4),0,$ifgreater($substr(%added%,5,6),2,1,0),0),$right(%added%,2)))),$puts(age,$mul(%play_counter%,$get(baselinefrequency))))

// calculate "recentness": number of days since song was last played$puts(recentness,$sub($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$ifequal($mod(%_system_year%,4),0,$ifgreater(%_system_month%,2,1,0),0),%_system_day%),$add($mul($substr(%last_played%,3,4),365),$select($substr(%last_played%,6,7),0,31,59,90,120,151,181,212,243,273,304,334),$ifequal($mod($substr(%last_played%,1,4),4),0,$ifgreater($substr(%last_played%,6,7),2,1,0),0),$substr(%last_played%,9,10))))

// calculate "age": the number of days the song has been in the library$if(%added%,$puts(age,$sub($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$add($div($right(%_system_year%,2),4),$if($or($greater(%_system_month%,2),$greater($mod(%_system_year%,4),0)),1,0)),%_system_day%),$add($mul($substr(%added%,3,4),365),$select($substr(%added%,5,6),0,31,59,90,120,151,181,212,243,273,304,334),$add($div($substr(%added%,3,4),4),$if($or($greater($substr(%added%,5,6),2),$greater($mod($substr(%added%,1,4),4),0)),1,0)),$right(%added%,2)))),$puts(age,$mul(%play_counter%,$get(baselinefrequency))))

// calculate "recentness": number of days since song was last played$puts(recentness,$sub($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$add($div($right(%_system_year%,2),4),$if($or($greater(%_system_month%,2),$greater($mod(%_system_year%,4),0)),1,0)),%_system_day%),$add($mul($substr(%play_date%,5,6),365),$select($substr(%play_date%,3,4),0,31,59,90,120,151,181,212,243,273,304,334),$add($div($substr(%play_date%,5,6),4),$if($or($greater($substr(%play_date%,3,4),2),$greater($mod($substr(%play_date%,5,6),4),0)),1,0)),$substr(%play_date%,1,2))))

// calculate "age": the number of hours the song has been in the library$if(%added%,$puts(age,$sub($add($mul($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$add($div($right(%_system_year%,2),4),$if($or($greater(%_system_month%,2),$greater($mod(%_system_year%,4),0)),1,0)),%_system_day%),24),%_system_hour%),$mul($add($mul($substr(%added%,3,4),365),$select($substr(%added%,5,6),0,31,59,90,120,151,181,212,243,273,304,334),$add($div($substr(%added%,3,4),4),$if($or($greater($substr(%added%,5,6),2),$greater($mod($substr(%added%,1,4),4),0)),1,0)),$right(%added%,2)),24))),$puts(age,$mul(%play_counter%,$get(baselinefrequency))))

// calculate "recentness": number of hours since song was last played$puts(recentness,$sub($add($mul($add($mul($right(%_system_year%,2),365),$select(%_system_month%,0,31,59,90,120,151,181,212,243,273,304,334),$add($div($right(%_system_year%,2),4),$if($or($greater(%_system_month%,2),$greater($mod(%_system_year%,4),0)),1,0)),%_system_day%),24),%_system_hour%),$add($mul($add($mul($substr(%last_played%,3,4),365),$select($substr(%last_played%,6,7),0,31,59,90,120,151,181,212,243,273,304,334),$add($div($substr(%last_played%,3,4),4),$if($or($greater($substr(%last_played%,6,7),2),$greater($mod($substr(%last_played%,1,4),4),0)),1,0)),$substr(%last_played%,9,10)),24),$substr(%last_played%,12,13))))

Paste into the "Variables" tab under the "Globals" tab in Columns UI settings, and make sure that "Make date info available" and "Use global variables for display" are both checked. Now you are free to use %_hotness% anywhere in your columns! It will appear on a scale of 100. Because this statistic is so effluvial, I think it will be best depicted in color, perhaps as a shade of red on a dot. To do this, create a column with ● as the display, check "Use custom colour spec," and in the "Colour" tab, type:

I read through rather quickly, but the topic is interesting. I personally love the extended playlist generator, and I've been using a simple weighted sorting for my generated charts, for some time now. The reason I started doing this, was as you mentioned, that rating and play count alone, isn't always that interesting. Some songs are really good, but at the same time of the kind I don't listen to that often (e.g. Eric Clapton). Hence, I don't want them to be at the top of my charts. So, what I did was pretty simple, but it still worked out pretty well for my usage and current play counts.The sorting code is this:

For short: Every rating point, counts as two play count points when these two tags are added. If two tracks gets the same total, they are sorted by rating. The ones that are still in the same group are randomized to avoid alphabetic influence.

To me it's really important that such a scheme can be used and works well, with the playlist generator. That makes it easy to update and to limit the number of entries. I also like to keep my column headers disabled, so to me, using them for sorting isn't really an option.

But, I would find a more sophisticated solution than the one I use, interesting, as long as it's still sensible. The code you (topdownjimmy) posted, looks rather slow for larger databases, but I haven't tested it yet, as it produced SYNTAX ERROR.

[Very personal opinion]I have never understood the desire to start out with a default rating of e.g. 3. At first when I started rating my tracks, I used three levels and struggled with that. Now I use five, and all of them are different degrees of positive. I use them something like this:5: Goosebumps4: Very good3: Good2: Quite good1: Stands out (from the masses)

If you need to mark some crappy tracks, add a %crap% tag or something, and use that to apply red color or skip them with foo_skip (havent' verified that that actually works). Going below average to rate for two degrees of crappyness, just doesn't make sense IMHO.[/Very personal opinion]

To me it's really important that such a scheme can be used and works well, with the playlist generator.

Yes, unfortunately my hotness scale can't be used in the epg. The epg is disappointing in that way. In my globals I define recentness to be usable by strings in the columns, but I can't use those globally defined variables in epg. Is there a reason for that? Can someone fix that?, because that would be incredible (how do you generate a playlist of songs you've listened to in the last two weeks? CAN you?)

Quote

But, I would find a more sophisticated solution than the one I use, interesting, as long as it's still sensible. The code you (topdownjimmy) posted, looks rather slow for larger databases, but I haven't tested it yet, as it produced SYNTAX ERROR.

It may be slow, I haven't noticed yet (but then I don't tend to pay attention). Not sure why you're getting a syntax error. The second code box isn't valid code, I just threw that together as an illustration, but the first one should work. I'll see if I can figure it out.

[edit]I found a stray parenthesis (the very last one). Fixed it above.[/edit]

Quote

[Very personal opinion]I have never understood the desire to start out with a default rating of e.g. 3. At first when I started rating my tracks, I used three levels and struggled with that. Now I use five, and all of them are different degrees of positive. I use them something like this:5: Goosebumps4: Very good3: Good2: Quite good1: Stands out (from the masses)

For me it makes sense because I tend to listen to full albums rather than singles or random order, so I do have (and listen to) a lot of songs that I don't like.

[edit again]I wanted to add that the reason I'm doing it this convoluted way is to take recentness into account. Under your scheme, the fact that I played a song just two days ago doesn't bump it up in the "charts." I'd like recent plays to "bump" songs. This will also cover the situation in which I listen to a song 100 times in one day, then don't listen to it for a whole year. I don't want that song on my "charts" anymore.[/edit again]

Yes, of course my simple solution doesn't come close to yours, as I can't take the time since the last play into account when weighting.

As you noted, "time/date" info isn't available "globally", and is added by each plugin individually. Not really an ideal situation, IMHO, but I don't know if there is a good reason for it. If there isn't, then I hope it could be added at some point.

No wonder your code doesn't work here, even after you fixed it, as I don't use %added% and I might also use another %play_date% format (didn't bother to check). In addition I use %JULIAN_DATE% as my second tag, so I don't have room for %added%.

I really like the idea of what you're doing, but I just don't see anyway I would be able to start using it myself (lack of epg support and no room for another foo_playcount tag).

As you noted, "time/date" info isn't available "globally", and is added by each plugin individually. Not really an ideal situation, IMHO, but I don't know if there is a good reason for it. If there isn't, then I hope it could be added at some point.

Let's hope so.

Quote

No wonder your code doesn't work here, even after you fixed it, as I don't use %added% and I might also use another %play_date% format (didn't bother to check). In addition I use %JULIAN_DATE% as my second tag, so I don't have room for %added%.

%added% is something I add manually whenever I import new music. If it was done by foo_playcount, it would update every time it was played (not what I want).

Quote

I really like the idea of what you're doing, but I just don't see anyway I would be able to start using it myself (lack of epg support and no room for another foo_playcount tag).[a href="index.php?act=findpost&pid=271385"][{POST_SNAPBACK}][/a]

Yeah, this was more of a distraction and a challenge than anything else. Right now I'm just implementing it as a colored dot, playing around with different decay settings and whatnot and seeing if the resultant hotness "feels" correct. Not an extremely useful measurement (or at least, not until playlistgen integrates with columnsui) but still fun. I'm looking forward to watching the hotness decay, since it's going to be a parabolic change, not linear (the more time passes, the more the hotness decays, the lower the play frequency gets, the shorter the decay period gets....)

%added% is something I add manually whenever I import new music. If it was done by foo_playcount, it would update every time it was played (not what I want).[a href="index.php?act=findpost&pid=271389"][{POST_SNAPBACK}][/a]

topdownjimmy, did you achieved a good result with this method? Do high rated song actually correspond with songs that are "hot" ?[a href="index.php?act=findpost&pid=271395"][{POST_SNAPBACK}][/a]

Well, I'm working on it.

Like I said, how "correct" it is is very dependent on the baseline frequency/decay settings you give. Right now I'm liking 14/14 (a play frequency of once every two weeks yields a decay period of two weeks).

I also changed the algorithm a bit, and I like my new system better. The old way that rating was incorporated into hotness was as a limiting factor, to set a "ceiling" for a song's hotness. 2-star songs could NEVER get above 40% "hot." That didn't sit right with me, I feel like if a song is played TODAY then its hotness should be 100% today. The way rating affects hotness now is by altering the length of the decay period. So, for example, if I play two songs today, and if both have the same frequency, but one is rated 1 and the other is rated 5....their hotness will be at 100% right after play, but the 1-star song will decay 5 times as fast as the 5-star one. Meaning, the 5-star song will be hot longer than the 1-star one. This makes more sense to me.

Here's my current code:

(code removed. the most recent code will always be at the top of this thread.)

If you want to know if it works just try it out. Just paste the code into your globals, add a column for %_hotness% and see how the figures come out. But remember, %play_count%, %play_date% and %added% (both YYYYMMDD) are ESSENTIAL tags for this to work. Rating is not (but it helps).

In my globals I define recentness to be usable by strings in the columns, but I can't use those globally defined variables in epg. Is there a reason for that? Can someone fix that?, because that would be incredible [a href="index.php?act=findpost&pid=271379"][{POST_SNAPBACK}][/a]

I hope no one minds that I'm "bumping" this thread. I've made a lot of changes to the algorithm, and the results are much better (I believe). Small improvements may still be made in the future. I believe that once %_system_date% tagz are implemented in playlistgen and trackinfo, this will be very useful code to have.

topdownjimmy and i have been communicating recently, basically to help me with this crazy columns_ui, and i kinda have it working. not quite how he uses it, but it gives me sweet info and everything. i like to see the numbers too, as colors can be too close to tell a difference (75 and 80 in my screenshot, for instance, are rather close together).

Before using this code, remember that three tags are NECESSARY for it to work: %added%, %play_date% (both in the form YYYYMMDD), and %play_counter%.[/EDIT]As I have %play_date% in the YYYY-MM-DD format I thought that just by changing this in your code should work...

Am I doing something wrong or is this completely impossible to use hotness with another time format? I guess that anyway, once a 'recommended' time (playcount) format is reached, this code should be changed because it'll use somethign like YYYY-MM-DD HH:MM:SS.

How did you add the %added%-tag? manually?[a href="index.php?act=findpost&pid=274624"][{POST_SNAPBACK}][/a]

Quote

%added% is something I add manually whenever I import new music. If it was done by foo_playcount, it would update every time it was played (not what I want).[a href="index.php?act=findpost&pid=271389"][{POST_SNAPBACK}][/a]

this looks like a cool idea. i can't seem to get it working. i paste the code into the columns globals section, just at the bottom. then i make a new column and put %_hotness% in the display of that column. i have the %added% test with YYYYMMDD and also %play_date% with YYYYMMDD. after all that i only see ? in that column. what am i doing wrong?

yeah i have both of those checked. it is wierd. i really want this thing to work, it seems like a really cool thing to have. There should be a tag of _hotness after i play the song right? i am not getting that tag after i play songs.

Wow, surprised at the sudden burst of interest this has received. I suppose this will be a little more useful once FIRST_PLAYED is added to foo_playcount. I changed the code in the first post to conform to the LAST_PLAYED standard, as well as a few other changes.

jkwarras, I didn't study the altered code you posted because the code I posted should work with that format. If it still doesn't, let me know. I did notice that you didn't mention having %added% tags, so that might be a problem.

edit: might be forced to add some line breaks to the code at some point...these codeboxes behave very differently in mozilla and ie.

how can i make the hotness dots display in different shades of a colour for different hotness values? i'm using lyx's navigator fcs and pasted the main hotness code at the end of the variables section in the globals tab. i also pasted this code: $blend(FFFFFF,0000FF,%_hotness%,100)● in the display tab of the hotness column, and still all the dots are identical shade of black BTW great idea with this hotness thing

hey topdownjimmy, i've been watching this thread for a while and am really looking forward to using this. i ried it once, but realized i didnt have the %added% values entered, so have ben waiting till i have some time to go through my archives and tag them appropriately, guestimating when they were "added". i am also eagerly awaiting the standardization of this last played tag and it's incorporation into existing plug-ins.