tag:blogger.com,1999:blog-5893419202319810209.post6172211530091169251..comments2017-03-28T07:15:46.685-07:00Comments on The Datographer: Recap: Iron Viz - Reviewing the ReviewersJohn Mathishttp://www.blogger.com/profile/11085951430241126217noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-5893419202319810209.post-26326083063922204882014-09-28T10:58:11.887-07:002014-09-28T10:58:11.887-07:00Thanks for the explanation, John. Yes, that does ...Thanks for the explanation, John. Yes, that does make sense and further highlights why I need to up my statistical literacy. :-) I appreciate the education.Michael Mixonhttps://www.blogger.com/profile/05168534725209307940noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-82030840491394467512014-09-22T22:36:54.753-07:002014-09-22T22:36:54.753-07:00Great question. You&#39;re right, the absolute val...Great question. You&#39;re right, the absolute value of the bias is similar to the reviewer error, although slightly different. The practical effect of using RMSD instead is that it &#39;punishes&#39; larger outliers more.<br /><br />For example, consider a reviewer that left four reviews: 2, 3, 4 and 5 stars each for businesses with overall 5 star rating. The average of the absolute value of bias would be 2.5 stars and the reviewer error would be 1.9. Now consider a reviewer who instead left four reviews of 2, 2, 5 and 5 for the same businesses. The absolute value of their bias is still 2.5, but now the reviewer error is 2.1. The second reviewer has a higher error because they have two reviews that are a full three stars off despite the average absolute bias being the same.<br /><br />While I don&#39;t think this would materially change the analysis, I would rather punish those who are far off slightly more so I went with reviewer error.John Mathishttps://www.blogger.com/profile/00756088358139856470noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-8408921937994973492014-09-22T21:32:41.283-07:002014-09-22T21:32:41.283-07:00Hey Michael,
I actually reversed the axis when I ...Hey Michael,<br /><br />I actually reversed the axis when I was practicing, but neglected to do it in the finals! I totally agree reversing the axis would make sense in the dashboard. The only thing I would add is, if I had any bar charts showing error, I&#39;d definitely keep vertical axes in sync otherwise in some charts &#39;high&#39; values are bad and in others they are good.John Mathishttps://www.blogger.com/profile/00756088358139856470noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-21806180326734942532014-09-21T22:19:09.566-07:002014-09-21T22:19:09.566-07:00Hi John,
I have another, more geeky question. Wh...Hi John,<br /><br />I have another, more geeky question. What would the difference be in your analysis if you used the absolute bias (absolute value of the reviewer bias) vs. the reviewer error? When I plotted the two on a scatter-plot, the difference between them was quite low (e.g. the Avg Error is 1.096 and the Avg Absolute Bias is .9). If the goal of the analysis is to show those users who deviate the most from the consensus, then both measures achieve that, but Absolute Bias is much easier to understand, at least to the statistically naive like me. :-) <br /><br />I&#39;d be interested to know whether the Reviewer Error formula provides a higher degree of accuracy in determining a reviewer&#39;s deviation from consensus, or whether both measures effectively achieve the same thing.<br /><br />-MikeMichael Mixonhttps://www.blogger.com/profile/05168534725209307940noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-91401374499218180622014-09-21T10:42:35.148-07:002014-09-21T10:42:35.148-07:00Congratulations on the win John! I&#39;m bummed th...Congratulations on the win John! I&#39;m bummed that I missed the session (the first one I&#39;ve ever missed), but there were just too many sessions to choose from at this conference. :-) But I can see why you won...your analysis of the Yelp data offered a unique perspective that was conveyed with clarity in your dashboard. <br /><br />Did you consider reversing the Error axis on the scatter-plot? Having the most trusted users at the bottom vs. the top seems counter-intuitive, at least to me. I realize there might be those who complain about having the axis start at 3 and go &quot;up&quot; to 0, so perhaps this is one of those design coin tosses. :-)<br />Michael Mixonhttps://www.blogger.com/profile/05168534725209307940noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-1910564192630823422014-09-21T06:48:22.807-07:002014-09-21T06:48:22.807-07:00It was cool to watch the three of you compete and ...It was cool to watch the three of you compete and take different approaches to the data. I thought your design was great and I really like the concept of using reviewer error as a means of determining the quality of the reviews. As others mentioned, there was no need for a color legend. Thanks for sharing the ideas behind your approach. Congrats on the win!Markhttps://www.blogger.com/profile/02269447516480875110noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-72935085577849779382014-09-20T12:28:28.838-07:002014-09-20T12:28:28.838-07:00Great dashboard and congratulations on your win! ...Great dashboard and congratulations on your win! I was thrown for a moment with the color, but I agree that the bar chart served as the legend for me. I also liked your examples highlighted above. They were a great examples to understand how to interpret the data.Josh Tapleyhttps://www.blogger.com/profile/01643837309030397187noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-18273013902666760402014-09-18T15:30:59.973-07:002014-09-18T15:30:59.973-07:00I&#39;d say you didnt need the legend, pixels are ...I&#39;d say you didnt need the legend, pixels are precious. Awesome work and under that much pressure tooMatt Francishttps://www.blogger.com/profile/14301128814221268389noreply@blogger.comtag:blogger.com,1999:blog-5893419202319810209.post-18080064298906659222014-09-18T11:15:50.286-07:002014-09-18T11:15:50.286-07:00I thought your bar chart was obviously also servin...I thought your bar chart was obviously also serving as the color legend, so didn&#39;t understand that criticism at all.PeterGhttps://www.blogger.com/profile/00017874900847973883noreply@blogger.com