SIRUPhttp://sirup.wmprojects.nl
Enhancing Serendipity In Recommendations via User PerceptionsThu, 23 Mar 2017 08:50:39 +0000en-UShourly1https://wordpress.org/?v=4.8.5Network Institute end of year presentationhttp://sirup.wmprojects.nl/uncategorized/network-institute-end-of-year-presentation/
http://sirup.wmprojects.nl/uncategorized/network-institute-end-of-year-presentation/#respondMon, 21 Jul 2014 12:58:37 +0000http://sirup.wmprojects.nl/?p=206Continue reading Network Institute end of year presentation→]]>On Friday the fourth of July we presented the outcomes of our SIRUP project at the Network Institute end of year presentation. Central to our presentation were the results of the testing of the theoretical model and the three preliminary studies (complexity, familiarity, and conflict). If you have missed our presentation or would like to take another look at it, you can check out our presentation here: Network Institute SIRUP end of year presentation.
]]>http://sirup.wmprojects.nl/uncategorized/network-institute-end-of-year-presentation/feed/0Testing the theoretical modelhttp://sirup.wmprojects.nl/uncategorized/testing-the-theoretical-model/
http://sirup.wmprojects.nl/uncategorized/testing-the-theoretical-model/#respondMon, 21 Jul 2014 12:39:35 +0000http://sirup.wmprojects.nl/?p=198Continue reading Testing the theoretical model→]]>For the SIRUP project we conducted a total of three preliminary studies in order to try to identify successful indicators for complexity, familiarity, and conflict. In addition to the measures for these three factors, we included measures to enable us to test the theoretical model we developed based on the work of Daniel E. Berlyne and David Sylvia (see Figure 1).

Figure 1. SIRUP theoretical model

First of all, it was hypothesized that novelty would have a positive effect on interest. However, novelty can be perceived as threatening and as a result hinder interest. The extent to which novelty is perceived as a threath depends on the individual. Therefore, it was argued that an individual’s ability to cope with novelty (i.e. coping potential) is also related to interest. The relationship between coping potential and interest was expected to be curvilinear, such that interest is highest at a midlevel of coping potential. This means that some novelty is required in order for interest to occur, but not too much novelty.

Based on Berlyne’s work, we argued that novelty is determined by complexity, familiarity, and conflict. It was hypothesized that complexity and conflict have a positive effect on novelty (i.e. higher complexity and conflict predict higher novelty), whereas familiarity has a negative effect on novelty (i.e. lower familiarity predicts higher novelty).

For each of the programs in the three preliminary studies, participants were not only asked to rate the perceived complexity , familiarity, conflict of the program, but also the programs’ perceived novelty and the level of interest as triggered by the program description. Novelty and interest were coded on a 7-point semantic scale, with “familiar/boring” on the left and “novel/interesting” extreme on the right. Furthermore, participants were asked to rate how much they would find it pleasant, relevant, and unexpected if a recommender system were to provide them with this particular recommendation (measurement of the three defined components of serendipity). Answers were coded on a 5-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). Lastly, a simple measure for coping potential was included by asking participants how many hours of television they watch on an average day. The measures are displayed in Figure 2 .

Figure 2. Measures of the theoretical model test

The results of the testing of the theoretical model were very consistent across the three preliminary studies. As was predicted, complexity and conflict had a positive effect on the perceived novelty of a television program. Although it was expected that familiarity would negatively relate to novelty, the results showed that familiarity positively predicted the perceived novelty of the program. This indicates that programs that were perceived as more familiar by participants were rated more novel. This is remarkable considering that familiarity is opposite of novelty. However, this finding may be attributed to the fact that familiarity was conceptualized as popularity (i.e. programs perceived as more popular were rated as more novel).

The results also showed that novelty positively predicted interest, supporting our expectation. Considering the measure for coping potential, the results suggested that coping potential had a positive influence on interest.

Looking at the measure for serendipity, the results consistently showed that interest had a strong positive association with pleasant and relevant. However, unexpectedness had a strong negative association with interest, suggesting that a recommendation should not be unexpected in order to evoke serendipity.

In sum, the results showed that the theoretical model seems correct and may be a useful framework to help induce serendipity in users. However, we have not yet succeeded in identifying the indicators for complexity, familiarity, and conflict. In the future we should define the new indicators or use another test method for the factors we defined.

]]>http://sirup.wmprojects.nl/uncategorized/testing-the-theoretical-model/feed/0Preliminary study 3: Conflicthttp://sirup.wmprojects.nl/uncategorized/preliminary-study-3-conflict/
http://sirup.wmprojects.nl/uncategorized/preliminary-study-3-conflict/#respondThu, 17 Jul 2014 17:41:25 +0000http://sirup.wmprojects.nl/?p=181Continue reading Preliminary study 3: Conflict→]]>In the third preliminary study of SIRUP we tried to identify indicators that predict the perceived conflict of a television program. In the current context, conflict refers to incongruency in peoples’ evaluation of a program. During this project we work with broadcasting data from BBC’s ViSTA-TV project. Based on the available data from this project and external public sources, we identified two possible indicators for conflict (see Figure 1):

Variance (SD) in IMDB user ratings

Variance (SD) in BBC user ratings

Figure 1. Testing the selected indicators for conflict

For both indicators, a positive relationship was hypothesized, such that a higher variance in IMDB and BBC user ratings would positively predict the perceived conflict of the program (e.g., programs with a high variance in IMDB user ratings are perceived as more conflicting by users).

Using the BBC ViSTA-TV data, we selected the five programs with the lowest and highest standard deviation in IMDB and BBC user ratings. As such, a total of 20 programs was rated by the participants. The selected programs can be found here: SIRUP Preliminary study 3, Conflict – Selected programs.

Similar to the previous preliminary studies (complexity and familiarity), participants were provided with a program episode description for each of the programs informing about the program title and season number, the episode number and title, and the episode synopsis, genre, format, and release year. Below an example of such a description is provided for the program What’s New Scooby Doo? (high SD IMDB user ratings; see Figure 2).

Figure 2. Program episode description example What’s New Scooby Doo?

Participants rated popularity on a 7-point semantic scale, with “people very much disagree about this show and evaluate it very differently” on the left and “people very much agree about this show and evaluate it very similary” on the right (see Figure 3).

Figure 3. Measure for conflict

Data were collected via Amazon Mechanical Turk and a total of 160 participants from the United States of America participated in the study. As was the case for the previous preliminary studies (complexity and familiarity), the results showed that none of the indicators we selected was related to conflict. This means that the variance in IMDB and BBC user ratings did not predict the perceived conflict of a program. More information about the results of the preliminary study for familiarity can be found here: SIRUP Preliminary study 3, Conflict – Results.

In short, we have not yet successfully indentified indicators that predict the perceived conflict of a program. If you have any ideas or suggestions, please leave us a message below!

]]>http://sirup.wmprojects.nl/uncategorized/preliminary-study-3-conflict/feed/0Preliminary study 2: Familiarityhttp://sirup.wmprojects.nl/uncategorized/preliminary-study-2-familiarity/
http://sirup.wmprojects.nl/uncategorized/preliminary-study-2-familiarity/#respondWed, 16 Jul 2014 16:59:49 +0000http://sirup.wmprojects.nl/?p=164Continue reading Preliminary study 2: Familiarity→]]>In the second preliminary study of SIRUP we tried to identify indicators that predict the perceived familiarity of a television program. During this project we work with broadcasting data from BBC’s ViSTA-TV project. Based on the available data from this project and external public sources, we identified five possible indicators for familiarity (see Figure 1):

Number of BBC viewers

Number of Google results

Number of Facebook likes

Number of Twitter references

IMDB user ratings

Figure 1. Testing the selected indicators for familiarity

For all five indicators, a positive relationship was hypothesized, such that a higher number of BBC ratings, Google results, Facebook likes, Twitter references, and IMDB user ratings would positively predict the perceived familiarity of the program (e.g., programs with more Facebook likes are perceived as more familiar by users).

Using the BBC ViSTA-TV data, we selected the five programs with the lowest and highest number of BBC viewers, Google results, Facebook likes, Twitter references, and IMDB user ratings. As such, a total of 50 programs was rated by the participants. The selected programs can be found here: SIRUP Preliminary study 2, Familiarity – Selected programs.

Similar to the first preliminary study (complexity), participants were provided with a program episode description for each of the programs informing about the program title and season number, the episode number and title, and the episode synopsis, genre, format, and release year. Below an example of such a description is provided for the program Only Fools and Horses (high IMDB user ratings; see Figure 2).

Figure 2. Program episode description example Only Fools and Horses

In its measurement, familiarity was conceptualized as popularity. For each of the programs participants were asked whether they had watched the program or not. If they had watched the program, they were asked to rate the perceived popularity of the program. If they had not watched the program, they were asked to rate the perceived popularity of the program based on its description. In both cases participants rated popularity on a 7-point semantic scale, with “unpopular” on the left and “popular” on the right (see Figure 3).

Figure 3. Measure for familiarity

Data were collected via Amazon Mechanical Turk and a total of 164 participants from the United States of America participated in the study. As was the case for the first preliminary study (complexity), the results showed that none of the indicators we selected was related to familiarity. This means that the number of BBC viewers, Google results, Facebook likes, Twitter references, and IMDB user ratings of a program did not predict the perceived familiarity of a program. This finding was replicated when we distinguished between participants that had watched the show and those that had not. More information about the results of the preliminary study for familiarity can be found here: SIRUP Preliminary study 2, Familiarity – Results.

In short, we have not yet successfully indentified indicators that predict the perceived familiarity of a program. If you have any ideas or suggestions, please leave us a message below!

]]>http://sirup.wmprojects.nl/uncategorized/preliminary-study-2-familiarity/feed/0Preliminary study 1: Complexityhttp://sirup.wmprojects.nl/uncategorized/preliminary-study-1-complexity/
http://sirup.wmprojects.nl/uncategorized/preliminary-study-1-complexity/#respondTue, 15 Jul 2014 13:18:28 +0000http://sirup.wmprojects.nl/?p=106Continue reading Preliminary study 1: Complexity→]]>In the first preliminary study of SIRUP we tried to identify indicators that predict the perceived complexity of a television program. During this project we work with broadcasting data from BBC’s ViSTA-TV project. Based on the available data from this project and the work of Daniel E. Berlyne, we identified three possible indicators for complexity (see Figure 1):

Number of credits (people involved in the production of a program)

Number of actors

Number of categories (amount of formats and genres of a program)

Figure 1. Testing the selected indicators for complexity

For all three indicators, a positive relationship was hypothesized, such that a higher number of credits, actors, and categories would positively predict the perceived complexity of the program (e.g., programs with more actors are perceived as more complex by users).

Using the BBC ViSTA-TV data, we selected the five programs with the lowest and highest number of credits, actors, and categories. As such, a total of 30 programs was rated by the participants. The selected programs can be found here: SIRUP Preliminary study 1, Complexity – Selected programs.

For each of the programs, participants were provided with a program episode description informing about the program title and season number, the episode number and title, and the episode synopsis, genre, format, and release year. Below an example of such a description is provided for the program Family Guy (low number of credits; see Figure 2).

Figure 2. Program episode description example Family Guy

For each of the programs participants were asked whether they had watched the program or not. If they had watched the program, they were asked to rate the perceived complexity of the program. If they had not watched the program, they were asked to rate the perceived complexity of the program based on its description. In both cases participants rated complexity on a 7-point semantic scale, with “simple” on the left and “complex” on the right (see Figure 3).

Figure 3. Measure for complexity

Data were collected via Amazon Mechanical Turk and a total of 172 participants from the United States of America participated in the study. Unfortunately, the results showed that none of the indicators we selected was related to complexity. This means that the number of credits, actors, and categories of a program did not predict the perceived complexity of a program. This finding was replicated when we distinguished between participants that had watched the show and those that had not. More information about the results of the preliminary study for complexity can be found here: SIRUP Preliminary study 1, Complexity – Results.

In short, we have not yet successfully indentified indicators that predict the perceived complexity of a program. If you have any ideas or suggestions, please leave us a message below!

]]>http://sirup.wmprojects.nl/uncategorized/preliminary-study-1-complexity/feed/0Network Institute poster sessionhttp://sirup.wmprojects.nl/uncategorized/network-institute-poster-session/
http://sirup.wmprojects.nl/uncategorized/network-institute-poster-session/#respondTue, 15 Apr 2014 10:16:38 +0000http://sirup.wmprojects.nl/?p=94Continue reading Network Institute poster session→]]>Today, we will present our SIRUP project at the Network Institute poster session. Central to our presentation is the question of how we will create the experience of serendipity in a user through a recommendation provided by the recommender system. If you want to find out, come check out our poster (VU University Amsterdam, April 15, 16.00-17.30, Intertain Lab, W&N Building S1.11) or take a look at our poster below.

]]>http://sirup.wmprojects.nl/uncategorized/network-institute-poster-session/feed/0Theoretical modelhttp://sirup.wmprojects.nl/uncategorized/theoretical-model/
http://sirup.wmprojects.nl/uncategorized/theoretical-model/#respondTue, 15 Apr 2014 10:00:14 +0000http://sirup.wmprojects.nl/?p=86Continue reading Theoretical model→]]>Serendipity is making a pleasant and relevant discovery that was unexpected. In SIRUP we argue that a serendipitous recommendation primarily induces interest in a user. Hence, serendipitous recommendations are those that trigger interest in users.

Building on the classic theory of interest by Daniel E. Berlyne and the follow-up work by Paul Silvia, we argue that user interest, and thus serendipity, is determined by two things: novelty (inherent to the recommendation) and coping potential (inherent to the user). Novelty, in turn, is determined by the complexity, familiarity and conflict characteristics of the recommended item.

Based on this logic, we present SIRUP’s theoretical model:

]]>http://sirup.wmprojects.nl/uncategorized/theoretical-model/feed/0Pre-study online nowhttp://sirup.wmprojects.nl/uncategorized/pre-study-online-now/
http://sirup.wmprojects.nl/uncategorized/pre-study-online-now/#respondTue, 01 Apr 2014 12:25:33 +0000http://sirup.wmprojects.nl/?p=78Continue reading Pre-study online now→]]>How can we enhance serendipity in user recommendations? After an intense period of theorizing, conceptualizing, and preparing the stimulus material, the first preliminary study of the SIRUP project is online now! The first data begin to trickle in and soon we will be able to present the very first results of our project. Want to know more about the things that we have been working on? Come and take a look (and oh yes: participate) here.
]]>http://sirup.wmprojects.nl/uncategorized/pre-study-online-now/feed/0Welcomehttp://sirup.wmprojects.nl/uncategorized/34/
http://sirup.wmprojects.nl/uncategorized/34/#respondThu, 10 Oct 2013 22:17:53 +0000http://sirup.wmprojects.nl/?p=34Continue reading Welcome→]]>Creating serendipity (i.e. “pleasant surprises for users”) is a primary goal of intelligent recommender systems. This project proposes an interdisciplinary approach to enhance the serendipity of TV recommendations that combines complementary knowledge from three disciplines – Computer Science, Language & Cognition and Communication Science.

The project examines the “back-end” or algorithms behind serendipitous TV recommendations (Computer Science), the “front-end” or the actual display of these recommendations (Language & Cognition), and the “effect” on users’ perceptions and satisfaction (Communication Science).

]]>http://sirup.wmprojects.nl/uncategorized/34/feed/0SIRUP Kick-offhttp://sirup.wmprojects.nl/event/sirup-kick-off/
http://sirup.wmprojects.nl/event/sirup-kick-off/#respondThu, 10 Oct 2013 22:13:37 +0000http://sirup.wmprojects.nl/?p=28Today we had the kick off of this very exciting project collaboration between Computer Scientists, Language & Cognition and Communication Sciences to study in an interdisciplinary setting ways to enhance the serendipity of TV recommendations.
]]>http://sirup.wmprojects.nl/event/sirup-kick-off/feed/0