Following the user’s interests in mobile context aware recommender systems

The wide development of mobile applications provides a considerable amount of data of all types (images, texts, sounds, videos, etc.). In this sense, Mobile Context-aware Recommender Systems (MCRS) suggest the user suitable information depending on her/his situation and interests. Two key questions have to be considered 1) how to recommend the user information that follows his/her interests evolution? 2) how to model the user’s situation and its related interests? To the best of our knowledge, no existing work proposing a MCRS tries to answer both questions as we do. This paper describes an ongoing work on the implementation of a MCRS based on the hybrid-ε-greedy algorithm we propose, which combines the standard ε-greedy algorithm and both content-based filtering and case-based reasoning techniques.

1.
Following the User’s Interests in Mobile Context-Aware Recommender Systems : The hybrid-ε-greedy algorithm Djallel Bouneffouf, Amel Bouzeghoub & Alda Lopes Gançarski Department of Computer Science, Télécom SudParis, UMR CNRS Samovar, 91011 Evry Cedex, France {Djallel.Bouneffouf, Amel.Bouzeghoub, Alda.Gancarski}@it-sudparis.euAbstract— The wide development of mobile applications based on his current context. This goal matches with the goalprovides a considerable amount of data of all types (images, of recommender systems.texts, sounds, videos, etc.). In this sense, Mobile Context-aware Both context-aware systems and recommender systemsRecommender Systems (MCRS) suggest the user suitable are used to provide users with relevant information and/orinformation depending on her/his situation and interests. Two services; the first based on the user’s context; the secondkey questions have to be considered 1) how to recommend the based on the user’s interests. Therefore, the next logical stepuser information that follows his/her interests evolution? 2) is to combine these two systems within the so-called mobilehow to model the user’s situation and its related interests? To context-aware recommender systems.the best of our knowledge, no existing work proposing a MCRS The notion of context is not new. In general, it refers totries to answer both questions as we do. This paper describesan ongoing work on the implementation of a MCRS based on any information that is relevant to the user and his/herthe hybrid-ε-greedy algorithm we propose, which combines the surroundings. The main difficulty when dealing with contextstandard ε-greedy algorithm and both content-based filtering is its high dynamicity and its constant change. A change inand case-based reasoning techniques. one contextual feature may generate changes in other contextual features. That’s why context-aware systems have Keywords-component; context-aware recommender systems; to be adapted to highly dynamic user situations and contextpersonalization; situation. models need to capture the dynamic interactions between contextual information. I. INTRODUCTION Several works in context-aware systems have addressed the problem of context modeling and interesting single or Mobile technologies have made access to a huge hybrid models exist. Most of them are gathered in the surveycollection of information, anywhere and anytime. Thereby, done in [5]. However, these models have not been widelyinformation is customized according to users’ needs and applied to the mobile application domain. The dynamicpreferences. This brings big challenges for the interactions between contextual dimensions and user profilesRecommender System field. Indeed, technical features of have not been fully represented by the existing models whichmobile devices yield to navigation practices which are more often focus on modeling static associations among differentdifficult than the traditional navigation task. contextual information. Recommender systems are systems that produce In recommender systems, a considerable amount ofindividualized suggestions concerning interesting content the research has been done in recommending relevantuser might like out of a large number of alternatives. Often, information for mobile users. Earlier techniques ([12], [15])recommender systems using collaborative or content-based are based solely on the computational behavior of the user tofiltering algorithms are applied. model his interests regardless of his surrounding In mobile applications, information personalization is environment (location, time, nearby people). The maineven more important, because of the limitations of mobile limitation of such approaches is that they do not take intodevices in terms of displays, input capabilities, bandwidth account the contextual evolution of the user interests w. r. t.etc. It is then desirable to personalize not only using pre- his context.defined user profiles, but also the user’s context such as the This study gives rise to another category ofcurrent location. recommendation techniques that tackle these limitations We can find similar challenges in context-aware systems when building situation-aware user profiles. Two keyarea. A general definition of context-aware systems is given questions have to be considered, namely 1) how toin [4]: “A system is context-aware if it uses context to recommend the user information that follows his/herprovide relevant information and/or services to the user, interests evolution? 2) how to model the user’s situation andwhere relevancy depends on the user’s task.” its related user’s interests? The key goal of context-aware systems is hence toprovide a user with relevant information and/or services

2.
In order to tackle these problems, our approach consists documents may be chosen. However, obtaining informationof: about the documents’ average rewards (i.e., exploration) can  Capturing context through communication with refine B’s estimate of the documents’ rewards and in turn diverse information sources: from the mobile device increase long-term user’s satisfaction. Clearly, neither a GPS capability, if it is possible, or by inferring a purely exploring nor a purely exploiting algorithm works user’s location based on the user’s calendar or other best in general, and a good tradeoff is needed. The authors devices in the neighborhood. on [7, 8, 16] describe a smart way to balance exploration and  Storing history for future use or user habit analysis. exploitation. However, none of them consider the user’s  Reasoning and learning capabilities to be able to (i) situation during the recommendation. use the collected contextual information for mapping B. Modeling the situation and user profile low-level information to symbolic contextual information (e.g. mapping an exact coordinate to a Few research works are dedicated to manage the user’s symbolic location such as an address) or to deduce situation on recommendation. In [2, 9, 10] the authors higher level information i.e. identify user situation propose a method which consists of building a dynamic and recommend the most relevant content according user’s profile based on time and user’s experience. The to the situation (e.g. determine that the user is in a user’s preferences in the user’s profile are weighted room where a meeting is in progress so the cell according to the situation (time, location) and the user’s phone should be switched to silent mode); (ii) learn behavior. To model the evolution on the user’s preferences the behavior based on reasoning and the history data according to his temporal situation in different periods, (like (e.g. whenever the user enters a room which has a workday or vacations), the weighted association for the meeting in progress or where the next event is a concepts in the user’s profile is established for every new meeting, the user turns the volume of the cell phone experience of the user. The user’s activity combined with the down and searches for the meeting agenda. The users profile are used together to filter and recommend system may learn this behavior for the next time). relevant content. Another work [6] describes a MCRS operating on three The remainder of the paper is organized as follows. dimensions of context that complement each other to getSection II reviews some related works. Section III presents highly targeted. First, the MCRS analyzes information suchthe user model. Section IV introduces the recommendation as clients’ address books to estimate the level of socialalgorithm and the last section concludes the paper and points affinity among users. Second, it combines social affinity without possible directions for future work. the spatiotemporal dimensions and the user’s history in order to improve the quality of the recommendations. II. BACKGROUND In [1], the authors present a technique to perform user- based collaborative filtering. Each user’s mobile device We mention now recent relevant recommendation stores all explicit ratings made by its owner as well as ratingstechniques that tackle the two issues mentioned above, received from other users. Only users in spatiotemporalnamely: following the evolution of user’s interests and proximity are able to exchange ratings and they show howmodeling the situation and user profile. this provides a natural ﬁltering based on social contexts.A. Following the evolution of user’s interests Each work cited above tries to recommend interesting The trend today on recommender systems is to suggest information to users on contextual situation; however theyrelevant information to users, using supervised machine do not consider the evolution of the user interest.learning techniques. In these approaches, the recommendersystem has to execute two steps: (1) the learning step, where To summarize, none of the mentioned works tackles boththe system learns from samples and gradually adjusts its problems. As a result, our approach exploits the followingparameters; (2) the exploitation step, where new samples are new features:presented to the system to perform a generalization [18].  We consider user situations as a multi-dimensional These approaches suffer from the following drawbacks: space where each dimension is represented by a(i) need for initial information about the user’s interests domain ontology. Unlike in [1, 2, 6, 10], whereprovided by an expert; (ii) difficulty in following the context items are low level data, in our approachevolution of the user’s interests. Some works found in the they correspond to concepts of social, location andliterature [7, 8, 16] address these problems as a need for time ontologies. Each situation is associated tobalancing exploration and exploitation studied in the “bandit specific user profile and preferences.algorithm”. A bandit algorithm B exploits its past experience  Inspired by models of human reasoning developedto select documents that appear more frequently. Besides, by [14] in robotic, we propose to consider usersthese seemingly optimal documents may in fact be situation in the bandit algorithm by using case-basedsuboptimal, due to imprecision in B’s knowledge. In order to reasoning technique, which is not considered in [7,avoid this undesired situation, B has to explore documents 8, 9, 16].by actually choosing seemingly suboptimal documents so as  In [7, 8, 16] authors use a smart bandit algorithm toto gather more information about them. Exploitation can manage the exploration/exploitation strategy,increase short-term user’s satisfaction since some suboptimal however they do not take into account the content in

3.
the strategy. Our intuition is that, considering the - The indirect preference: it is the information that we content when managing the exploration/exploitation extract from the user system interaction, for example the strategy will improve it. This is why we propose to number of clicks on the visited documents or the time spent use content-based filtering techniques together with on a document. ε-greedy algorithm. Let UP be the preferences submitted by a specific user in the system at a given situation. Each document in UP is In what follows, we define the structure of the proposed represented as a single vector d=(c1,...,cn), where ci (i=1, ..,user model and the methods for inferring the n) is the value of a component characterizing the preferencesrecommendation situations. Then, we explain how to build of d. We consider the following components: the totaldynamic user profiles, and how to manage the number of clicks on d, the total time spent reading d, theexploration/exploitation strategy, according to the current number of times d was recommended, and the directsituation. preference rate on d. 2) History All the interactions between the user and the system are III. THE PROPOSED USER’S MODEL stored as well as the situations in order to exploit this data to As stated in Section II, static approaches for building the improve the recommendation process.user’s profile [12, 15] are poorly useful in our context, so we calendar Before each meeting, the user has to fill-up therather focus on more dynamic techniques, capable of calendar with information concerning the person(s) to meet,continuously adjusting the user’s interests to the current Time and Location instances are automatically inferred bysituation. the system. Figure 1 depicts the diagram of the proposed user model. B. User Situation A situation S can be represented as a triple whose features X are the values assigned to each dimension: S = (Xl, Xt, Xs), where Xl (resp. Xt and Xs) is the value of the location (resp. time and social) dimension. Suppose the user is associated to: the location "38.868143, 2.3484122" from his phone’s GPS; the time "Mon Oct 3 12:10:00 2011" from his phone’s watch; and the meeting with Paul Gerard from his calendar. To build the situation, we associate to this kind of low level data, directly acquired from mobile devices capabilities, more abstracted concepts using ontologies reasoning means. 1) Location There are different ways to characterize a location. As Figure 1. User model diagram returned by location sensor systems (like GPS), location is aA. User Profile position in systems based on geographic coordinates, or may also be defined by an address. Simple automated placeThe user profile is composed of the user’s personal data and labeling systems are already commercialized (Google maps,other dynamic information, including his preferences, his Yahoo local...) and consist of merging data such as postalcalendar and the history of his interactions with the system. addresses with maps. 1) UserPreferences In our user model, we use a local spatial ontology to The user preferences are contextual and might depend on represent and reason on geographic information. Using thismany factors, like the location or the current task within an ontology, for the above example, we get, from locationactivity. Thus, they are associated to the user situation and "38.86, 2.34", the value “Paris” to insert in the locationthe user activity. Preferences are deduced during user dimension of the situation.navigation activities. They contain the set of navigation 2) Timedocuments used by the user in a situation. A navigation The temporal information is complex: it is continuousactivity expresses the following sequence of events: (i) the and can be represented at different levels of granularity. Touser logs in the system and navigates across documents to define the temporal aspects characterizing the user’sget the desired information; (ii) the user expresses his situation, we suggest to abstract the continuum time intopreferences on the visited documents. We assume that a specific and significant periods (abstract time classes), whichvisited document is relevant, and thus belongs to the user’s we expect having an effect on the user’s behavior (e.g.preferences, if there are some observable user’s behaviors morning, weekend). To allow a good representation of thethrough 2 types of preference: temporal information and its manipulation, we propose to - The direct preference: the user expresses his interest in use OWL-Time ontology [11] which is today a reference forthe document by inserting a rate, like for example putting representing and reasoning about time. We propose to basestars (“*”) at the top of the document. our work on this ontology and extend it if necessary. Taking the example above, for the time value "Mon Oct 3 12:10:00

4.
2011", we get, using the OWL-Time ontology, the value defines the exploration/exploitation tradeoff; ε is the“workday”. probability of recommending a random exploratory 3) Social connection document. The social connection refers to the information of theuser’s interlocutors (e. g. a friend, an important customer, a Algorithm 1 The ε-greedy algorithmcolleague or his manager). To define the neighborhood 1: Input: ε, UPc, Npeople aspects characterizing the user, a clear model for the 2: Output: Drepresentation and reasoning on social clustering isnecessary. We use the FOAF Ontology [13] to describe the 3: D = Øsocial network by a set of concepts and properties. For 4: For i =1 to N doexample, the information about “the meeting with Paul 5: q = Random({0,1})Gerard” can yield the value “wine client” for the socialdimension. 6: arg max (getCTR(d)) if q ≤ ε d(UP  D ) IV. THE PROPOSED RECOMMENDATION ALGORITHM 7: di = In our MCRS, documents’ recommendation is modeled 8: Random(UPc) otherwiseas a multi-armed bandit problem. Formally, a bandit 9: D= D ∪ dialgorithm proceeds in discrete rounds t = 1,… T. For each 10: Endforround t, the algorithm performs the following tasks: 11: Return D Task 1. It observes the current user’s situation St and a setDt of documents with their feature vectors xt,d for d  Dt. The To select the document to be recommended, there arevector xt,d gives information of both user’s situation St and several strategies which provide an approximate solution fordocument d. the bandit problem. Here, we focus on two of them (as in Task 2. Based on observed rewards in previous rounds, it [16]):chooses a document dt  Dt, and receives reward rt ,d whose t - the greedy strategy (Alg. 1, line 6), which estimates eachexpectation depends on both the user’s situation St and the document’s CTR; then it always chooses the best documents,document dt. i.e. the ones having the higher CTR; thus it only performs Task 3. It then improves its document-selection strategy exploitation;with the new observation. - the ε-greedy strategy, which adds some greedy exploration In tasks 1 to 3, the total T-round reward of D is defined policy to the greedy strategy, choosing the best document as T r with probability 1–ε (Alg. 1, line 6) or a randomly picked t 1 t , d t document otherwise (probability ε) (Alg. 1, line 8). B. The proposed hybrid-ε-greedy algorithm  while the optimal expected T-round reward is defined as  t 1 rt ,d * T We propose a two-fold improvement on the performance t of the ε-greedy algorithm: integrating case base reasoningwhere dt* is the document with maximum expected reward at (CBR) and content based filtering (CBF). This new proposedround t. Our goal is to design the bandit algorithm so that the algorithm is called hybrid-ε-greedy and is described inexpected total reward is maximized. Algorithm 4. In the field of document recommendation, when a To improve exploitation of the ε-greedy algorithm, wedocument is presented to the user and this one selects it by a propose to integrate CBR into each iteration: before choosingclick, a reward of 1 is incurred; otherwise, the reward is 0. the document, the algorithm computes the similarity betweenWith this definition of reward, the expected reward of a the present situation and each one in the situation base; ifdocument is precisely its Click Through Rate (CTR). The there is a situation that can be re-used, the algorithmCTR is the average number of clicks on a recommended retrieves it, and then applies an exploration/exploitationdocument, computed diving the total number of clicks on it strategy.by the number of times it was recommended. In this situation-aware computing approach, the premiseA. The ε-greedy algorithm part of a case is a specific situation S of a mobile user when he navigates on his mobile device, while the value part of aThe ε-greedy strategy is sketched in Algorithm 1. For a case is the user’s preferences UP to be used for thegiven user’s situation, the algorithm recommends a pre- recommendation. Each case from the case base is denoted asdefined number of documents, specified by the parameter N. C= (S, UP).In this algorithm, UPc={d1,…,dP} is the set of documents Let Sc=(Xlc, Xtc, Xsc) be the current situation of the user,corresponding to the current user’s preferences; UP the current user’s preferences and PS={S1,....,Sn} the set cD={d1,….,dN} is the set of documents to recommend; of past situations. The proposed hybrid-ε-greedy algorithmgetCTR (Alg. 1, line 6) is the function which estimates the involves the following four methods.CTR of a given document; Random (Alg. 1, lines 5 and 8) isthe function returning a random element from a given set; q 1) RetriveCase() (Alg. 4, line 4)is a random value uniformly distributed over [0, 1] which

5.
Given the current situation Sc, the RetrieveCase methoddetermines the expected user preferences by comparing Scwith the situations in past cases in order to choose the most Algorithm 2 The RecommendDocuments() methodsimilar one Ss. The method returns, then, the corresponding 1: Input: ε, UPc, Ncase (Ss, UPs). 2: Output: D Ss is selected from PS by computing the following 3: D=Øexpression as it done by [9]: 4: For i=1 to N do   S s = arg max   α j  sim j X c ,X ij  (1) 5: q = Random({0, 1})  j  S PS i  j  6: j = Random({0, 1}) In equation 1, simj is the similarity metric related to 7: arg max (getCTR(d)) if j<q<εdimension j between two situation vectors and αj the weight d(UP  D )associated to dimension j. αj is not considered in the scope of 8: di = CBF(UPc-D, arg max (getCTR(d)) if q≤ j≤εthis paper, taking a value of 1 for all dimensions. d(UP  D ) The similarity between two concepts of a dimension j in 9: Random(UPc) otherwisean ontological semantic depends on how closely they are 10: D = D ∪ {di }related in the corresponding ontology (location, time or 11: Endforsocial). We use the same similarity measure as [17] defined 12: Return Dby equation 2: sim X c , X i   2  deph( LCS ) (2) Algorithm 3 The CBF() algorithm (deph( X c )  deph( X ij )) j j j j 1: Input: UP, dbHere, LCS is the Least Common Subsumer of Xjc and Xji, and 2: Output: dsdepth is the number of nodes in the path from the node to the 3: ds= arg max (cossim( db , d))ontology root. dUP 2) RecommendDocuments() (Alg. 4, line 6) 4: Return ds In order to insure a better precision of the recommenderresults, the recommendation takes place only if the followingcondition is verified: sim(Sc, Ss) ≥ B (Alg. 4, line 5), where B 3) UpdateCase() & InsertCase()is a threshold value and After recommending documents applying the sim(S c , S s ) = sim j X c ,X s  j j RecommendDocuments method (Alg. 4, line 6), the user’s j preferences are updated w. r. t. number of clicks and number In the RecommendDocuments() method, sketched in of recommendations for each recommended document onAlgorithm 2, we propose to improve the ε-greedy strategy by which the user clicked at least one time. This is done by theapplying CBF in order to have the possibility to recommend, UpdatePreferences function (Alg. 4, line 7).not the best document, but the most similar to it (Alg. 2, line Depending on the similarity between the current situation8). We believe this may improve the user’s satisfaction. Sc and its most similar situation Ss (computed with The CBF algorithm (Algorithm 3) computes the RetriveCase()), being 3 the number of dimensions in thesimilarity between each document d=(c1,..,ck) from UP context, two scenarios are possible:(except already recommended documents D) and the best - sim(Sc, Ss) ≠ 3: the current situation does not exist in thedocument db=(cjb ,.., ckb ) and returns the most similar one. case base (Alg. 4, line 8); the InsertCase() method adds toThe degree of similarity between d and db is determined by the case base the new case composed of the current situationusing the cosine measure, as indicated in equation 3: Sc and the updated UP.  ck  ck (3) - sim(Sc, Ss) = 3: the situation exists in the case base (Alg. 4, d db line 10); the UpdateCase() method updates the case having cos sim(d , d b )   k d  db premise situation Sc with the updated UP. c k b k 2  c kb2 k