Open Access
Translator Disclaimer
September 2011 Modeling item–item similarities for personalized recommendations on Yahoo! front page
Deepak Agarwal, Liang Zhang, Rahul Mazumder
Ann. Appl. Stat. 5(3): 1839-1875 (September 2011). DOI: 10.1214/11-AOAS475


We consider the problem of algorithmically recommending items to users on a Yahoo! front page module. Our approach is based on a novel multilevel hierarchical model that we refer to as a User Profile Model with Graphical Lasso (UPG). The UPG provides a personalized recommendation to users by simultaneously incorporating both user covariates and historical user interactions with items in a model based way. In fact, we build a per-item regression model based on a rich set of user covariates and estimate individual user affinity to items by introducing a latent random vector for each user. The vector random effects are assumed to be drawn from a prior with a precision matrix that measures residual partial associations among items. To ensure better estimates of a precision matrix in high-dimensions, the matrix elements are constrained through a Lasso penalty. Our model is fitted through a penalized-quasi likelihood procedure coupled with a scalable EM algorithm. We employ several computational strategies like multi-threading, conjugate gradients and heavily exploit problem structure to scale our computations in the E-step. For the M-step we take recourse to a scalable variant of the Graphical Lasso algorithm for covariance selection.

Through extensive experiments on a new data set obtained from Yahoo! front page and a benchmark data set from a movie recommender application, we show that our UPG model significantly improves performance compared to several state-of-the-art methods in the literature, esp"otherecially those based on a bilinear random effects model (BIRE). In particular, we show that the gains of UPG are significant compared to BIRE when the number of users is large and the number of items to select from is small. For large item sets and relatively small user sets the results of UPG and BIRE are comparable. The UPG leads to faster model building and produces outputs which are interpretable.


Download Citation

Deepak Agarwal. Liang Zhang. Rahul Mazumder. "Modeling item–item similarities for personalized recommendations on Yahoo! front page." Ann. Appl. Stat. 5 (3) 1839 - 1875, September 2011.


Published: September 2011
First available in Project Euclid: 13 October 2011

zbMATH: 1231.62207
MathSciNet: MR2884924
Digital Object Identifier: 10.1214/11-AOAS475

Keywords: Collaborative filtering , graphical lasso , item–item similarities , matrix factorization , recommender systems

Rights: Copyright © 2011 Institute of Mathematical Statistics


Vol.5 • No. 3 • September 2011
Back to Top