About us
Publications
Home
Publications from 2020 and before are not direct results of the SFI MediaFutures, but are key results from our team members working on related topics in MediaFutures.
2020
Krisztian Balog; Filip Radlinski
Measuring Recommendation Explanation Quality: The Conflicting Goals of Explanations Conference
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20), New York, 2020, (Pre SFI).
Abstract | BibTeX | Tags: HCI design and evaluation methods, Human-centered computing, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement | Links:
@conference{Balog2020,
title = {Measuring Recommendation Explanation Quality: The Conflicting Goals of Explanations},
author = {Krisztian Balog and Filip Radlinski},
url = {https://dl.acm.org/doi/pdf/10.1145/3397271.3401032},
doi = {https://doi.org/10.1145/3397271.3401032},
year = {2020},
date = {2020-07-01},
booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20)},
pages = {329–338},
address = {New York},
abstract = {Explanations have a large effect on how people respond to recommendations. However, there are many possible intentions a system may have in generating explanations for a given recommendation -from increasing transparency, to enabling a faster decision, to persuading the recipient. As a good explanation for one goal may not be good for others, we address the questions of (1) how to robustly measure if an explanation meets a given goal and (2) how the different goals interact with each other. Specifically, this paper presents a first proposal of how to measure the quality of explanations along seven common goal dimensions catalogued in the literature. We find that the seven goals are not independent, but rather exhibit strong structure. Proposing two novel explanation evaluation designs, we identify challenges in evaluation, and provide more efficient measurement approaches of explanation quality.},
note = {Pre SFI},
keywords = {HCI design and evaluation methods, Human-centered computing, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement},
pubstate = {published},
tppubtype = {conference}
}
Explanations have a large effect on how people respond to recommendations. However, there are many possible intentions a system may have in generating explanations for a given recommendation -from increasing transparency, to enabling a faster decision, to persuading the recipient. As a good explanation for one goal may not be good for others, we address the questions of (1) how to robustly measure if an explanation meets a given goal and (2) how the different goals interact with each other. Specifically, this paper presents a first proposal of how to measure the quality of explanations along seven common goal dimensions catalogued in the literature. We find that the seven goals are not independent, but rather exhibit strong structure. Proposing two novel explanation evaluation designs, we identify challenges in evaluation, and provide more efficient measurement approaches of explanation quality.
Alain D. Starke; Martijn C. Willemsen; Chris C.P. Snijders
With a little help from my peers: depicting social norms in a recommender interface to promote energy conservation Conference
no. March 2020, 2020.
Abstract | BibTeX | Tags: Decision Support System, Human computer interaction, Human-centered computing, Information Systems, User studies | Links:
@conference{Starke2020b,
title = {With a little help from my peers: depicting social norms in a recommender interface to promote energy conservation},
author = {Alain D. Starke and Martijn C. Willemsen and Chris C.P. Snijders},
url = {https://dl.acm.org/doi/10.1145/3377325.3377518},
doi = {10.1145/3377325.3377518},
year = {2020},
date = {2020-03-17},
number = {March 2020},
pages = {1-11},
abstract = {How can recommender interfaces help users to adopt new behaviors? In the behavioral change literature, nudges and norms are studied to understand how to convince people to take action (e.g. towel re-use is boosted when stating that `75% of hotel guests' do so), but what is advised is typically not personalized. Most recommender systems know what to recommend in a personalized way, but not much research has considered how to present such advice to help users to change their current habits. We examine the value of presenting normative messages (e.g. `75% of users do X') based on actual user data in a personalized energy recommender interface called `Saving Aid'. In a study among 207 smart thermostat owners, we compared three different normative explanations (`Global', `Similar', and `Experienced' norm rates) to a non-social baseline (`kWh savings'). Although none of the norms increased the total number of chosen measures directly, we show evidence that the effect of norms seems to be mediated by the perceived feasibility of the measures. Also, how norms were presented (i.e. specific source, adoption rate) affected which measures were chosen within our Saving Aid interface.},
keywords = {Decision Support System, Human computer interaction, Human-centered computing, Information Systems, User studies},
pubstate = {published},
tppubtype = {conference}
}
How can recommender interfaces help users to adopt new behaviors? In the behavioral change literature, nudges and norms are studied to understand how to convince people to take action (e.g. towel re-use is boosted when stating that `75% of hotel guests' do so), but what is advised is typically not personalized. Most recommender systems know what to recommend in a personalized way, but not much research has considered how to present such advice to help users to change their current habits. We examine the value of presenting normative messages (e.g. `75% of users do X') based on actual user data in a personalized energy recommender interface called `Saving Aid'. In a study among 207 smart thermostat owners, we compared three different normative explanations (`Global', `Similar', and `Experienced' norm rates) to a non-social baseline (`kWh savings'). Although none of the norms increased the total number of chosen measures directly, we show evidence that the effect of norms seems to be mediated by the perceived feasibility of the measures. Also, how norms were presented (i.e. specific source, adoption rate) affected which measures were chosen within our Saving Aid interface.
2019
Maurizio Ferrari Dacrema; Paolo Cremonesi; Dietmar Jannach
Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches Conference
Proceedings of the 2019 ACM Conference on Recommender Systems (RecSys 2019), Copenhagen, 2019, (Pre SFI).
Abstract | BibTeX | Tags: Collaborative filtering, General and referance, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement | Links:
@conference{Dacrema2019,
title = {Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches},
author = {Maurizio Ferrari Dacrema and Paolo Cremonesi and Dietmar Jannach},
url = {https://arxiv.org/pdf/1907.06902.pdf},
year = {2019},
date = {2019-08-16},
booktitle = {Proceedings of the 2019 ACM Conference on Recommender Systems (RecSys 2019)},
address = {Copenhagen},
abstract = {Deep learning techniques have become the method of choice for
researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in
general, it has, as a result, become difficult to keep track of what
represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications
point out problems in today’s research practice in applied machine
learning, e.g., in terms of the reproducibility of the results or the
choice of the baselines when proposing new models.
In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically,
we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however
turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or
graph-based techniques. The remaining one clearly outperformed
the baselines but did not consistently outperform a well-tuned nonneural linear ranking method. Overall, our work sheds light on a
number of potential problems in today’s machine learning scholarship and calls for improved scientific practices in this area.},
note = {Pre SFI},
keywords = {Collaborative filtering, General and referance, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement},
pubstate = {published},
tppubtype = {conference}
}
Deep learning techniques have become the method of choice for
researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in
general, it has, as a result, become difficult to keep track of what
represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications
point out problems in today’s research practice in applied machine
learning, e.g., in terms of the reproducibility of the results or the
choice of the baselines when proposing new models.
In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically,
we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however
turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or
graph-based techniques. The remaining one clearly outperformed
the baselines but did not consistently outperform a well-tuned nonneural linear ranking method. Overall, our work sheds light on a
number of potential problems in today’s machine learning scholarship and calls for improved scientific practices in this area.
researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in
general, it has, as a result, become difficult to keep track of what
represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications
point out problems in today’s research practice in applied machine
learning, e.g., in terms of the reproducibility of the results or the
choice of the baselines when proposing new models.
In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically,
we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however
turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or
graph-based techniques. The remaining one clearly outperformed
the baselines but did not consistently outperform a well-tuned nonneural linear ranking method. Overall, our work sheds light on a
number of potential problems in today’s machine learning scholarship and calls for improved scientific practices in this area.
2018
Malte Ludewig; Dietmar Jannach
Evaluation of Session-based Recommendation Algorithms Journal Article
In: User-Modeling and User-Adapted Interaction, vol. 28, no. 4-5, pp. 331-390, 2018, (Pre SFI).
Abstract | BibTeX | Tags: Evaluation, General and referance, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement | Links:
@article{Ludewig2018,
title = {Evaluation of Session-based Recommendation Algorithms},
author = {Malte Ludewig and Dietmar Jannach},
url = {https://arxiv.org/pdf/1803.09587.pdf},
doi = {10.1007/s11257-018-9209-6},
year = {2018},
date = {2018-03-26},
journal = {User-Modeling and User-Adapted Interaction},
volume = {28},
number = {4-5},
pages = {331-390},
abstract = {Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user's immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like GRU4REC, factorized Markov model approaches such as FISM or FOSSIL, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today's more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.},
note = {Pre SFI},
keywords = {Evaluation, General and referance, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement},
pubstate = {published},
tppubtype = {article}
}
Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user's immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like GRU4REC, factorized Markov model approaches such as FISM or FOSSIL, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today's more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.
Massimo Quadrana; Paolo Cremonesi; Dietmar Jannach
Sequence-Aware Recommender Systems Journal Article
In: ACM Computing Surveys, vol. 51, no. 4, pp. 1-35, 2018, (Pre SFI).
Abstract | BibTeX | Tags: Collaborative filtering, Computing Methodology, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement | Links:
@article{Quadrana2018,
title = {Sequence-Aware Recommender Systems},
author = {Massimo Quadrana and Paolo Cremonesi and Dietmar Jannach},
url = {https://arxiv.org/pdf/1802.08452.pdf},
year = {2018},
date = {2018-02-23},
journal = {ACM Computing Surveys},
volume = {51},
number = {4},
pages = {1-35},
abstract = {Recommender systems are one of the most successful applications of data mining and machine learning technology in practice. Academic research in the field is historically often based on the matrix completion problem formulation, where for each user-item-pair only one interaction (e.g., a rating) is considered. In many application domains, however, multiple user-item interactions of different types can be recorded over time. And, a number of recent works have shown that this information can be used to build richer individual user models and to discover additional behavioral patterns that can be leveraged in the recommendation process. In this work we review existing works that consider information from such sequentially-ordered user- item interaction logs in the recommendation process. Based on this review, we propose a categorization of the corresponding recommendation tasks and goals, summarize existing algorithmic solutions, discuss methodological approaches when benchmarking what we call sequence-aware recommender systems, and outline open challenges in the area.},
note = {Pre SFI},
keywords = {Collaborative filtering, Computing Methodology, Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement},
pubstate = {published},
tppubtype = {article}
}
Recommender systems are one of the most successful applications of data mining and machine learning technology in practice. Academic research in the field is historically often based on the matrix completion problem formulation, where for each user-item-pair only one interaction (e.g., a rating) is considered. In many application domains, however, multiple user-item interactions of different types can be recorded over time. And, a number of recent works have shown that this information can be used to build richer individual user models and to discover additional behavioral patterns that can be leveraged in the recommendation process. In this work we review existing works that consider information from such sequentially-ordered user- item interaction logs in the recommendation process. Based on this review, we propose a categorization of the corresponding recommendation tasks and goals, summarize existing algorithmic solutions, discuss methodological approaches when benchmarking what we call sequence-aware recommender systems, and outline open challenges in the area.
2017
Alain D. Starke; Martijn C. Willemsen; Chris C.P. Snijders
Effective User Interface Designs to Increase Energy-efficient Behavior in a Rasch-based Energy Recommender System Conference
no. August 2017, 2017.
Abstract | BibTeX | Tags: Applied computing, Human-centered computing, Information Systems | Links:
@conference{Starke2017,
title = {Effective User Interface Designs to Increase Energy-efficient Behavior in a Rasch-based Energy Recommender System},
author = {Alain D. Starke and Martijn C. Willemsen and Chris C.P. Snijders},
url = {https://dl.acm.org/doi/abs/10.1145/3109859.3109902},
doi = {10.1145/3109859.3109902},
year = {2017},
date = {2017-08-27},
number = {August 2017},
pages = {1-9},
abstract = {People often struggle to find appropriate energy-saving measures to take in the household. Although recommender studies show that tailoring a system's interaction method to the domain knowledge of the user can increase energy savings, they did not actually tailor the conservation advice itself. We present two large user studies in which we support users to make an energy-efficient behavioral change by presenting tailored energy-saving advice. Both systems use a one-dimensional, ordinal Rasch scale, which orders 79 energy-saving measures on their behavioral difficulty and link this to a user's energy-saving ability for tailored advice. We established that recommending Rasch-based advice can reduce a user's effort, increase system support and, in turn, increase choice satisfaction and lead to the adoption of more energy-saving measures. Moreover, follow-up surveys administered four weeks later point out that tailoring advice on its feasibility can support behavioral change.},
keywords = {Applied computing, Human-centered computing, Information Systems},
pubstate = {published},
tppubtype = {conference}
}
People often struggle to find appropriate energy-saving measures to take in the household. Although recommender studies show that tailoring a system's interaction method to the domain knowledge of the user can increase energy savings, they did not actually tailor the conservation advice itself. We present two large user studies in which we support users to make an energy-efficient behavioral change by presenting tailored energy-saving advice. Both systems use a one-dimensional, ordinal Rasch scale, which orders 79 energy-saving measures on their behavioral difficulty and link this to a user's energy-saving ability for tailored advice. We established that recommending Rasch-based advice can reduce a user's effort, increase system support and, in turn, increase choice satisfaction and lead to the adoption of more energy-saving measures. Moreover, follow-up surveys administered four weeks later point out that tailoring advice on its feasibility can support behavioral change.
Mehdi Elahi; Yashar Deldjoo; Farshad Bakhshandegan Moghaddam; Leonardo Cella; Stefano Cerada; Paolo Cremonesi
Exploring the semantic gap for movie recommendations Conference
Proceedings of the Eleventh ACM Conference on Recommender Systems, Association for Computing Machinery New York, 2017, (Pre SFI).
Abstract | BibTeX | Tags: Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement | Links:
@conference{Elahi2017,
title = {Exploring the semantic gap for movie recommendations},
author = {Mehdi Elahi and Yashar Deldjoo and Farshad Bakhshandegan Moghaddam and Leonardo Cella and Stefano Cerada and Paolo Cremonesi },
url = {https://dl.acm.org/doi/pdf/10.1145/3109859.3109908},
doi = {https://doi.org/10.1145/3109859.3109908},
year = {2017},
date = {2017-08-01},
booktitle = {Proceedings of the Eleventh ACM Conference on Recommender Systems},
pages = {326–330},
address = {New York},
organization = {Association for Computing Machinery},
abstract = {In the last years, there has been much attention given to the semantic gap problem in multimedia retrieval systems. Much effort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it.
In this paper, we explore a different point of view by leveraging the gap between low-level and high-level features. We experiment with a recent approach for movie recommendation that extract low-level Mise-en-Scéne features from multimedia content and combine it with high-level features provided by the wisdom of the crowd.
To this end, we first performed an offline performance assessment by implementing a pure content-based recommender system with three different versions of the same algorithm, respectively based on (i) conventional movie attributes, (ii) mise-en-scene features, and (iii) a hybrid method that interleaves recommendations based on movie attributes and mise-en-scene features. In a second study, we designed an empirical study involving 100 subjects and collected data regarding the quality perceived by the users. Results from both studies show that the introduction of mise-en-scéne features in conjunction with traditional movie attributes improves both offline and online quality of recommendations.},
note = {Pre SFI},
keywords = {Information Systems, Recommender systems, WP2: User Modeling Personalization and Engagement},
pubstate = {published},
tppubtype = {conference}
}
In the last years, there has been much attention given to the semantic gap problem in multimedia retrieval systems. Much effort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it.
In this paper, we explore a different point of view by leveraging the gap between low-level and high-level features. We experiment with a recent approach for movie recommendation that extract low-level Mise-en-Scéne features from multimedia content and combine it with high-level features provided by the wisdom of the crowd.
To this end, we first performed an offline performance assessment by implementing a pure content-based recommender system with three different versions of the same algorithm, respectively based on (i) conventional movie attributes, (ii) mise-en-scene features, and (iii) a hybrid method that interleaves recommendations based on movie attributes and mise-en-scene features. In a second study, we designed an empirical study involving 100 subjects and collected data regarding the quality perceived by the users. Results from both studies show that the introduction of mise-en-scéne features in conjunction with traditional movie attributes improves both offline and online quality of recommendations.
In this paper, we explore a different point of view by leveraging the gap between low-level and high-level features. We experiment with a recent approach for movie recommendation that extract low-level Mise-en-Scéne features from multimedia content and combine it with high-level features provided by the wisdom of the crowd.
To this end, we first performed an offline performance assessment by implementing a pure content-based recommender system with three different versions of the same algorithm, respectively based on (i) conventional movie attributes, (ii) mise-en-scene features, and (iii) a hybrid method that interleaves recommendations based on movie attributes and mise-en-scene features. In a second study, we designed an empirical study involving 100 subjects and collected data regarding the quality perceived by the users. Results from both studies show that the introduction of mise-en-scéne features in conjunction with traditional movie attributes improves both offline and online quality of recommendations.
2016
Belgin Mutlu; Eduardo Veas; Christoph Trattner
VizRec: Recommending Personalized Visualizations Journal Article
In: ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 6, no. 4, pp. 1-40, 2016, (Pre SFI).
Abstract | BibTeX | Tags: CCS Concepts, Collaborative filtering;, Content ranking, Human-centered computing, Information Systems, Personalization | Links:
@article{Mutlu2016,
title = {VizRec: Recommending Personalized Visualizations},
author = {Belgin Mutlu and Eduardo Veas and Christoph Trattner},
url = {https://www.christophtrattner.info/pubs/ACM-TIIS.pdf},
year = {2016},
date = {2016-01-01},
journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
volume = {6},
number = {4},
pages = {1-40},
abstract = {Visualizations have a distinctive advantage when dealing with the information overload problem: since they
are grounded in basic visual cognition, many people understand them. However, creating the appropriate
representation requires specific expertise of the domain and underlying data. Our quest in this paper is to
study methods to suggest appropriate visualizations autonomously. To be appropriate, a visualization has
to follow studied guidelines to find and distinguish patterns visually, and encode data therein. Thus, a
visualization tells a story of the underlying data; yet, to be appropriate, it has to clearly represent those aspects
of the data the viewer is interested in. Which aspects of a visualization are important to the viewer? Can
we capture and use those aspects to recommend visualizations? This paper investigates strategies to
recommend visualizations considering different aspects of user preferences. A multi-dimensional scale is used to
estimate aspects of quality for charts for collaborative filtering. Alternatively, tag vectors describing charts
are used to recommend potentially interesting charts based on content. Finally, a hybrid approach combines
information on what a chart is about (tags) and how good it is (ratings). We present the design principles
behind VizRec, our visual recommender. We describe its architecture, the data acquisition approach with a
crowd sourced study, and the analysis of strategies for visualization recommendation.},
note = {Pre SFI},
keywords = {CCS Concepts, Collaborative filtering;, Content ranking, Human-centered computing, Information Systems, Personalization},
pubstate = {published},
tppubtype = {article}
}
Visualizations have a distinctive advantage when dealing with the information overload problem: since they
are grounded in basic visual cognition, many people understand them. However, creating the appropriate
representation requires specific expertise of the domain and underlying data. Our quest in this paper is to
study methods to suggest appropriate visualizations autonomously. To be appropriate, a visualization has
to follow studied guidelines to find and distinguish patterns visually, and encode data therein. Thus, a
visualization tells a story of the underlying data; yet, to be appropriate, it has to clearly represent those aspects
of the data the viewer is interested in. Which aspects of a visualization are important to the viewer? Can
we capture and use those aspects to recommend visualizations? This paper investigates strategies to
recommend visualizations considering different aspects of user preferences. A multi-dimensional scale is used to
estimate aspects of quality for charts for collaborative filtering. Alternatively, tag vectors describing charts
are used to recommend potentially interesting charts based on content. Finally, a hybrid approach combines
information on what a chart is about (tags) and how good it is (ratings). We present the design principles
behind VizRec, our visual recommender. We describe its architecture, the data acquisition approach with a
crowd sourced study, and the analysis of strategies for visualization recommendation.
are grounded in basic visual cognition, many people understand them. However, creating the appropriate
representation requires specific expertise of the domain and underlying data. Our quest in this paper is to
study methods to suggest appropriate visualizations autonomously. To be appropriate, a visualization has
to follow studied guidelines to find and distinguish patterns visually, and encode data therein. Thus, a
visualization tells a story of the underlying data; yet, to be appropriate, it has to clearly represent those aspects
of the data the viewer is interested in. Which aspects of a visualization are important to the viewer? Can
we capture and use those aspects to recommend visualizations? This paper investigates strategies to
recommend visualizations considering different aspects of user preferences. A multi-dimensional scale is used to
estimate aspects of quality for charts for collaborative filtering. Alternatively, tag vectors describing charts
are used to recommend potentially interesting charts based on content. Finally, a hybrid approach combines
information on what a chart is about (tags) and how good it is (ratings). We present the design principles
behind VizRec, our visual recommender. We describe its architecture, the data acquisition approach with a
crowd sourced study, and the analysis of strategies for visualization recommendation.