Topics of interest

The papers and discussion at the workshop are expected to address –though need not be limited to– the following topics:

  • Recommendation quality dimensions
    • Effective accuracy, ranking quality
    • Novelty, diversity, unexpectedness, serendipity
    • Utility, gain, cost, risk, benefit
    • Robustness, confidence, coverage, usability, etc.
  • Matching metrics to tasks, needs, and goals
    • User satisfaction, user perception, human factors
    • Business-oriented evaluation
    • Multiple objective optimization, user engagement
    • Quality of service, quality of experience
  • Evaluation methodology and experimental design
    • Definition and evaluation of new metrics, studies of existing ones
    • Adaptation of methodologies from related fields: IR, Machine Learning, HCI, etc.
    • Evaluation theory
  • Practical aspects of evaluation
    • Offline and online experimental approaches
    • Simulation-based evaluation
    • Datasets and benchmarks
    • Validation of metrics
RUE 2012 - 1st International Workshop on Recommendation Utility Evaluation: Beyond RMSE
6th ACM Conference on Recommender Systems (RecSys 2012)
Dublin, Ireland, 9 or 13 September 2012