An Eigensystem for Topic Term Weighting yields Fair and Effective Document Rankings
DOI:
https://doi.org/10.54195/irrj.23480Keywords:
group fairness, probability ranking principle, eigensystemAbstract
By incorporating fairness into the ranking function at retrieval time through topic term weights, this paper suggests a way to at the same time achieve effective and fair rankings in document collections. The topic term weights are calculated using the eigensystem of a matrix that linearly combines effectiveness and fairness matrices in a single matrix, whose main eigenvector provides the topic term weights. The latter can then be utilized as a new topic representation, thus providing a new ranking. In this way, topic word selection and weighting can influence ranking relevance as well as exposure, since the topic term weights are adjusted to benefit unprotected groups while minimizing the impact on effectiveness. The experiments described in the paper demonstrate that the proposed approach works.
Downloads
References
Ricardo Baeza-Yates. Bias on the web. Communication of the ACM, 61(6):54–61, 2018.
Aparna Balagopalan, Abigail Z. Jacobs, and Asia J. Biega. The role of relevance in fair ranking. In Proceedings of SIGIR, pages 2650–2660, 2023. doi: 10.1145/3539618.3591933.
Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. Equity of attention: Amortizing individual fairness in rankings. In Proceedings of SIGIR, pages 405–414. ACM, 2018. doi: 10.1145/3209978.3210063.
Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Morteza Zihayat, and Ebrahim Bagheri. On the orthogonality of bias and utility in ad hoc retrieval. In Proceedings of SIGIR, SIGIR ’21, pages 1748–1752, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380379. doi: 10.1145/3404835.3463110.
Reuben Binns. On the apparent conflict between individual and group fairness. In Proceedings of FAccT, 2020.
Lidia Ceriani and Paolo Verme. Gini on mutability. METRON, 82(3):269–292, 2024. doi: 10.1007/s40300-024-00279-2.
Olivier Chapelle, Shihao Ji, Ciya Liao, Emre Velipasaoglu, Larry Lai, and Su-Lin Wu. Intent-based diversification of web search results: metrics and algorithms. Information Retrieval, 14(6):572–592, 2011.
Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. This is not what we ordered: Exploring why biased search result rankings affect user attitudes on debated topics. In Proceedings of SIGIR, pages 295–305, 2021. doi: 10.1145/3404835.3462851.
Michael D. Ekstrand, Graham McDonald, Amifa Raj, and Isaac Johnson. Overview of the TREC 2021 fair ranking track. In The Thirtieth Text REtrieval Conference (TREC 2021) Proceedings, 2022a.
Michael D. Ekstrand, Graham McDonald, Amifa Raj, and Isaac Johnson. TREC 2022 Fair Ranking Track Participant Instructions. https://fair-trec.github.io/docs/Fair_Ranking_2022_Participant_Instructions.pdf, 2022b.
Michael D. Ekstrand, Graham McDonald, Amifa Raj, and Isaac Johnson. Overview of the TREC 2022 fair ranking track. In The Thirtieth Text REtrieval Conference (TREC 2021) Proceedings, 2023.
Yi Fang, Ashudeep Singh, and Zhiqiang Tao. Fairness in search systems. Foundations and Trends in Information Retrieval, 18(3):262–416, 2024. doi: 10.1561/1500000101. Corrado Gini. Variabilità e mutabilità. Tipografia di Paolo Cuppin, 1912.
Christina Ilvento, Meena Jagadeesan, and Shuchi Chawla. Multi-category fairness in sponsored search auctions. In Proceedings of FAccT, FAT* ’20, pages 348–358. ACM, 2020. ISBN 9781450369367. doi: 10.1145/3351095.3372848.
Thomas Jaenich, Graham McDonald, and Iadh Ounis. Fairness-aware exposure allocation via adaptive reranking. In Proceedings of SIGIR, SIGIR ’24, pages 1504–1513, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704314. doi: 10.1145/3626772.3657794.
Thomas Jaenich, Graham McDonald, and Iadh Ounis. Fair exposure allocation using generative query expansion. In Claudia Hauff, Craig Macdonald, Dietmar Jannach, Gabriella Kazai, Franco Maria Nardini, Fabio Pinelli, Fabrizio Silvestri, and Nicola Tonellotto, editors, Advances in Information Retrieval, pages 267–281, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031-88717-8.
K. Jarvëlin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422–446, 2002.
Till Kletti, Jean-Michel Renders, and Patrick Loiseau. Introducing the expohedron for efficient pareto-optimal fairness-utility amortizations in repeated rankings. In Proceedings of WWW, pages 498–507, 2022. doi: 10.1145/3488560.3498490.
Graham McDonald, Craig Macdonald, and Iadh Ounis. Search results diversification for effective fair ranking in academic search. Information Retrieval, 25(1):1–26, March 2022. doi: 10.1007/s10791-021-09399-z.
Massimo Melucci. A model of the relationship between the variations of effectiveness and fairness in information retrieval. Discover Computing, 27(3), 2024a.
Massimo Melucci. On the trade-off between ranking effectiveness and fairness. Expert Systems with Applications, 241, 2024b.
Massimo Melucci. Preference eigensystems for fair ranking. Expert Systems with Applications, 269:126324, 2025a.
doi: 10.1016/j.eswa.2024.126324. https://www.sciencedirect.com/science/article/pii/S0957417424031919
Massimo Melucci. Three methods for fair ranking of multiple protected items. Scientific Reports, to appear, 2025b.
Stefano Mizzaro. Relevance: The whole history. Journal of the American Society for Information Science, 48(9):810–832, 1997.
Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. Controlling fairness and bias in dynamic learning-to-rank. In Proceedings of SIGIR, pages 429–438. ACM, 2020. doi: 10.1145/3397271.3401100.
Amifa Raj and Michael D. Ekstrand. Comparing fair ranking metrics, Arxiv. 2022. https://arxiv.org/abs/2009.01311
S.E. Robertson, M.E. Maron, and W.S. Cooper. Probability of relevance: A unification of two competing models for document retrieval. Information Technology: Research and Development, 1:1–21, 1982.
Stephen E. Robertson. The probability ranking principle in information retrieval. Journal of Documentation, 33(4):294–304, 1977.
Tetsuya Sakai, Jin Young Kim, and Inho Kang. A versatile framework for evaluating ranked lists in terms of group fairness and relevance. ACM Transactions on Information Systems, 42(1), 2023. doi: 10.1145/3589763.
G. Salton. Automatic Information Organization and Retrieval. Mc Graw Hill, New York, NY, 1968.
T. Saracevic. Relevance: a review of and a framework for thinking on the notion in information science. Journal of the American Society for Information Science, 26:321–343, 1975.
T. Saracevic. Relevance: a review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance. Journal of the American Society for Information Science and Technology, 58(3):1915–1933, 2007a.
T. Saracevic. Relevance: a review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance. Journal of the American Society for Information Science and Technology, 58(13):2126–2144, 2007b.
E. Seneta. Non-negative Matrices and Markov Chains. Springer, 2006.
Cornelis Joost Van Rijsbergen. Information Retrieval. Butterworths, London, 1979.
E.M. Voorhees and D.K. Harman, editors. TREC: Experiment and Evaluation in Information Retrieval. The MIT Press, Cambridge, MA, 2005.
Lin Xiao, Zhang Min, Zhang Yongfeng, Gu Zhaoquan, Liu Yiqun, and Ma Shaoping. Fairness-aware group recommendation with Pareto-efficiency. In Proceedings of RecSys, pages 107–115, 2017.
Meike Zehlike, Ke Yang, and Julia Stoyanovich. Fairness in ranking, part i: Score-based ranking. ACM Computing Surveys, 55(6), December 2022a. ISSN 0360-0300. doi: 10.1145/3533379.
Meike Zehlike, Ke Yang, and Julia Stoyanovich. Fairness in ranking, part ii: Learning-to-rank and recommender systems. ACM Computing Surveys, 55(6), December 2022b. ISSN 0360-0300. doi: 10.1145/3533380.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Massimo Melucci (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
CC-BY 4.0
