FairBook: Fairness in Book Recommendation

Results

The performance of models on different evaluation metrics.

	MAE	MSE	RMSE	AUC	MAP	MRR	F1@5	F1@10	F1@20	F1@50	Train(s)	Test(s)
Random	--	--	--	--	--	--	--	--	--	--	--	--
MostPop	2.5096	9.98	2.7869	0.6482	0.0153	0.0369	0.0105	0.0109	0.0105	0.009	0.0132	18.3305
UserKNN	--	--	--	--	--	--	--	--	--	--	--	--
BPR	6.603	46.9219	6.697	0.6718	0.0155	0.0369	0.0105	0.0109	0.0104	0.0089	2.535	24.6737
MF	1.296	2.8645	1.4443	0.5176	0.0023	0.0061	0.0011	0.0013	0.0016	0.0014	0.5621	25.1844
PMF	1.4408	3.2561	1.5808	0.5992	0.0051	0.0117	0.0028	0.0037	0.0038	0.0039	3.6371	29.6608
NMF	2.2118	6.6627	2.368	0.5263	0.0015	0.003	0.0003	0.0005	0.0006	0.0007	0.9117	24.5384
WMF	5.7309	37.1711	5.8926	0.7267	0.0169	0.0343	0.0113	0.0154	0.0148	0.0123	67.7862	29.1564
PF	6.7383	48.6651	6.83	0.6561	0.0125	0.0275	0.0089	0.0102	0.0101	0.008	194.7136	31.0155
NeuMF	6.7514	48.8437	6.8423	0.671	0.015	0.0359	0.0098	0.0097	0.0101	0.0086	9.8069	47.7535
VAECF	6.7514	48.8437	6.8423	0.7796	0.0222	0.0471	0.0145	0.0168	0.0163	0.0137	218.3885	36.7076

	Precision				Recall				NDCG
	@5	@10	@20	@50	@5	@10	@20	@50	@5	@10	@20	@50
Random	--	--	--	--	--	--	--	--	--	--	--	--
MostPop	0.0097	0.008	0.0066	0.005	0.0161	0.0263	0.0422	0.0771	0.0151	0.0185	0.0237	0.0333
UserKNN	--	--	--	--	--	--	--	--	--	--	--	--
BPR	0.0095	0.008	0.0065	0.005	0.0163	0.0263	0.0417	0.0762	0.0151	0.0185	0.0236	0.0331
MF	0.0012	0.001	0.001	0.0008	0.0014	0.0027	0.0057	0.0107	0.0015	0.0019	0.0029	0.0043
PMF	0.0026	0.0027	0.0024	0.0021	0.0042	0.0089	0.0164	0.0346	0.0033	0.005	0.0074	0.0123
NMF	0.0003	0.0004	0.0004	0.0004	0.0004	0.001	0.0021	0.0051	0.0004	0.0007	0.001	0.0019
WMF	0.01	0.0109	0.0091	0.0068	0.0182	0.04	0.0646	0.1118	0.0131	0.0214	0.0294	0.0422
PF	0.008	0.0074	0.0063	0.0044	0.0137	0.0247	0.0417	0.0708	0.0113	0.0154	0.021	0.0289
NeuMF	0.0092	0.0072	0.0063	0.0048	0.015	0.0227	0.0402	0.0741	0.0146	0.017	0.0227	0.032
VAECF	0.0132	0.0122	0.0101	0.0076	0.0223	0.041	0.0676	0.1252	0.0197	0.0266	0.0353	0.0505

Paper

Citation and Paper

@inproceedings{naghiaei2022fairbook,
title = {The Unfairness of Popularity Bias in Book Recommendation},
author={Naghiaei, Mohammadmehdi and Rahmani, Hossein A. and Dehghan, Mahdi},
booktitle = {Third International Workshop on Algorithmic Bias in Search and Recommendation (Bias@ECIR 2022)},
year = {2022},
url = {https://arxiv.org/abs/XXX},
}

Citation (Bibtex)

Fairness Analysis

Experimental results on different user groups based on MAE, Precision, Recall, and NDCG.

	Niche				Diverse				BestSeller
	MAE	Precision	Recall	NDCG	MAE	Precision	Recall	NDCG	MAE	Precision	Recall	NDCG
MostPop	2.783	0.0017	0.0077	0.0043	2.492	0.0084	0.0232	0.0169	2.2878	0.0128	0.0553	0.0382
BPR	6.7712	0.0017	0.0077	0.0044	6.5882	0.0084	0.0232	0.0169	6.4793	0.0128	0.0553	0.0382
MF	1.2754	0.0007	0.0024	0.0013	1.3005	0.0012	0.0026	0.002	1.3022	0.0007	0.0033	0.0019
PFM	1.527	0.001	0.0043	0.0029	1.4242	0.003	0.0083	0.005	1.4068	0.0034	0.0155	0.0075
NMF	2.3132	0.0005	0.0012	0.0008	2.1915	0.0004	0.001	0.0008	2.1741	0.0001	0.0005	0.0002
WMF	6.127	0.0081	0.0394	0.0177	5.6899	0.0101	0.0292	0.0176	5.4596	0.0161	0.0756	0.0377
PF	6.8119	0.0017	0.0063	0.0035	6.715	0.0075	0.0219	0.0141	6.7388	0.013	0.0528	0.032
NeuMF	6.8139	0.0011	0.0046	0.0032	6.7316	0.0076	0.0207	0.0157	6.752	0.0119	0.0478	0.0354
VAECF	6.8139	0.0042	0.0152	0.0087	6.7316	0.0123	0.0342	0.0238	6.752	0.0201	0.0893	0.0542

Team

Mohammadmehdi Naghiaei

PhD Student

University of Southern California
naghiaei@usc.edu

Hossein A. Rahmani

Visiting Researcher

University College London
h.rahmani@ucl.ac.uk

Mahdi Dehghan

MSc in Artificial Intelligence

Shahid Beheshti University
mahdi.dehghan551@gmail.com

FairBook: Popularity Fairness in Book Recommendation

Analysis, Recommendation, Group Fairness

Analysis

Recommendation

Fairness