Robust Rank Aggregation and Its Applications
Yuangang Pan · UTS ePRESS (University of Technology Sydney) · 2019
Rank aggregation (RA) refers to the task of recovering the total order over a set of items, given a collection of preferences over the items.The flexible collection of preferences enables successful application of RA in various fields, e.g., image rating and bioinformatics.A basic assumption underlying the vanilla RA is that all preferences are provided by homogeneous users.However, this assumption is rarely satisfied in real applications, due to the complex real situation.Therefore, RA usually suffers from model misspecification, namely the inconsistency between the collected preferences and the homogeneity assumption.Another challenge associated with RA is the scalability issue.In particular, RA usually involves ranking over tens of thousands of items, leading to an exponential volume of preferences for aggregation.Therefore, an inappropriate inference method would limit the application of the proposed model.This thesis considered RA under model misspecification in the following three scenarios:• In a crowdsourcing scenario, sufficient annotations from each user are available, which enables exploration of user heterogeneity to account for model misspecification.Therefore, I proposed a reliable CrowdsOUrced Plackett-LucE (COUPLE) model, which introduces an uncertainty vector to make a fine-grained categorization of users.Meanwhile, a general Bayesian Moment Matching (OnlineGBMM) was proposed, to ensure an analytic Bayesian update with an almost twice differentiable likelihood function.• In a general setting, typical model augmentation methods would cause overfitting, because insufficient annotations from each user are available.Inspired by the distributional robust literature, I proposed CoarsenRank, which performs regular RA over a neighborhood of preferences.The resultant inference would enjoy robustness against model misspecification.To this end, I first defined a neighborhood of the rank dataset using relative entropy.Then, I instantiated CoarsenRank with three popular probability ranking models and discussed the optimization strategies. vi• RA for mental fatigue monitoring.Common practices for mental fatigue monitoring refer to predicting the reaction time (RT) by aggregating the EEG signal from multiple heterogeneous EEG channels.Let us consider the RT as the item score and view each EEG channel as a user.The mental fatigue monitoring task could be formulated as RA under model misspecification, particularly in a crowdsourcing scenario.To address this problem, a Self-Weight Ordinal REgression (SWORE) model with Brain Dynamics table (BDtable) is proposed.The SWORE model could give a reliable evaluation of brain dynamics preferences from multiple channels, while the BDtable is employed to calibrate the SWORE model by utilizing the proposed online generalized Bayesian moment matching (OGMM) algorithm.5.2 Test accuracy (in %, the larger the better) w.r.t hyperparameter (α, β ) with hyperparameter (µ, Σ) fixed to (10 -2 , 10 -4 ), dropout rate θ = 0.5, data augmentation number T = 1.The best parameter settings are marked in gray.Some parameter settings do not consistently perform very well and may fail on some participants (marked in bold). . . . . . . . .