An Inferential Measure of Dependence Between Two Systems Using Bayesian Model Comparison
Guillaume Marrelec, Alain Giron · IEEE Transactions on Systems Man and Cybernetics Systems · 2024
We propose to quantify dependence between two systems$\mathcal {X}$and$\mathcal {Y}$in a dataset D based on the Bayesian comparison of two models: one,$H_{0}$, of statistical independence and another one,$H_{1}$, of dependence. In this framework, dependence between$\mathcal {X}$and$\mathcal {Y}$in D, denoted${\mathfrak {B}}_{\mathrm {}} (\mathcal {X}, \mathcal {Y} | D)$, is quantified as$P (H_{1} | D)$, the posterior probability for the model of dependence given D, or any strictly increasing function thereof. It is therefore a measure of the evidence for dependence between$\mathcal {X}$and$\mathcal {Y}$as modeled by$H_{1}$and observed in D. We review several statistical models and reconsider standard results in the light of${\mathfrak {B}}_{\mathrm {}} (\mathcal {X}, \mathcal {Y} | D)$as a measure of dependence. Using simulations, we focus on two specific issues: 1) the effect of noise and 2) the behavior of${\mathfrak {B}}_{\mathrm {}} (\mathcal {X}, \mathcal {Y} | D)$when$H_{1}$has a parameter coding for the intensity of dependence. We then derive some general properties of${\mathfrak {B}}_{\mathrm {}} (\mathcal {X}, \mathcal {Y} | D)$, showing that it quantifies the information contained in D in favor of$H_{1}$versus$H_{0}$. While some of these properties are typical of what is expected from a valid measure of dependence, others are novel and naturally appear as desired features for specific measures of dependence, which we call inferential. We finally put these results in perspective; in particular, we discuss the consequences of using the Bayesian framework as well as the similarities and differences between${\mathfrak {B}}_{\mathrm {}} (\mathcal {X}, \mathcal {Y} | D)$and mutual information.