Page 6 - Karlstad News Today : Breaking News, Live Updates & Top Stories | Vimarsana

Conditional entropy is an important concept that naturally arises in fields such as finance, sociology, and intelligent decision making when solving problems involving statistical inferences. Formally speaking, given two random variables X and Y, one is interested in the amount and direction of information flow between X and Y . It helps to draw conclusions about Y while only observing X. Conditional entropy H(Y |X) quantifies the amount of information flow from X to Y . In practice, calculating H(Y |X) exactly is infeasible. Current estimation methods are complex and suffer from estimation bias issues. In this paper, we present a simple Machine Learning based estimation method. Our method can be used to estimate H(Y |X) for discrete X and bi-valued Y. Given X and Y observations, we first construct a natural binary classification training dataset. We then train a supervised learning algorithm on this dataset, and use its prediction accuracy to estimate H(Y |X). We also present a simple

Karlstad universityMachine learningAutomated data miningBinary classificationOnditional entropyInformation theoryAchine learning in sociologyUpervised machine learning