Statistical comparison of models

Pascal Bordé
Institut d'Astrophysique Spatiale

In their daily scientific activity, scientists model objects or phenomena. Very often they need to make a choice between various models (or hypotheses) considering old or recently acquired data. This is usually resolved by selecting as the best model the most likely one. But what does "most likely" really mean and how can this be computed? In this advanced course, we will review two approaches to model selection based on frequentist and Bayesian probability theories. In the first part, the analysis of actual interferometric observations of a faint companion around Theta Draconis will be used to review familiar statistical tools such as the chi-square test, as well as to introduce more advanced ones, such as the Aikake information criterion. We will then discuss the shortcomings of the frequentist (classical) theory, and proceed to introduce Bayes-Laplace statistical theory. In this second part, Bayesian model comparison will be illustrated by analyzing the radial velocity data for the star Gliese 581 that was the subject of a recent debate: how many planets are actually orbiting this star? Is there enough evidence - based on existing measurements - that a planet orbits in the habitable zone? We will conclude by recalling the pros and the cons of each theory, as well as pointing out a few common caveats.

Course slides available here.