|
This book provides a
comprehensive introduction and reference guide to the minimum
description length (MDL) Principle, a powerful method of inductive
inference that holds that the best explanation, given a limited set of
observed data, is the one that permits the greatest compression of the
data. The central concepts of this theory are explained in great
detail. The book should
be accessible to researchers dealing with inductive inference in
diverse areas including statistics, machine learning, data mining,
biology, econometrics, and experimental psychology, as well as
philosophers interested in the foundations of statistics.
The book consists of four parts.
Part I provides a basic introduction to MDL and an overview of the
concepts in statistics and information theory needed to
understand MDL. Part II treats universal coding, the
information-theoretic notion on which MDL is built, and
Part III gives a formal treatment of MDL theory as a
theory of inductive inference based on universal
coding. Part IV provides a comprehensive overview of the
statistical theory of exponential families with an
emphasis on their information-theoretic properties. The
book contains some new results that have not been
published elsewhere.
|