Information Theory, Inference & Learning Algorithms

Description

Book Jacket: > This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks. Publisher Description: > This textbook offers comprehensive coverage of Shannon's theory of information as well as the theory of neural networks and probabilistic data modelling. It includes explanations of Shannon's important source encoding theorem and noisy channel theorem as well as descriptions of practical data compression systems. Many examples and exercises make the book ideal for students to use as a class textbook, or as a resource for researchers who need to work with neural networks or state-of-the-art error-correcting codes.

Excerpt

You cannot do inference without making assumptions.

First Sentence

In this chapter we discuss how to measure the information content of the outcome of a random experiment.

Subjects

Links

• http://www.inference.phy.cam.ac.uk/mackay/itila/book.html

Other Editions

• Information Theory, Inference & Learning AlgorithmsHardcoverCambridge University Press2003-01-01
• INFORMATION THEORY, INFERENCE, AND LEARNING ALGORITHMS.CAMBRIDGE UNIV PRESS2003-01-01
• Information Theory, Inference and Learning AlgorithmsTBS2004-01-01