An Introduction To Information Theory And Probabilistic Inference

This comprehensive compendium addresses a critical need in the AI and machine learning era by bridging foundational information theory (IT) concepts with practical applications in statistical learning. Unlike traditional IT textbooks, this volume emphasizes how IT principles, such as entropy and source coding, underpin modern machine learning techniques like cross-entropy, decision trees, and evidence-lower bounds.This unique book connects IT with probabilistic inference, illustrated through real-world applications, such as decoding LDPC codes for error correction. The inclusion of the Lea probabilistic programming package is particularly valuable for pedagogy, offering students a hands-on tool to solve numerical problems and reinforce theoretical concepts.The useful reference text benefits professionals, researchers, academics and students in the fields of calculus and probability, communications and information sciences.