Case Western Reserve University

Mathematics Department


Under Construction
Last updated on 3/14/2016

Quick link to assignments



The course: The course is intended as an introduction to information and coding theory with emphasis on the mathematical aspects. It is suitable for advanced undergraduate and graduate students in mathematics, applied mathematics, statistics, physics, computer science and electrical engineering.

Course content: Information measures - entropy, relative entropy, mutual information, and their properties. Typical sets and sequences, asymptotic equipartition property, data compression. Channel coding and capacity: channel coding theorem. Differential entropy, Gaussian channel, Shannon-Nyquist theorem. Information theory inequalities (400 level). Additional topics, which may include compressed sensing and/or elements of quantum information theory.

Material to be covered: Chapters 2, 3, 5, 7 and parts of chapters 8, 9 from the Cover&Thomas book (material from other chapters may be touched on lightly, but not covered systematically) plus some additional topics as indicated under "Course Content."

Grades: 394 level: homework 20%, midterm 25%, quizzes/attendance/class participation 15%, final 40%
494 level: homework 20%, midterm 25%, quizzes/attendance 10%, presentation/report 15%, final 30%
Exam Dates: Midterm: Feb. 24; Final: May 2, 8:30-11:30 a.m
Students with special needs should contact Educational Services for Students.

Assigments etc: Homework will be assigned approximately every 2 weeks. Some assignments may require Mathematica or Matlab, but no expert knowledge of these will be required. There will be a midterm, a final exam, and occasional quizzes. Additionally, students registered for the 494 level will be required to make a presentation and/or submit a written report on an approved topic, and to attend presentations made by fellow students. Presentation/report topics may involve information theory inequalities (chapter 17 from Cover&Thomas), an article from an information theory journal, or some mathematical topic underlying or relevant to material covered in the course.
Regularly updated assignments, exam dates and such will be posted here. Solutions and some other documents will be progressively posted on  Blackboard.

Integrity: It is OK (and indeed encouraged) to discuss homework assignments with fellow students. However, any submitted work must be your own. Merely copying someone else's work is unethical, a waste of time, and may be penalized. (This includes copying solutions found on the internet.) Any substantive collaboration and/or usage of sources has to be acknowledged. See CWRU academic integrity policy.

Information on this page (or a version of it) in PDF format

Dr. Szarek's Home Page
Math/Stat Graduate Courses
Email webmaster -- Copyright 1993-2015 CWRU