The course: The course is intended as an introduction to information and coding theory with emphasis on the mathematical aspects. It is suitable for advanced undergraduate and graduate students in mathematics, applied mathematics, statistics, physics, computer science and electrical engineering.
Course content: Information measures - entropy, relative entropy, mutual information, and their properties. Typical sets and sequences, asymptotic equipartition property, data compression, Huffman codes. Channel coding and capacity: channel coding theorem. Differential entropy, Gaussian channel, Shannon-Nyquist theorem. Additional topics, which may include compressed sensing and/or elements of quantum information theory.
Material to be covered: Chapters 2, 3, 5, 7 and parts of chapters 8, 9 from the
Cover&Thomas book (material from other chapters may be touched on lightly,
but not covered systematically) plus some additional topics as indicated
under "Course Content."
The lectures will be devoted to introducing concepts, presentation of examples, but also to proofs or outlines of proofs.
While the emphasis will be on understanding concepts and on being able to analyze them in specific instances,
the students will be also expected to come up with simple rigorous arguments.
Weekly updates, course documents etc. will be posted on Canvas under Modules.
Grades: 394 level: homework 20%, midterm 25%, quizzes/attendance/class participation 15%, final 40%
494 level: homework 20%, midterm 25%, quizzes/attendance 10%, presentation/report 10%, final 35%
Exam Dates: Midterm: October 11 (tentative); Final: Mon, Dec 16, 8:30-11AM
Students with special needs should contact
Division of Student Affairs/Disability Resources.
Please keep in mind that accommodations are not retroactive.
Assigments etc: Homework will be assigned weekly to bi-weekly. Some assignments may require Mathematica or Matlab, but no expert knowledge of these will be required. There will be a midterm, a final exam, and occasional quizzes. Additionally, students registered for the 494 level will be required to make a presentation and/or submit a written report on an approved topic, and to attend presentations made by fellow students. Presentation/report topics may involve information theory inequalities (chapter 17 from Cover&Thomas), an article from an information theory journal, or some mathematical topic underlying or relevant to material covered in the course. A very tentative schedule of assignments can be found here. Finalized assignments and solutions, and other documents will be progressively posted on Canvas under Assignments. However, the information posted under "tentative assignments" is a close approximation of the "finalized assignments" and so you are advised to start working on the problems as soon as the relevant material is covered in class. The scores for the assignments will be awarded primarily for "good faith effort." More information about modalities of quizzes, tests etc. can be found via Home page on Canvas.
Integrity: It is OK (and indeed encouraged) to discuss homework assignments with fellow students. However, any submitted work must be your own. Merely copying someone else's work is unethical, a waste of time, and may be penalized. Any substantive collaboration and/or usage of sources or AI tools has to be properly acknowledged. See CWRU academic integrity policy and Academic Integrity Board