Last updated on

*Class #: 10867/10868/11162/11163**Class meets:*MWF 2:00-2:50, Sears 354

*Instructor:*Stanislaw J. Szarek (pronounced 'Shareck', more or less)*Office:*Yost 332*Office hours:*Currently Tu 1:30-2:30, W 1-2 or by appointment (or just drop by)*Phone:*216-368-2913*E-mail:*szarek AT case.edu Asking questions by e-mail is particularly encouraged, even at strange hours.*WWW Home Page:*http://www.cwru.edu/artsci/math/szarek/*Class WWW Page (this page):*http://www.cwru.edu/artsci/math/szarek/MATH394/*Prerequisite:*MATH 223 (multivariate calculus) and MATH 380 (intro to probability) or equivalent (graduate students registering at 494 level will be presumed to satisfy the prerequisites and given permissions to enroll on request, but this does not relieve them from having to know the material).

*Recommended:*some familiarity with linear algebra, for example MATH 201 or MATH 307*Primary text:*: "Elements of Information Theory" by T. M. Cover and J. A. Thomas, 2nd Edition, New York, Wiley, 2006.*Note:*If you want to use an earlier edition, you may try but AT YOUR OWN risk (different enumeration of chapters and many new exercises in the 2nd edition).

**The course:** The course is intended as an introduction to information and coding theory with emphasis on the mathematical aspects. It is suitable for advanced undergraduate and graduate students in mathematics, applied mathematics, statistics, physics, computer science and electrical engineering.

*Course content: * Information measures - entropy, relative entropy, mutual information, and their properties. Typical sets and sequences, asymptotic equipartition property, data compression. Channel coding and capacity: channel coding theorem. Differential entropy, Gaussian channel, Shannon-Nyquist theorem. Information theory inequalities (400 level). Additional topics, which may include compressed sensing and/or elements of quantum information theory.

**Material to be covered: ** Chapters 2, 3, 5, 7 and parts of chapters 8, 9 from the
Cover&Thomas book (material from other chapters may be touched on lightly,
but not covered systematically) plus some additional topics as indicated
under "Course Content."

**Grades:** *394 level*: homework 20%, midterm 25%, quizzes/attendance/class participation 15%, final 40%

*494* level: homework 20%, midterm 25%, quizzes/attendance 10%, presentation/report 15%, final 30%

*Exam Dates*: Midterm: Feb. 24; Final: May 2, 8:30-11:30 a.m

Students with special needs should contact
Educational Services for Students.

**Assigments etc:** Homework will be assigned approximately every 2 weeks. Some assignments may require Mathematica or Matlab, but no expert knowledge of these will be required. There will be a midterm, a final exam, and occasional quizzes. Additionally, students registered for the 494 level will be required to make a presentation and/or submit a written report on an approved topic, and to attend presentations made by fellow students. Presentation/report topics may involve information theory inequalities (chapter 17 from Cover&Thomas), an article from an information theory journal, or some mathematical topic underlying or relevant to material covered in the course.

Regularly updated assignments, exam dates and such will be posted
**here**. Solutions and some other documents will be progressively posted on
Blackboard.

**Integrity:** It is OK (and indeed encouraged) to discuss homework assignments with fellow students. However, **any** submitted work must be your own.
Merely copying someone else's work is unethical,
a waste of time, and may be penalized.
(This includes copying solutions found on the internet.)
Any substantive collaboration and/or usage of sources has to be acknowledged.
See
CWRU academic integrity policy.

**Information on this page (or a version of it) in PDF format**

Assignments

Dr. Szarek's Home Page

Math/Stat Graduate Courses

Email webmaster -- Copyright 1993-2015 CWRU