Lecture Time and Location

Thursdays, 6:30PM - 9:20PM. CAT 77

Instructor

Prof. John MacLaren Walsh. Office: Bossone 203. Telephone: (215) 895-2360. Email: jwalsh@ece.drexel.edu

Office Hours: Wednesday 10AM-Noon.

Course Website

This class has a website: http://www.ece.drexel.edu/walsh/eces690/eces690.html. You will need to check it periodically throughout the semester for information concerning reading assignments, homework assignments, exam information, and projects.

Prerequisites

Courses covering probability and random variables such as ECES-521 and ECES-522. A first course on information theory, e.g. ECET-602 covering the fundamental limits for lossless compression, lossy compression, and channel coding; or an equivalent level of familiarity with this material.

Textbooks and Resources

Throughout the quarter we will make use of material from the following texts:

• Network Information Theory, A. El Gamal and Young-Han Kim, Cambridge Univ. Press, 2011.
• Information Theory and Network Coding, R. W. Yeung, Springer, 2008.
• Elements of Information Theory, 2nd. Ed, T. Cover and J. Thomas, Wiley, 2006.

Additionally, a significant amount of material will be drawn from journal publications available through IEEE explore. The primary “text” for the course will be the Course Lecture Notes which will be available in class and online (see schedule below).

Grading Policy

There will be graded homework assignments, a midterm exam, a final project, and a final exam. These will count towards the final grade as follows: homework (25%), midterm exam (25%), final exam (25%), final project (25%). Due to the few, yet condensed, nature of the lectures, late homework can not be accepted. If you wish to dispute the grading of a homework/project/exam, you must attach to the homework/project/exam a piece of paper outlining your complaint and return it to the instructor within 2 business days after the homework/project/exam is returned. Any disputation of grading which does not follow these guidelines will not be accepted, and may lead to a reduction in your grade.

Tentative Outline of Course Content

This course covers advanced topics in multiterminal information theory. The course will cover topics from multiterminal source coding and network coding over directed acyclic graphs. Multiterminal channel coding is left for a follow-on course.

• Lossless Distributed Source Coding (Slepian Wolf)
  • Joint Typicality
  • Random Binning Achievability Construction.
• Lossy Distributed Source Coding
  • Lossy Source Coding With Side Information (Wyner-Ziv)
  • The Berger Tung Bounds.
  • The CEO Problem.
  • The Multiple Descriptions Problem.
  • Successive Refinement.
• Network Coding (Wired Directed Acyclic Graph Variant)
  • Single Multicast
    • max-flow / min-cut bounds
    • Insufficiency of Routing
    • Sufficiency of Linear Codes
    • Random Linear Network Coding
  • Multiple Multicast Rate Region
    • Matroids & Insufficiency of Linear Codes
    • Region of Entropic Vectors & Shannon and Non-Shannon Information Inequalities
    • Implicit Capacity Region & LP Outer Bound

Course Schedule & Assignments

• April 5: Lossless Distributed Source Coding
  • Read chapter 10 in El Gamal and Kim.
• April 12: Strongly Typical Sequences and the Markov Lemma – Section 2
  • Read chapter 11 in El Gamal and Kim.
  • Hand in Homework 1
• April 19: Lossy Source Coding with Side Information – Section 3
  • Read chapter 12 in El Gamal and Kim
• April 26: Lossy Source Coding with Multiple Degraded Side Informations – Section 4
  • Read chapter 13 in El Gamal and Kim
• May 3: Berger Tung Bounds for Lossy Multiterminal Source Coding. CEO Problem. (Section 1) Successive Refinements.
  • Solve Take Home Midterm Exam. Due by email on Tuesday, May 8.
• May 10: Exam discussion. Berger Tung Bounds for Lossy Multiterminal Source Coding Cont’d. CEO Problem. (Section 1)
  • Read chapters 17-19 in Yeung. Optionally, read chapter 15 in El Gamal and Kim.
• May 17: Multiple Descriptions Problem. (Section 2)
  • Read chapters 13-15 and 21 in Yeung.
• May 24: Successive Refinement. (Section 3)
  • Revise Midterm Solutions.
• May 31: Network Coding on Directed Acyclic Graphs. Multiple Multicast Rate Region Converse.
  • Continue Revising Midterm Solutions.
• June 7: Network Coding on Directed Acyclic Graphs. Multiple Multicast Rate Region Achievability.
  • Work on Final Project.
• June 14: Final Exam (in class).
  • Hand in final project.

This document is current as of June 7, 2012.