Lecture Time and Location

Mondays, 6:00PM - 8:50PM. Curtis 344.

Instructor

Prof. John MacLaren Walsh. Office: Bossone 203. Telephone: (215) 895-2360. Email: jwalsh@ece.drexel.edu

Office Hours: Wednesday 10AM-noon. Other times available by email appointment.

Course Website

This class has a website: http://www.ece.drexel.edu/walsh/eces632/eces632.html. You will need to check it periodically throughout the semester for information concerning reading assignments, homework assignments, exam information, and projects.

Prerequisites

A graduate level deterministic digital signal processing course (e.g. ECES 631) covering the discrete time fourier transform, FIR and IIR filters, the discrete fourier transform, and discrete time fourier transform. Also, familiarity with probability, random variables, and random processes: including cumulative distribution functions (CDF), probability density functions (PDF), probability mass functions (PMF), markov chains (MC) and markov processes, stationary and wide sense stationary (WSS) discrete time random processes.

Textbooks

• Handouts. Available in class and online:
  • Lecture Notes 1: Review of Probability and Random Processes
  • Lecture Notes 2: Overview of Estimation Theory
  • Lecture Notes 3: Methods for Spectral Estimation
  • Lecture Notes 4: MMSE Linear Prediction
  • Lecture Notes 5: Estimation of Sinusoids in Noise & Related Array Processing
  • Lecture Notes 6: Inference in Hidden Markov Models.
• Statistical Signal Processing, vols. I and II, S. Kay, Prentice Hall, 1993,1998 ISBN: 0-13-345711-7,0-13-504135-X. Supplementary (Not Required)
• Detection, Estimation, and Modulation Theory, Vols I-IV, H. L. van Trees, J. Wiley Interscience, 2001. Supplementary (Not Required)
• Digital Spectral Analysis with Applications, S. Lawrence Marple, Jr., Prentice Hall, 1987 ISBN: 0-13-214149-3 025 Supplementary (Not Required)

Grading Policy

There will be graded homework assignments, one take home project, a midterm and a cumulative final exam. These will count towards the final grade as follows: homework (20%), take home projects (30%), midterm exam (20%), final exam (30%). Due to the few, yet condensed, nature of the lectures, late homework can not be accepted. If you wish to dispute the grading of a homework/project/exam, you must attach to the homework/project/exam a piece of paper outlining your complaint and return it to the instructor within 2 business days after the homework/project/exam is returned. Any disputation of grading which does not follow these guidelines will not be accepted, and may lead to a reduction in your grade.

Outline of Course Content

• Review of Probability and Random Processes
  • Probability Space.
  • Random Variables: PMFs, CDFs, PDFs, Common R.V.s, expectation, transformations between.
  • Conditional probability, conditional expectation, Bayes’ Theorem, total probability.
  • Stationary and Wide Sense Stationary Random processes.
  • Auto-regressive (AR), moving average (MA), and auto-regressive moving average (ARMA) time series models. Spectral Factorization.
• Some Prerequisite Estimation Theory
  • Problem setup and notion of sufficient statistics.
  • Maximum Likelihood Estimation
  • Bayesian Estimation
  • Unbiased Estimators
  • Minimum Variance Unbiased Estimators (MVUE)
  • Minimum Mean Squared Error Estimation (MMSE) & Linear MMSE
• Some Important Signal Processing Problem Families
  • Spectral Estimation
  • Linear Prediction
  • Source Separation & Independent Component Analysis (blind v.s. trained)
  • System Identification (blind v.s. trained)
  • Equalization (blind v.s. trained)
  • Adaptive Implementations of the Above
• Methods for Spectral Estimation
  • Periodogram (Schuster) & smooth version (Welch)
  • Correlogram (Blackman- Tukey) & smooth version (Bartlett)
  • Maximum Entropy (Burg)
• Linear Prediction
  • Analytical AR MMSE solution
  • Block “Efficient” computation with Levinson Durbin Algorithm.
  • (Efficient and Important Gradient Based) Adaptive Implementations.
• Estimation and Detection of Sinusoids in Noise
  • Prony’s Method
  • Pisarenko
  • MUSIC
• Hidden Markov Models
  • Review of Markov Chains: Recurrence, Transience, Communicating Classes, stationarity, ergodicity.
  • Inference in HMMs: Viterbi and forward backward algorithms.
  • Signal Processing Applications in Speech and Communications.

Assignment Schedule

This will be updated periodically throughout the semester, and is thus subject to change.

• January 11, 2010: Homework 1 due in class.
• January 18, 2010: Dr. Martin Luther King, Jr. Day. (University Holiday)
• January 25, 2010: Homework 2 due in class.
• February 8, 2010: Homework 3 due in class.
• February 15, 2010: Midterm exam. You will not be able to use your notes or book. The exam will test, in order of decreasing priority, Lecture 4, Lecture 2, Lecture 3, Lecture 1.
• March 1, 2010: Homework 4 due in class.
• March 8, 2010: Submit Final Project. Course Review.
• March 15, 2010: Final Exam (in class).

This document is current as of March 14, 2010.