Image and video processing: From Mars to Hollywood with a stop at the hospital
In this class you will look behind the scenes of image and video processing, from the basic and classical tools to the most modern and advanced algorithms.
|Jan 14th 2013 (9 weeks long)
Workload: 4 – 6 hours/ week
About the Course
What is image and video processing? Images and videos are everywhere, from those we take with our mobile devices and share with our friends to those that we receive from Mars and the ones we see in the movie theatre, without forgetting the whole ensemble of images of our bodies that are taken in hospital visits. Image and video processing is the art of working with such images and movies, from making it possible to store and transmit them to making those dark and blurry images look nice, as well as interpreting and analyzing the medical data and recognizing our friends’ faces in social pictures. This discipline is also fascinating because it uses tools from many areas of applied mathematics. In this class you will look behind the scenes of image and video processing, from the basic and classical tools to the most modern and advanced algorithms.The course will start with an introduction to the basics of image formation and the fundamental concepts that translate a physical scene into a digital image. We will then describe the underlying concepts of image compression, the enabling technology that makes it possible for images to be sent from Mars and videos to be stored in our mobile phones. We will cover the most fundamental tools in image enhancement, showing how simple tools can significantly improve images. Both geometric and non-geometric tools as well as spatial and non-spatial operations will be presented. Details on image segmentation will be provided, one of the most fundamental and useful problems in image processing. The above topics will be extended to color images and video. Once we have covered the fundamentals, which both provide the basis for modern image and video processing and serve many important applications until today, we will move into recent progress in the area, covering image inpainting (how to remove objects from images and video), image processing via sparse modeling and compressed sensing, geometric partial differential equations for image analysis, image processing for HIV and virus research, and image processing for neurosurgery and other medical applications.
About the Instructor(s)
Guillermo Sapiro received his B.Sc. (summa cum laude), M.Sc., and Ph.D. from the Department of Electrical Engineering at the Technion, Israel Institute of Technology. After post-doctoral research at MIT, he became Member of Technical Staff at the research facilities of HP Labs in Palo Alto, California, where he co-developed the image compression techniques used in the original Mars Rovers expedition. He was with the Department of Electrical and Computer Engineering at the University of Minnesota, where he held the position of Distinguished McKnight University Professor and Vincentine Hermes-Luh Chair in Electrical and Computer Engineering. Currently he is with Duke University. His awards include the Office of Naval Research Young Investigator Award in 1998, the Presidential Early Career Awards for Scientist and Engineers (PECASE) in 1998 (awarded in the White House by President Clinton), the National Science Foundation Career Award in 1999, the National Security Science and Engineering Faculty Fellowship in 2010, and the Test of Time Award in 2011 for his paper on image segmentation. His algorithms appear in Adobe’s products, leading medical imaging packages such as ITK, and are also roaming on Mars. He has been teaching image processing for over 15 years and delivered numerous invited plenary talks and short courses at leading imaging and applied mathematics conferences. G. Sapiro is the founding Editor-in-Chief of the SIAM Journal on Imaging Sciences, currently ranked as the second highest impact journal in the whole discipline of applied mathematics.
– Introduction to Image and Video Processing: We will cover the fundamentals, including some elements of visual perception, sensing, sampling, and quantization.Week 2
– Image and Video Compression: We will learn the fundamental tools enabling us to receive images from Mars, to upload images to the web, and to store a lot of images and videos in our mobile phones.
Week 3– Spatial Processing: This week we will learn some of the most classical and fundamental tools that help us still today to make noisy, blurry, and dark images look much better.
Week 4– Image Restoration: When something is known or estimated about the degradation process, we can do much better, and in this week we will learn how.
Week 5– Image Segmentation: How do we split an image or video in its core components?
Week 6– Geometric PDEs: We will learn about the use of partial differential equations and geometric deformations for problems like image enhancement and object detection.
Week 7– Image and Video Inpainting: How to make objects disappear and other special effects.
Week 8– Sparse Modeling and Compressed Sensing: We will cover some of the most modern tools for image enhancement and image analysis.
Week 9– Medical Imaging: As an example of medical image analysis, we will illustrate examples and techniques in the areas of brain research and virus analysis.
Image and video analysis can be approached from numerous areas of mathematics, from linear algebra to geometry, optimization, and differential equations. We plan to make all the lectures as self-contained as possible, but basic background in linear algebra and digital signal processing will be helpful.
The first 5 lectures will follow, in part, “Digital Image Processing, 3rd edition”
Gonzalez and Woods. The more advanced material will be based on material the instructor will make available. Some interesting books for the advanced material include:Michael Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing, Springer.
Guillermo Sapiro, Geometric Partial Differential Equations in Image Analysis, Cambridge University Press.
Alex Bronstein, Michael Bronstein, and Ron Kimmel, Numerical Geometry of Non-Rigid Shapes, Springer
One of the first and still outstanding books in digital image processing is: Azriel Rosenfeld and Avinash Kak, Digital Picture Processing, Academic Press.
The class will consist of lecture videos, normally less then 15 minutes in length. Several such segments will constitute a weekly class. Weekly homework/quizzes will help students to stay on track. There will also be frequently assigned optional programming projects to help students experience the practice of image and video analysis. Weekly subjects will stay as self-contained as possible, starting every week with a new topic in the rich area of image and video analysis.