SIGMM Education Column

Welcome to the SIGMM Educational Column. In this column, we report interesting and notable multimedia educational activities around the SIGMM community. This issue's column brings forward the Multimedia Systems course taught in the Department of Computer Science at the University of Illinois, Urbana-Champaign, by Professor Klara Nahrstedt. In rest of this column, you will find the detail of the course in Prof. Nahrstedt's own words.

- Pradeep K. Atrey

Reflection and Discussion of the CS414 Multimedia Systems Design Course in the Department of Computer Science at the University of Illinois, Urbana-Champaign

The history of the course

I have designed and started to teach the Multimedia Systems course, CS414, in Spring 1996, based on the textbook Steinmetz & Nahrstedt: "Multimedia Computing, Communications and Applications", published in Prentice Hall 1995. Since then it has been offered every year.

The CS414 multimedia system course and the overall multimedia area have been started as a response to a large NSF (National Science Foundation) infrastructure grant that the Computer Science Department at University of Illinois, Urbana-Champaign received 1994-1995. As part of the grant missions, systematic teaching of multimedia systems design became one of the educational goals.

Three instructors taught this course over the last 15 years: Klara Nahrstedt, Karrie Karahalios and Brian Bailey.

The rationale for offering the course

The rationale for offering the course is to teach students an integrated view of the overall multimedia system. As we know, since multimedia data is time-sensitive, if one component in the end-to-end performance does not work right, the overall system performance suffers. For example, if networking component does not have a bounded delay control among distributed nodes, a conversational application such as video teleconference, Skype, collaborative WebEx, will have interactivity problems and usage issues. If video servers are not properly designed, we can see long starting delays to playback movies or overprovisioning to cache movies for fast upload and playback. So the main rationale of the course was and is to show students integration and interdependencies among media characteristics, compression, networking, operating system, storage, applications and user interfaces.

The target students for the course

The course is targeted for undergraduate students and first year graduate students. It is interesting that over the years, the makeup of the class changed. When I started to offer this course in 1996, 90-95% of the class was graduate students. Only very few undergraduate students had the background to take this course. The reason was that undergraduate students learned many important system concepts only in their senior year and so they run out of time to take the multimedia systems class which assumed that students had had already some systems course (operating system or distributed system or networking class). Hence, only graduate students who finished their undergraduate studies were able to take this class.

During the last 5-6 years, I have seen a great change and now 80-90% of students are undergraduate students. The reason is that I have designed a sophomore/junior class in 2003, called "systems programming", where undergraduate students get exposed to many basic system concepts such as threads, synchronization, file systems, memory management, socket programming. This course is now mandatory for each computer science undergraduate. With these systems skills under their belt, the undergraduate students are comfortable to take the multimedia systems design course and succeed.

The educational objective

The educational objective of the multimedia systems course is two-folded: (a) the students should be aware of the most important multimedia systems concepts that have been developed over the last 20 years in the multimedia systems community, and of their integrating aspect; (b) the students need to get hand-on experiences and design/implement a multimedia system to understand concepts and the integrating needs.

The topics covered

The class covers all major components of an end-to-end multimedia system, such as a conversational interactive system or an on-demand system. The course starts with discussion of multimedia characteristics, especially audio, images and video, and then goes into some depth of coding and compression, especially JPEG, MPEG-X/MP3 and H.26X standards families. The compression topic is then followed with multimedia networking component, especially concepts such as Quality of Service aspects, protocols such as RSVP, IPv6, RTP/RTCP, SIP, RTSP, streaming, peer-to-peer streaming, buffering, adaptation. The networking is followed by video servers and content distribution networks for VOD services, synchronization issues, and user interfaces. The course ends with applications discussions such as Skype, retrieval /VOD systems (e.g., Flickr, PPLive), and others. All components are updated every year, but especially the applications module changes as new applications arise.

The mode of instruction (lecture, paper reading, etc.)

The mode of instruction is lectures, but since the course material is being regularly updated (multimedia systems area is fast changing area), for many lectures, I recommend supplementary paper reading. Even though I lecture, since the material is not in the textbook, the students need to read papers.

The course meets three times a week for 50 minutes lecturing. In addition to lectures, we offer tutorials on a need-to-know basis, exposing students to practical tools to be used in their projects. These tutorials and implementation Q&A sessions are being run by the teaching assistant. Usually these sessions are 3-4 times in one semester. Furthermore, the students have access to the multimedia laboratory where computers are equipped with cameras and microphones for their projects.

The course has usually 3-4 machine problems and through these machine problems, the students develop a small multimedia systems in a group, i.e., the students form groups at the beginning of the semester and then stay together throughout the semester to build the required system. Each machine problem (MP) has a deadline. The students are requested to do a careful system design and software engineering since they reuse their MPs. For example, in the MP1 they build a video and audio recorder and player (on a local machine). The goal is that they get familiar with the audio and video formats (e.g., MPEG2,/MPEG4) and some of the timing aspects of real-time audio and video when recording multimedia or playing them. In the MP2, the students experiment with control and streaming protocols between the sender(s) and the receiver(s) to enable captured multimedia to send over the Internet and play the multimedia in real-time at the receiver. In MP3, the students explore synchronization concepts between audio and video since they need to play the streamed video/audio flows together in synchronized way. In MP4, they put additional user interface and other application services together to demonstrate an interesting video-on-demand system, a teleconferencing system or other multimedia system.

Assignments/projects that is a result from the course

Every year the class builds a different multimedia system to expose students to the dynamics and changes that happen in this space. Over the years the students built (a) multimedia mailing system, (b) multi-party video-conferencing system, (c) authoring system, (d) peer-to-peer streaming system, (e) peer-to-peer video server, and others. In Spring 2011, the students built a video chat system between two Android phones.

Over the last five years, we have had industrial sponsors for the multimedia systems class who sponsored competition among the students in terms of the final demonstration project. Since each group in the class builds the same system, it is very easy to compare the results of each group. Over the years, the following companies sponsored the cs414 class competition: Pavlov, Technicolor, Google Inc., Vodafone and Qualcomm.

Feedback from students

The students always appreciate the hands-on experience with the machine problems. They would comment that the concepts are interesting, but having built a real system benefits them on the job market and in the future. Often they comment that MPs have been most helpful in understanding the material. One student commented "I have learned more in this class than any other and all of it seems very useful." Often students would come back years after they graduate and provide me a very positive feedback about this course, the breath of the integrated multimedia system material they learned, the system design skills they acquired, and the software engineering methods they employed. It is a hard and time consuming class, but often this class represents the jumping board for many students to get a unique job offer as a multimedia system designer in the industry which is difficult to get without this class.

Other interesting information that might be useful for others who might want to offer similar Multimedia Systems Design courses

There are several issues to consider when offering this type of course:

1. What is the makeup of the class in terms of students? I have to admit that the class projects are getting simpler and will even get simpler as more and more students taking the class are undergraduate students younger than seniors. The reason is that the undergraduate students have much higher load than graduate students. Usually an undergraduate student takes 5-6 courses per semester, where a graduate student has only 2-3 courses per semester. Hence, the undergraduate students do not have the time to work on complex multimedia systems. When I started this course and all of the students were graduate students (1st year MS/PhD students), the groups built extensive multi-party video conferencing systems and multimedia mailing systems. This is not possible with the current undergraduate class. Even more, as the undergraduate students, taking the class, are at the junior or even sophomore level (after taking the systems programming class), the undergraduate students do not have the experience in building larger scale systems. This means that (a) the instructions for building the systems must be much more detailed and directed (with graduate students, more open-ended projects were possible), and (b) the system complexity must be much lower.
2. What textbook should the class use? The multimedia systems field is evolving very fast. This also causes that it is not possible to write up-to-date textbook fast enough, unless one does it full time. I use currently Steinmetz & Nahrstedt, "Multimedia Systems" book from Springer Verlag, published in 2004, but I heavily supplement this book with papers from the premier system and networking conferences and journals. This also means that the slides are more dense than usual since the slides serve as a reading material. Since there is much less trend to write textbook in this area, it would be good to have some written educational material in form of surveys and tutorial modules. This might be an interesting effort to undertake by the SIGMM educational committee. If we split the load and each professor in his/her area prepares a good set of slides as an educational module, this could be used by others and provide much more updated curriculum.
3. What software, programming language, operating system should the class use? This is a difficult question since it depends on each educational institution. It also depends on the comfort and background of the students from the particular institution.
   1. It would be good if we would have a database of good libraries/tools for teaching multimedia concepts, as well as strong documentation for the various tools. Few years ago, I used ffmpeg as a tool for getting students started on accessing video driver and also playback of video. At the beginning the ffmpeg documentation was very sparse and that made it very difficult to use this tool. However, ffmpeg requires a very steep learning curve which is not possible for the undergraduate students, hence using easier libraries for multimedia are being deployed (e.g., including Java libraries for multimedia support) currently.
   2. If students are taught Java programming language, this might be a good option for the machine problems. However, with Java, often some of the multimedia system interfaces are hidden and students do not learn as much the internals of multimedia devices and behaviors. Hence, one needs to carefully analyze what concepts to teach via Java in the multimedia systems design. If one uses C/C++ for the machine problems in the course, the students get much better insights into the system issues, but the multimedia tools require much longer learning curve (e.g., if using ffmpeg for MPs).
   3. Similar considerations are with Windows OS vs Linux OS. I use for my MPs Linux since multimedia devices (cameras, microphones, etc) are now having good device drivers. But in the past, drivers on the Linux platform were non-existent (e.g, for cameras). Solaris OS was an excellent platform for teaching multimedia systems, it had elaborate libraries for compression functions, etc. but Solaris labs were replaced with Linux labs and this meant to change the platforms for cs414 machine problems.
4. What hardware should the class use? Establishing a laboratory, where computing platforms have multimedia I/O requires additional funds. I was lucky that we had and have a multimedia laboratory where machines are equipped with cameras, microphones, speakers and nice displays. However, as students use the I/O devices, they get faulty. This also means to have a system staff personal to check on the devices, and replace the faulty I/O. The faulty hardware (e.g., sound cards do not work) becomes a sore point for students as they test their MPs. This requires institutional commitment.
5. What assistance one needs for the course? Over the years, I felt that a very important part of the overall course was a knowledge-able teaching assistant, who had strong background in building multimedia systems. This also led me to always take a student as a teaching assistant who passed through the course with very strong final results and enjoyed the class. Another important component is to have a strong system staff support of the general hardware and software infrastructure. Assistance is needed with faulty cameras, sound boards, phones, and other computing platforms, and constant help is needed throughout the semester.

Klara Nahrstedt