880243 :Social Signal Processing (HAIT/DJ)

General info

Instruction language English
Type of Instruction Lecture and practical assignments (Lecture schedule)
Type of exams 1 practical group assignment (40%) and 1 individual final exam (60%) (Examination schedule)
Course load:6 ECTS credits
Registration:This course has a maximum capacity of 40 participants. If you filled out the survey you received recently (“selecting master courses CIS Fall semester 2016) you will be enrolled automatically. Other students: see specifics.
Blackboard InfoLink to Blackboard (When you see 'Guest are not allowed in this course', please login at Blackboard itself)


No photo available
dr. M. Postma-Nilsenova (coordinator)

dr. K.A. de Rooij


The general goal of the course is to provide students with theoretical and practical knowledge and skills required to identify and compute social signals. Social signals are non-verbal cues to speakers' emotions, attitudes and social roles. Automatic processing of the cues can play a supporting role in a number of areas (e.g., tutoring and control systems, crowd control, group interaction analysis).

Learning goals:

Assignments (students may work in pairs):

  • Describe a highly relevant complex social signal (e.g., engagement in the context of gaming)
  • Construct an experimental task in which the social signal can be elicited (is present/absent) and collect video and audio recordings
  • Process the data collected in the task with social signal processing software
  • Quantitatively analyze the outcomes
  • Draw conclusions from the quantitative analysis regarding the role of the social signals and the succes of the task design.

Final Exam (open questions):
  • Describe social signals and the field of Social Signal Processing
  • Describe the use of SSP technology in practice
  • Reason about the relationship between behavioral and social cues in different situations; be able to illustrate the relationship with examples
  • Evaluate the use of SSP techniques for novel applications


The course consists of one lecture and one tutorial each week. In the lectures we discuss theories and research literature related to the main research topics. During the tutorials we practice with selected technology. Students will learn to analyze automatically behavioral cues displayed by speakers (postures, vocalizations, gestures, facial expressions, gait, as well as psychophysiological markers such as heart rate variability) and how to interpret them as social signals (interest, hostility, empathy, agreement, dominance) and affective signals (joy, sadness, anger, fear).


This course has a maximum capacity of 40 participants. If you filled out the survey you received recently (selecting master courses CIS Fall semester 2016) you will be enrolled automatically. All other students have to send an e-mail to: mastercoursesCIW@uvt.nl and will be enrolled if places are available.

  • Attendance of practical sessions is expected. During the practical sessions, students can collect bonus points for in-class assignments.
  • For the practical assignment in this course, students are expected to have working knowledge of Methods and statistics; students should be able to work with SPSS. If a student lacks experience in using this program, it is their responsibility to acquire this knowledge independently.
  • The final grade for this course is the weighted average of the practical group assignment and the final exam which is individual. In order to pass the course, both the average grade AND the grade of the final exam must be sufficient. Resits are only allowed for the final exam.

Compulsory Reading

  1. The literature for this course (on average two journal articles or chapters per class) will be announced through Blackboard.

Recommended Reading

  1. Picard, R. (1997). Affective Computing. Massachusetts Institute of Technology.
  2. Calvo, R.A., Peters, D. (2014). Positive Computing: Technology for Wellbeing and Human Potential. MIT Press.

Required Prerequisites


Recommended option for