880243 :Social Signal Processing (CSAI/HAIT/DJ)

General info

Instruction language English
Type of Instruction Lecture and practical assignments (Lecture schedule)
Type of exams 1 practical group assignment (40%) and 1 individual final exam (60%) (Examination schedule)
Course load:6 ECTS credits
Registration:Enrollment via Blackboard before start of lectures
Blackboard InfoLink to Blackboard (When you see 'Guest are not allowed in this course', please login at Blackboard itself)


No photo available
dr. M. Postma-Nilsenova (coordinator)
No photo available
dr. M. Atzm├╝ller


The general goal of the course is to provide students with theoretical and practical knowledge and skills required to identify, compute, model, and analyze social signals and according social. We focus on social signals including non-verbal cues to speakers' emotions, attitudes, social roles, and interaction structure, also reflected in (collective) social interactions and dynamics. Automatic processing and analysis of these can play an important supporting role in a number of areas and applications (e.g., intelligent tutoring and control systems, crowd control, analysis of group interaction). For investigation, we apply data analysis and (complex) network analytics methods.

Learning goals:

Assignments (students may work in pairs):

  • Describe a highly relevant complex social signal (e.g., engagement in the context of gaming)
  • Construct an experimental task in which social signals and social interactions can be elicited and collected using sensors and appropriate recordings (e.g., audio/video and special sensors)
  • Process the data collected in the task utilizing according software
  • Quantitatively analyze the outcomes
  • Draw conclusions from the quantitative analysis regarding the role of the collected social signal data and the success of the task design.

Final Exam (open questions):
  • Describe social signals and the field of Social Signal Processing
  • Describe the use of SSP technology in practice
  • Reason about the relationship between behavioral and social cues in different situations; be able to illustrate the relationship with examples
  • Evaluate the use of SSP techniques for novel applications


The course consists of one lecture and one tutorial each week. In the lectures we discuss theories and research literature related to the main research topics. During the tutorials we practice with selected technology. Students will learn to analyze automatically behavioral cues displayed by speakers (e.g., postures, vocalizations) and their interaction structures, and how to interpret them as social signals and affective signals. Furthermore, students will learn how to analyze social structures and behavior both on the individual as well as the group level.


  • Attendance of practical sessions is expected. During the practical sessions, students can collect bonus points for in-class assignments.
  • For the practical assignment in this course, students are expected to have working knowledge of Methods and statistics; practical analysis will be performed using the R software for statistical computing. There will be a short targeted introduction to R in the course, while acquiring basic knowledge of R before taking the course is recommended, and is the responsibility of the student.
  • The final grade for this course is the weighted average of the practical group assignment and the final exam which is individual. In order to pass the course, both the average grade AND the grade of the final exam must be sufficient. Resits are only allowed for the final exam.

Compulsory Reading

  1. The literature for this course (on average two journal articles or chapters per class) will be announced through Blackboard.
  2. Further recommended literature and reading materials will be announced in the course and via blackboard.

Required Prerequisites


Recommended option for

  • Master Business Communication and Digital Media ( 2014, 2015, 2016, 2017 )
  • Master Intercultural Communication ( 2014 )
  • Master Communication Design ( 2014, 2015, 2016, 2017 )
  • Master Human Aspects of Information Technology ( 2014, 2015, 2016 )
  • Master Data Journalism ( 2014, 2015, 2016 )
  • Master Communication and Information Sciences ( 2014, 2015, 2016, 2017 )
  • Cognitive Science and Artificial Intelligence ( 2017 )