Dear MultiMediate participants,

evaluations on the challenge test sets are now possible also for this years tasks (bodily behaviour recognition and engagement estimation). Test set evaluations on MultiMediate’21 and ’22 tasks were already, and will continue to be possible by sending your predictions to Michael Dietz. In the following we provide information on how to evaluate your models for this years’ novel tasks.

Evaluation is performed by submitting predictions to Eval.ai. Please do the following:

1. Create an account here: https://hcai.eu/challenges/auth/signup
2. Login with your account here: https://hcai.eu/challenges/auth/login
3. Create a new participant team here: https://hcai.eu/challenges/web/teams
4. Join the challenge with your participant team here: https://hcai.eu/challenges/web/challenges/challenge-page/25/participate
5. Submit your results here: https://hcai.eu/challenges/web/challenges/challenge-page/25/submission

The test set for engagement estimation was already available to participants. The test sets for bodily behaviour recognition (clips and features) can be downloaded here using the credentials you received after submitting your EULA:

https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Dataset/clips_test.zip

https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Dataset/openface_test.zip

https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Dataset/openpose_test.zip

If you have any questions, don’t hesitate to get in touch. All the best for your submissions!

Best regards,

Philipp Müller

Dr. Philipp Müller
Senior Researcher, DFKI GmbH,
Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany
Email: philipp.mueller@dfki.de

--------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
Geschäftsführung:
Prof. Dr. Antonio Krüger (Vorsitzende)
Helmut Ditzer
Vorsitzender des Aufsichtsrats:
Dr. Ferri Abolhassan
Amtsgericht Kaiserslautern, HRB 2313
--------------------------------------------------------------