July 2023 - Multimediate Challenge - lists.hcics.simtech.uni-stuttgart.de

If you did not receive your reviews, please get in touch!
by Philipp Mueller 26 Jul '23

26 Jul '23

Dear participants, one team told us that they did not yet receive the reviews via the CMT. If this is the case for you, please send me your paper ID, and I can send you the reviews. Best, Philipp -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

1 0

Instructions for Camera Ready Submission
by Philipp Mueller 25 Jul '23

25 Jul '23

Dear participants, many congratulations to all of you whose paper was accepted! We expect that you make use of the reviews to improve your paper for the camera ready submission. We will also try to get the doi for the baseline paper asap and I will send it to you once I have it. If this takes too long however then you should submit your camera ready paper and simply cite the baseline paper without the doi. I am giving you some information on the ranking of participating teams that have accepted papers for the engagement estimation and bodily behaviour recognition tasks: Engagement Estimation: 1. USTC-IAT-United, 0.71 2. PolyU Power, 0.68 3. PRL, 0.66 Bodily Behaviour Recognition 1. HFUT-VUT, 0.63 2. LASII, 0.57 Please let me know in case of any furhter questions! Best, Philipp Müller -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

1 0

Information on Invited Submissions
by Philipp Mueller 13 Jul '23

13 Jul '23

Dear MultiMediate participants, in the following we provide the team names who are among the top3 teams who indicated their intent to submit a paper. These are the teams that are invited to submit a paper. We do this for each task. In case you spot any error, please notify us asap: Engagement Estimation: USTC-IAT-United PolyU Power PRL (AI_lab, happy game, and non_deep_learning did not indicate their intent to submit a paper) Bodily Behaviour Recognition: Anonymous (same paper as Eye Contact Estimation submission) LASII (USTC_AiLab and Challenge did not indicate their intent to submit a paper) Backchannel Detection: PolyU Power Eye Contact Detection: Anonymous (same paper as Bodily Behaviour Recognition submission) UNB (same paper as Next Speaker Prediction submission) Next Speaker Prediction: UNB (same paper as Eye Contact Detection submission) (Anonymous and JYNB did not indicate their intent to submit a paper) We did not yet receive the submission link from the ACM MM grand challenge chairs. We will send the submission link to the respective teams as soon as possible once we receive it. So please keep checking your inboxes. Btw: you will be able to continue evaluating until the end of the paper submission deadline (and can include updated results in your submission). Best, Philipp -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

1 0

Evaluation on the
by Philipp Mueller 12 Jul '23

12 Jul '23

Dear MultiMediate participants, for all of those who submit to previous years’ tasks, this is a reminder that we collected all previous approaches on this task here: https://multimediate-challenge.org/Leaderboard/ It is important to compare your performance and discuss the relation of your work to these previous approaches in your submission, at least for those previous approaches that are already published. Best, Philipp -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

1 1

[MultiMediate'23] Important information on paper submission
by Philipp Mueller 11 Jul '23

11 Jul '23

Dear MultiMediate participants, tl;dr If you want to submit a paper, and haven’t yet told this to us, please do so ASAP! We are allowed to send out 3 invites for paper submission for each challenge task. On the engagement detection task we might not be able to send out an invite to every participating team. We will send out the invite to the 3 best teams on this task that also indicated their intent to submit a paper based on the achieved test performance on Wednesday 12, AOE. Long Version: The ACM MM grand challenge chairs allowed us to send out the link for paper submission to the top 3 teams of each challenge. As we have 6 different tasks and there is no way how to compare results between these in a fair way, they allowed us to invite the 3 top teams of each challenge task. I.e. 3 teams for the engagement estimation task, 3 teams for the bodily behaviour recognition task, etc. We asked the MultiMediate participants on June 26 via the mailing list (and on June 30 via the website) to indicate whether they would like to submit a paper. We received 2 positive replies for bodily behaviour recognition, 2 for engagement estimation, and 1 for eye contact detection. Now we observed that we have more than 2 teams submitting their predictions to the engagement detection task. Please let us know asap if you would also like to submit a paper for this task. If we have more than 3 teams who want to submit a paper, we will need to select the top ones amongst them. We will do so based on the performance achieved until Wednesday 12, AOE. Please be aware that while the achieved performance is an important criterion, a successful paper submission also needs to be well-written and show scientific insights or novelty. Please get in touch if you have any questions! Best, Philipp -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

2 2

Test Set Evaluations Possible Now!
by Philipp Mueller 01 Jul '23

01 Jul '23

Dear MultiMediate participants, evaluations on the challenge test sets are now possible also for this years tasks (bodily behaviour recognition and engagement estimation). Test set evaluations on MultiMediate’21 and ’22 tasks were already, and will continue to be possible by sending your predictions to Michael Dietz. In the following we provide information on how to evaluate your models for this years’ novel tasks. Evaluation is performed by submitting predictions to Eval.ai. Please do the following: 1. Create an account here: https://hcai.eu/challenges/auth/signup <https://hcai.eu/challenges/auth/signup> 2. Login with your account here: https://hcai.eu/challenges/auth/login <https://hcai.eu/challenges/auth/login> 3. Create a new participant team here: https://hcai.eu/challenges/web/teams <https://hcai.eu/challenges/web/teams> 4. Join the challenge with your participant team here: https://hcai.eu/challenges/web/challenges/challenge-page/25/participate <https://hcai.eu/challenges/web/challenges/challenge-page/25/participate> 5. Submit your results here: https://hcai.eu/challenges/web/challenges/challenge-page/25/submission <https://hcai.eu/challenges/web/challenges/challenge-page/25/submission> The test set for engagement estimation was already available to participants. The test sets for bodily behaviour recognition (clips and features) can be downloaded here using the credentials you received after submitting your EULA: https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data… <https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data…> https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data… <https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data…> https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data… <https://www.perceptualui.org/closed-area/MPIIGroupInteraction-Complete-Data…> If you have any questions, don’t hesitate to get in touch. All the best for your submissions! Best regards, Philipp Müller -- Dr. Philipp Müller Senior Researcher, DFKI GmbH, Campus D3 2 Stuhlsatzenhausweg 3 D-66123 Saarbrücken, Germany Email: philipp.mueller(a)dfki.de -------------------------------------------------------------- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzende) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 --------------------------------------------------------------

1 0