July 2024 - Multimediate Challenge - lists.hcics.simtech.uni-stuttgart.de

Information on Team Information
by Philipp Mueller 18 Jul '24

18 Jul '24

Dear all, we received some emails concerning team information. Please send us this information together with your paper submission, i.e.: Team Name Member Names Member Affiliations Email Addresses Many thanks and best wishes, Philipp –––––––––––––––––––––––––––––––––––––––––––––––––––––– Dr. Philipp Müller Senior Researcher Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Stuhlsatzenhausweg 3, Campus D3 1 66119 Saarbrücken Germany +49 681 85775 7752 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Website: https://www.dfki.de

1 0

Paper format
by Philipp Mueller 16 Jul '24

16 Jul '24

Dear all, as we received questions concerning this, here again the details for paper format: The page limit for challenge solution papers is 6 pages, with max. one additional page for references. This page limit is larger than in previous year (4 pages + references) - however if you only need 4 pages (+ one page of references) to present your work this is also fine. Please do not try to stretch the content artificially to 6 pages. We will instruct you how to submit your papers in time before the deadline. Apart from the points made above, we follow the submission instructions of the ACM MM main conference: https://2024.acmmm.org/regular-papers Best, Philipp –––––––––––––––––––––––––––––––––––––––––––––––––––––– Dr. Philipp Müller Senior Researcher Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Stuhlsatzenhausweg 3, Campus D3 1 66119 Saarbrücken Germany +49 681 85775 7752 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Website: https://www.dfki.de

1 0

Final Leaderboard and Submission Instructions
by Philipp Mueller 15 Jul '24

15 Jul '24

Dear MultiMediate'24 participants, many thanks for your participation in the challenge! As a result of your efforts, we could make clear progress on Multi-Domain Engagement Estimation, Bodily Behaviour Recognition, and Eye Contact Detection. We consolidated the final leaderboard here: https://docs.google.com/spreadsheets/d/1TUsixmFEp__mL5_ix0Qj5GlIXoC5FQhgN87… For these we included the approaches from previous years which were documented in a scientific publication. When writing your papers, it is important to correctly represent your achieved test results in relation to this table. In addition it is important to discuss the previously published approaches for the task you tackled: both in the related work section, as well as when discussing your results in the results or discussion section. Please be aware that we are evaluating achieved performance as well as scientific novelty and quality. This means that a submission reaching top performance on a task can still be rejected if the paper is badly written or does not provide meaningful insights. On the other hand, you can have good chances of acceptance even without top results if you can contribute to the scientific understanding of approaches and tasks. Please report all metrics from the leaderboard in your results tables or figures. E.g., for multi-domain engagement estimation please include all four CCC values (NoXi, NoXi Additional Languages, MPIIGroupInteraction, Overall). Please always support your method contributions with appropriate ablation experiments (on the validation set if the allowed test set evaluations were not enough). We strongly encourage publishing your source code. For submission, please mail your pdf to philipp.mueller(a)dfki.de <mailto:philipp.mueller@dfki.de> until July 19, 23:59 AoE. I will send you a receipt confirmation latest on the following day. Submission is single-blind, i.e. you can add your names to the paper, and you should refer to the team names you used on eval.ai. Please let me know in case of any further questions! Best, Philipp –––––––––––––––––––––––––––––––––––––––––––––––––––––– Dr. Philipp Müller Senior Researcher Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Stuhlsatzenhausweg 3, Campus D3 1 66119 Saarbrücken Germany +49 681 85775 7752 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Website: https://www.dfki.de

1 0

Evaluation server misconfiguration and paper submission details
by Philipp Mueller 12 Jul '24

12 Jul '24

Dear all, the deadline on eval.ai was set to a date before the official deadline. We are sorry for this! In case you still wanted to submit but did not have the chance to, please send your predictions to Michael Dietz (michael.dietz(a)informatik.uni-augsburg.de <mailto:michael.dietz@informatik.uni-augsburg.de>) before 23:59 AoE. Please put myself in CC. We will then evaluate it for you and send you the results. There might be slight delays however - you should be able to expect the result starting next week. Best, Philipp –––––––––––––––––––––––––––––––––––––––––––––––––––––– Dr. Philipp Müller Senior Researcher Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Stuhlsatzenhausweg 3, Campus D3 1 66119 Saarbrücken Germany +49 681 85775 7752 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Website: https://www.dfki.de

1 0

Update on paper submission
by Philipp Mueller 03 Jul '24

03 Jul '24

Dear MultiMediate participants, We received an update from the ACM Multimedia publication chairs on the paper submission details for grand challenges. As a result, we are able to give participants one week more time to write their papers. The new paper submission deadline is July 19, AoE. The evaluation deadline remains the same (July 12). This means that you will not be able to get new test set evaluation results after July 12. However, you can use the time until July19 to e.g. run insightful ablation experiments on the validation set, generate qualitative results, etc. Please keep in mind that papers will be evaluated not only with respect to test set performance, but also with respect to the scientific quality and novelty. This for example means you always should support method contributions with adequate ablation studies. We hope the additional week for paper submission will help you to submit papers with high scientific quality. The page limit for challenge solution papers is 6 pages, with max. one additional page for references. This page limit is larger than in previous year (4 pages + references)- however if you only need 4 pages (+ one page of references) to present your work this is also fine. Please do not try to stretch the content artificially to 6 pages. We will instruct you how to submit your papers in time before the deadline. Apart from the points made above, we follow the submission instructions of the ACM MM main conference: https://2024.acmmm.org/regular-papers Please let us know in case of any further questions! Best, Philipp –––––––––––––––––––––––––––––––––––––––––––––––––––––– Dr. Philipp Müller Senior Researcher Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Stuhlsatzenhausweg 3, Campus D3 1 66119 Saarbrücken Germany +49 681 85775 7752 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Deutsches Forschungszentrum für Künstliche Intelligenz GmbH Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern Geschäftsführung: Prof. Dr. Antonio Krüger (Vorsitzender) Helmut Ditzer Vorsitzender des Aufsichtsrats: Dr. Ferri Abolhassan Amtsgericht Kaiserslautern, HRB 2313 –––––––––––––––––––––––––––––––––––––––––––––––––––––– Website: https://www.dfki.de

1 0