(Apologize for multiple postings)
CALL FOR PAPERS*****************************************************************************************
IPMU 2022 -Information Processing and Management of Uncertainty in Knowledge-Based Systems /
July 11-15, 2022 ��� Milan, Italy
https://ipmu2022.disco.unimib.it/Special Session on "Data Perspectivism in Ground Truthing and Artificial Intelligence"S3 -
https://ipmu2022.disco.unimib.it/special-sessions/**************************************************
��Description, scope and aimsMany
Artificial Intelligence applications are based on supervised machine
learning (ML), which ultimately grounds on manually annotated data. The
annotation process (often called
ground-truthing) is often performed
in terms of a majority vote and this has been proved to be often
problematic, as highlighted by recent studies on the evaluation of ML
models. Recently, a different paradigm for ground-truthing has started
to emerge, called data perspectivism [1], which moves away from
traditional majority aggregated datasets, towards the adoption of
methods that integrate different opinions and perspectives within the
knowledge representation, training, and evaluation steps of ML
processes, by adopting a non-aggregation policy. This alternative
paradigm obviously implies a radical change in how we develop and
evaluate ML systems: such ML systems have to take into account multiple,
uncertain, and potentially mutually conflicting views [2]. This
obviously brings both opportunities and difficulties: novel models or
training techniques may need to be designed, and the validation phase
may become more complex. Nonetheless, initial works have shown that data
perspectivism can lead to better performances [3,4], and could also
have important implications in terms of human-in-the-loop and
interpretable AI, as well as in regard to the ethical issues or concerns
related to the use of AI systems [5]. Data perspectivism is a framework
to treat uncertainty (the main theme of IPMU) at the level of knowledge
modeling and its integration in the development and evaluation of
systems.
The scope of this special session is to attract
contributions related to the management of subjective, uncertain,
multi-perspective, or otherwise non-aggregated data in ground-truthing,
machine learning, and more generally artificial intelligence systems.
Invited contributions: full research papers and research in progress papers.
Topics of interest:�� �� Subjective, uncertain, or conflicting information in annotation and crowdsourcing processes;
�� �� Limits and problems with standard data annotation and aggregation processes;
�� �� Theoretical studies on the problem of learning from multi-rater and non-aggregated data;
�� �� Participation mechanisms/incentives/gamification for rater engagement and crowdsourcing;
�� �� Ethical and legal concerns related to annotation and aggregation processes in ground-truthing;
�� �� Creation and documentation of multi-rater and non-aggregated datasets and benchmarks;
�� �� Development of ML algorithms for multi-rater and non-aggregated data;
�� �� Development of techniques to detect and manage multiple forms of uncertainty in multi-rater and non-aggregated data;
�� �� Techniques for the evaluation of ML systems based on multi-rater and non-aggregated data;
�� �� Applications of data perspectivism and non-aggregated data to interpretable, human-in-the-loop AI and algorithmic fairness;
��
�� Experimental and application studies of ML/AI systems on multi-rater
and non-aggregated data, in possibly different application domains (e.g.
NLP, medicine,legal studies, etc.)
***************************************************
Important dates