Invited Talk at the Austrian Computer Science Day
On 18.06.2021, the Austrian Computer Science Day (ACSD) took place as an online event providing a forum for academic exchange and networking among computer scientists in Austria and beyond. The theme of the conference was “Humans in the Digital Age”.
The SemSys group was represented with an invited talk given by Marta Sabou that complemented an exciting program on a range of aspects of digitalisation and its impact on human society. In line with the theme of the conference, the invited talk focused on “Human-centric Evaluation of Semantic Resources” and presented recent results from the Systematic Literature Reviews performed in the HOnEst and Obaris projects.
The talk highlighted that, as semantic resources (ontologies, Knowledge Graphs) enable a suit of modern intelligent systems, their quality and correctness has a direct effect on the functioning of these systems. While automatic verification techniques can identify and resolve a range of quality issues, our recent systematic literature study in the HOnEst project revealed that several quality aspects can be best assessed through human involvement. Additionally, while such human-centric evaluation practices are spread across several domains and research areas, there is also a lack of methodology on how to perform them correctly. Against this backdrop, the talk proposes possible solutions based on the emerging field of Human Computation and outline avenues for future research.
