Cássia Trojahn dos Santos, Christian Meilicke, Jérôme Euzenat, Heiner Stuckenschmidt, Automating OAEI Campaigns (First Report), in: Asunción Gómez Pérez, Fabio Ciravegna, Frank van Harmelen, Jeff Heflin (eds), Proc. 1st ISWC international workshop on evaluation of semantic technologies (iWEST), Shanghai (CN), 2010
This paper reports the first effort into integrating OAEI and SEALS evaluation campaigns. The SEALS project aims at providing standardized resources (software components, data sets, etc.) for automatically executing evaluations of typical semantic web tools, including ontology matching tools. A first version of the software infrastructure is based on the use of a web service interface wrapping the functionality of a matching tool to be evaluated. In this setting, the evaluation results can visualized and manipulated immediately in a direct feedback cycle. We describe how parts of the OAEI 2010 evaluation campaign have been integrated into this software infrastructure. In particular, we discuss technical and organizational aspects related to the use of the new technology for both participants and organizers of the OAEI.
ontology matching, evaluation workflows, evaluation criteria, automating evaluation