Ontology Building Competition (BOC) is a competition for developing ontologies, where the evaluation and the final ranking is done automatically, based on five dimensions of evaluation: structural, semantic, design patterns, worst practices and the ability of answering to a set of competency questions. The rules of the competition are described directly in formal language. The reference ontology for this competition has been developed in RacerPro. The RacerPro LISP API was used to define evaluation metrics on the ontologies and to compute ranking. For checking the domain coverage, the ontologies should i) provide answers to some predefined competency questions (CQ) and ii) cover some specific pre-defined terms. CQs have two sources: 1) pre-defined by the organizers, or 2) proposed directly by the competitors. The first set of CQs aims to assure convergence of the ontologies developed by the participants. This set assures that all the ontologies cover a common kernel of concepts and relations in different modelling approaches. The common kernel is proportional to the size of the CQs set. With the second set of CQs, the competitors have incentives to formulate questions that their ontology can easily answer, but which can cause problems for the other competitors. The participants had to map their ontology concepts to the terms appearing in critical questions. BOC2013 represents the first edition of the competition, which a participation of 25 teams. In this paper the formal specification, the evaluation metrics that were used, and an analysis of the results of BOC 2013 are presented. Both the evaluation framework and the participating ontologies are public available on the competition page.

ontology evaluation, ontology engineering, semantic web