Task #16: Evaluation of wide Coverage Knowledge ResourcesSummary
This task tries to establish the relative quality of the semantic
resources available (derived by manual or automatic means) in a neutral
environment. The quality of each large-scale knowledge resource will be
indirectly evaluated on a Word Sense Disambiguation task. In articular,
we will use the English Lexical Sample task as evaluation bechmark
to evaluate the relative quality of each resource. Furthermore, trying
to be as neutral as possible with respect the knowledge bases studied,
we will apply sistematically the same disambiguation method to all the
resources. In order to measure the relative quality of the knowledge
resources submitted for the task, we will perform an indirect
evaluation by using all the resources delivered as Topic Signatures
(TS). That is, word vectors with weights which are associated to a
particular WordNet synset.
Last updated: February 1, 2007 |