Task #16: Evaluation of wide Coverage Knowledge Resources

Summary

This task tries to establish the relative quality of the semantic resources available (derived by manual or automatic means) in a neutral environment. The quality of each large-scale knowledge resource will be indirectly evaluated on a Word Sense Disambiguation task. In articular, we will use the English Lexical Sample task as evaluation bechmark to evaluate the relative quality of each resource. Furthermore, trying to be as neutral as possible with respect the knowledge bases studied, we will apply sistematically the same disambiguation method to all the resources. In order to measure the relative quality of the knowledge resources submitted for the task, we will perform an indirect evaluation by using all the resources delivered as Topic Signatures (TS). That is, word vectors with weights which are associated to a particular WordNet synset.


Full Details of Task


Last updated:  February 1, 2007