Topics

human judgements

Exploiting the human judgements that are already implicit in available resources, we avoid purpose-specific annotation.

Page 1, “Abstract”

A main problem we face is that evaluating the performance of these systems ultimately requires human judgement .

Page 1, “Introduction”

Fortunately there is already an abundance of data that meets our requirements: every scientific paper contains human “judgements” in the form of citations to other papers which are contextually appropriate: that is, relevant to specific passages of the document and aligned with its argumentative structure.

Page 1, “Introduction”

Citation Resolution is a method for evaluating CBCR systems that is exclusively based on this source of human judgements .

Page 1, “Introduction”

Third, as we outlined above, existing citations between papers can be exploited as a source of human judgements .

Page 2, “Related work”

The core criterion of this task is to use only the human judgements that we have clearest evidence for.