Navigation

Temporal Summarization Track

News events such as protests, accidents or natural disasters represent a unique
information access problem where traditional approaches fail. For example,
immediately after an event, the corpus may be sparsely populated with relevant
content. Even when, after a few hours, relevant content becomes available,
it is often inaccurate or highly redundant. At the same time, crisis events
demonstrate a scenario where users urgently need information, especially if they
are directly affected by the event.

The goal of the TREC Temporal Summarization Track is to develop systems for efficiently monitoring the
information associated with an event over time. Specifically, we are interested
in developing systems which can broadcast short, relevant, and reliable sentencelength
updates about a developing event. The track has the following four main
aims:

To develop algorithms which detect sub-events with low latency,

To model information reliability in the presence of a dynamic corpus,

To understand and address the sensitivity of text summarization algorithms
in an online, sequential setting, and

To understand and address the s sensitivity of information extraction algorithms
in dynamic settings.

New

Nov 20, 2015: Thank you for participating in this year's track! And thanks to participants over the last three years. The track has been succeeded by the Real Time Summarization Track. Check them out (and their server tools) and sign up to participate!

Archive

June 10, 2015: Test events, guidelines, and metrics available on downloads page.

April 23, 2015: The organizers are actively developing new topics for this year's evaluation. Please join the google group for updates!

November 21, 2014: Thanks for attending TREC! Stay tuned for more concrete information about the track changes. Also, the Google Group has changed name in order to be year-agnostic.

July 7, 2014: TS-specific corpus subset released. This uses the same gpg key (and access agreement) as the full corpus. You are welcome to submit runs using either the full corpus or this sample.

June 12, 2014: draft guidelines, metrics, and test topics released. All topics from 2013 may be used as training.

April 15, 2014: we are currently in topic development and will release guidelines for the 2014 track soon.

October 22, 2013: evaluation results have been released. Evaluation script availablehere. Also see the Match View Interface.