UKP participated in the GermEval-2017 : Shared Task on Aspect-based Sentiment in Social Media Customer Feedback. We participated in all four subtasks, namely relevance classification, document-level sentiment classification, aspect-category and sentiment detection, and aspect target extraction. The provided data contains customer feedback about Deutsche Bahn AG and was crawled from various web-sources.

We used sentence embeddings and an ensemble of classifiers for two sub-tasks as well as state-of-the-art sequence taggers for two other sub-tasks. First is a modified version of the stacked learner which has shown good performance for the SemEval 2017 Task 10, and second a BiLSTM-CRF using word- and character-level embeddings. Our systems achieved top ranks in two subtasks and placed mid-rank in the other two subtasks.

Further information on the task and the original dataset can be found on the competition website.

The goal of the Fake News Challenge, in general, is to explore, how approaches based on machine learning and natural language processing, can be leveraged to address the fake news problem. Stage 1 of the Fake News Challenge (FNC-1) was focusing on the task of Stance Detection. Stance Detection is the process of estimating the relative perspective (or stance) of a text with respect to a topic, claim or issue. The version of Stance Detection, which was selected for FNC-1, is based on the work of Ferreira and Vlachos.

For FNC-1, estimating the stance of a body text from a news article relative to a headline was chosen as the task. The body text either agrees, disagrees, discusses or is unrelated to the statement represented by the headline. In the competition, the teams were given a labeled data set for system development.

At the end of the competition, an unseen set of test data was provided for system evaluation.

Out of 50 participating teams, UKP/AIPHES came in second with the score of 81.97%.

For more details please refer to the original Fake News Challenge website.

UKP participated in the SemEval-2017 Task 10: Extracting Keyphrases and Relations from Scientific Publications. In particular, we participated in Task (B): “Classification of identified keyphrases”. The goal of this subtask was to classify identified keyphrases from scientific publications into one of the three classes “Material”, “Task”, and “Process”.

Our system is an ensemble of neural techniques: an attention-based Bi-LSTM model, a character-level convolutional neural net and a stacked learner with an MLP meta-classifier. Our system had a micro-F1-score of 0.63 and ranked second out of 5 systems, closely behind the first system with an F1-score of 0.64. Erroneously, we only used about 15% of the available training data. With the full training data our system has an F1-score of 0.69. More information can be found here.

UKP participated in the Story Cloze Test challenge at the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem 2017). The goal of this competition was to provide a common ground for the evaluation of systems on language understanding. Given four sentences of a story on everyday life events, a system had to identify the correct ending from a set of two predefined ending sentences. More information can be found here.

Our system is based on a deep learning architecture combined with a rich set of manually-crafted linguistic features. The system outperformed all known baselines for the task, achieving an accuracy of 71.7 %. The system was placed fourth among 8 systems.

Event Nugget Detection at TAC 2015

UKP participated in the shared task of Event Nugget Detection at the Text Analysis Conference (TAC 2015). The goal of this shared task was to identify the explicit mentioning of events and to classify the event type. Our system is based on state-of-the-art deep neural networks and uses only minimal preprocessing. The system achieves an F1-measure of 65.31% and is placed first among 14 systems. More information can be found here.

Named Entity Recognition in German at KONVENS 2014

UKP participated in the shared task of GermEval at KONVENS 2014. In this task, systems were expected to automatically annotate nested named entities in German. For the shared task, a new dataset with sentences from German news articles and Wikipedia was created. Our system is based on state-of-the-art deep neural networks combined with specifically designed features. The system achieves an F1-measure of 75.1% and is placed 2nd among 11 systems. You can read further details on the task and about our system.

Sentiment Analysis in Twitter at SemEval 2014

UKP participated in the Sentiment Analysis in Twitter task for SemEval 2014, collocated with COLING conference on 23-24 August, 2014 in Dublin, Ireland. According to the official results, our system (ukp-dipf) achieved the 8th place on tweet level (subtask B) in terms of macro-averaged F-score among 50 systems. On the expression level (subtask A), our system ranked 14 out of 27 systems, based on macro-averaged F-score. For more details please refer to the dedicated page.

Uncovering Plagiarism, Authorship and Social Software Misuse 2013

UKP participated in the first edition of the Author Profiling Task in the PAN Lab at CLEF 2013. The author profiling task aims at revealing certain categorical information about the author, such as his/her age or gender, rather than reveal his/her exact identity. Our system placed 4th among 20 systems in Spanish and 15th among 22 systems in English in terms of accuracy. You can read further details on the task and about our system.

CrossLink2 at NTCIR-10

UKP participated in the Cross-lingual Link Discovery Task (CrossLink-2) at the 10th NTCIR Workshop (NTCIR-10) held on 18-21 June 2013 at the National Center of Sciences in Tokyo, Japan. CrossLingual Link Discovery (CrossLink) is a task of discovering potential links between cross-lingual documents. At NTCIR-9, the UKP team developed a CrossLink framework consisting of anchor selection, anchor ranking, anchor translation, and target discovery subtasks for English-to-{Chinese, Japanese, Korean} directions. At NTCIR-10, the framework is further extended to work in the reverse direction ({Chinese, Japanese, Korean}-to-English), to find out the properties of the tasks in respect of resources, language pair, language direction as well as link discovery methods and evaluation approach. You can read further details about the task and about our system.