Two greedy approaches have been proposed to construct a graph of temporal relations: relations are added one-by-one, in a certain order, checking temporal consistency after each addition. The order can be natural reading order, or follow the score from a local classifier.

Greedy Inference

It first predicts the full graph with a local model, and detect and resolve inconsistencies afterwards, by removing conflicting edges based on confidence among others. Most works report a positive impact of this method, but differ in how they remove edges.

Post-Hoc Conflict Resolution

The sieve-based method is flexible in incorporation of both rule-based and machine learning components. This approach extract TLinks in consecutive phases by separate
model components (or sieves). Each sieve extracts TLinks, using the original input text, and the outputs from earlier sieves. TR is incorporated by taking a transitive closure after applying each sieve.

Sieve-Level and Stacked Inference

There are various ways how integration of TR in training and prediction can improve TIE models: (1) ensure temporal consistency among predicted relations, and (2) to constrain the output space and improve prediction accuracy.

University of Michigan - Ann Arbor

Temporal Information Extraction (TIE) is the process of automatically extracting temporal cues from text, with the goal to construct a (possibly underspecified) timeline of events from them.

TIE not only plays a major role in the general problem of natural language understanding (NLU), but is also used in many applications: information retrieval, question answering, and multidocument summarization, and has great potential in the clinical domain, for applications like patient timeline visualization, forecasting treatment effects, and patient selection for clinical trials.

Temporal Information Extraction (TIE)

Leeuwenberg, A., & Moens, M. F. (2019). A survey on temporal reasoning for temporal information extraction from text. Journal of Artificial Intelligence Research, 66, 341-380.

A Survey on Temporal Reasoning for Temporal Information Extraction from Text (Extended Abstract)

Temporal cues can refer to various components of the event intervals: the entire interval, but also just its start, end, or duration. Example sentence: Antibiotics were started on 6/2 and continued for 10 days. Respiratory status improved up til the last 2 days.

The type of information given for each component can also differ: it can be absolute, or relative to other intervals.

Temporal information provided through language is generally underspecified. Therefore, world knowledge about the typical order of events, their durations, and even typical time between events may be needed to properly interpret the temporal meaning that is to be conveyed.

Temporal cues

AIA is based on a set of thirteen mutually exclusive basic interval relations that can be assigned to any pair of definite intervals where relative positions of start and end-points are known. If for two intervals x and y information about the relative positions of their start and end-points is incomplete, a general Allen relation can be assigned, expressed as the disjunction of all possible basic relations. The full AIA covers the powerset of all thirteen basic relations, comprising $2^{13}=8192$ relations.


TR framework: Allen’s interval algebra (AIA)

The most popular scheme is TimeML, used as the basis for many corpora, and for TempEval, a series of shared tasks on TIE evaluation that resulted in many of the existing TIE systems. The core TimeML concepts are event expressions and time expressions (timex).

In TimeML, events and timex expressions are temporally connected through interval relations, called temporal links (TLinks), approximately following Allen’s basic interval relations, consequently focusing on definite information.

Scheme for temporal annotation

A widely recognized problem in temporal annotation, is that annotators regularly miss temporal cues in the text, especially when annotating TLinks between pairs of events, as there are so many possible relations to annotate, and not all event-pairs are equally useful to annotate.

A frequently made TR-based preprocessing step is to calculate the transitive closure over the annotated relations, to obtain a more dense graph. The effect of temporal closure on performance for models that use no TR during prediction is ambivalent: it could improve or decrease improvement. When using a model that exploits TR also during prediction, data expansion through a temporal closure can be beneficial.

Learn Before

Related

Learn After