Word alignment-based methods aim to capture idiom translation accuracy that complements LitTER but are sensitive to words’ literal meaning. They should be used with caution for idiom translation evaluation. The authors discovered noisy or empty alignments were common and choose statistical over embedding-based methods. 


Alignment Metrics 


APT-Eval metrics consist of unigram precision which measures the ratio of words in the reference spans occurring in the hypothesis spans; seen in Fadaee et al. (2018). ChrF (Popovic, 2015) measures character n-gram overlap 


University of California, Berkeley

Alignment-based Phrase Translation Evaluation (APT-Eval) extends Fadee et al. (2018) and subword-level metrics to measure idiom translation accuracy. It uses word alignments to find words in the hypothesis and reference sentences which align with the annotated idiom source span and compares the two. 

Learn Before

Related

Learn After