Concept

Sentence segmentation

Punctuation, like periods, question marks, and exclamation points, are the most useful sentence-boundary markers. However, punctuation like periods are ambiguous among a sentence boundary marker, a number marker, and a marker of abbreviations. For this reason, sentence tokenization and word tokenization may be addressed jointly.

0

1

Updated 2020-07-16

Tags

Data Science