Named entity tagging is a useful first step in lots of natural language processing
tasks. In sentiment analysis we might want to know a consumer’s sentiment toward a particular entity. Entities are a useful first stage in question answering, or for linking text to information in structured knowledge sources like Wikipedia. And named entity tagging is also central to tasks involving building semantic representations, like extracting events and the relationship between participants.

Application and Advantages

Named entity recognizers are evaluated by recall, precision, and F1 measure.


We can also use the paired bootstrap test to measure if the difference of the F1 score between two systems is significant or not.


Importantly, the unit of response in the NER models should match between training and test conditions. In fact, NER has a segmentation component to separate the unit of response.


Evaluation of NER


The NER task  often applies the rule-base methods in the commercial to be more efficient.
Basically, it uses high-precision rules to tag entities and then applies supervised sequence labeling techniques that use tags from previous stages as additional features.


Rule-based Methods


A crucial step after training a Named Entity Recognition (NER) model is to determine the best possible sequence of labels for a given input. This inference problem is a well-established challenge in NLP. A common and efficient solution involves using dynamic programming algorithms. This method, which is analogous to finding the optimal path through a lattice of potential tags, operates with linear complexity.

Finding the Optimal Label Sequence in NER

Named entities are the specific pieces of key information identified and classified during the Named Entity Recognition (NER) process. They typically consist of proper names that refer to real-world objects and are sorted into distinct, predefined classes. Common categories for these entities include people, locations, organizations, dates, monetary values, and percentages.

Named Entities

Relation extraction is an information extraction task focused on identifying and classifying the semantic relationships between named entities that have already been identified in a text. It serves as a subsequent processing step after Named Entity Recognition (NER), using the extracted entities as a foundation for discovering how they are connected.

Relation Extraction

This diagram illustrates a common architecture for Named Entity Recognition (NER) using BERT, which is a direct application of the sequence labeling approach. An input sequence of tokens ($x_1, x_2, ..., x_m$), prepended with a `[CLS]` token and appended with a `[SEP]` token, is converted into embeddings ($e_i$) and fed into the BERT model. BERT processes the entire sequence and outputs a contextualized hidden state vector ($h_i$) for each token. For the NER task, a separate classification layer is applied to each token's hidden state (e.g., $h_1$ through $h_m$) to predict a corresponding tag from a predefined set, such as {B, I, O} (Begin, Inside, Outside).

Illustration of BERT-based Architecture for Named Entity Recognition

A financial technology company is developing a tool to automatically process business news articles. The goal is to extract specific pieces of information from each article, such as company names, monetary values, and dates, and categorize them accordingly (e.g., 'Apple Inc.' as an ORGANIZATION, '$2.7 billion' as MONEY, 'October 26, 2023' as a DATE). Which of the following processes best describes this core task of identifying and classifying these specific pieces of information?

A marketing team wants to analyze thousands of customer feedback emails to understand which specific products and competitor brands are being mentioned most frequently. They need a system that can automatically find these names in the text and label them as either 'Our Product' or 'Competitor Brand'.

The team is evaluating two automated systems:

*   **System A:** Reads each email and assigns it an overall score from -1 (very negative) to +1 (very positive).
*   **System B:** Reads each email, identifies specific product and brand names, and assigns a category label (e.g., 'Our Product', 'Competitor Brand') to each identified name.

Which system should the marketing team choose to achieve their specific goal? Justify your choice by describing the fundamental task that the correct system performs.

Choosing the Right Text Processing Approach

A basic Named Entity Recognition (NER) task can focus on extracting a single category of entities, such as person names. For example, if a system is instructed to 'Identify all person names' in the text 'For Tom Jenkins, CEO of the European Tourism Organisation, it’s the latter...', it would process the text and produce an output that pairs the identified entity with its classification, such as 'Tom Jenkins - Person Name'. This demonstrates a fundamental NER application focused on a single entity type.

Simple Example of an NER Task: Extracting Person Names

A fundamental task in Named Entity Recognition (NER) is the identification and categorization of multiple entity types within a single text. This process requires a system to locate all named entities and assign them to predefined classes. The final output is typically a structured list where each identified entity is explicitly paired with its assigned category. For example, a system might be given the instruction: 'Identify and classify all named entities in the provided text into categories such as person names, locations, dates, and organizations. List each entity with its type on one line.'

Multi-Category Named Entity Recognition Task

Analyze the sentence below. Identify three distinct pieces of information that are proper names referring to specific things like people, places, or groups. For each piece of information you identify, assign it to a logical category (e.g., Person, Organization, Location, Date) and briefly explain why you chose that category.

Sentence: 'On March 15, 2021, the CEO of InnovateCorp, Dr. Evelyn Reed, announced a new research facility would be built in Geneva.'

Deconstructing Text for Specific Information

A Named Entity Recognition (NER) model generates a probability distribution over the available tag set for each position in an input sequence. For a sequence of length $$m$$, the model outputs distributions $$\mathbf{p}_1, \dots, \mathbf{p}_m$$, where each $$\mathbf{p}_i$$ indicates the likelihoods of various tags for the token at position $$i$$. The model's training and fine-tuning are conducted using these position-wise distributions.

NER Output Distributions

Named Entity Recognition (NER) is a foundational task in Natural Language Processing and a subfield of Information Extraction. It involves a process that automatically detects and classifies key pieces of information, known as named entities, from unstructured text into predefined groups or categories.

Google

Boston University

Sequence labeling is a machine learning methodology used across many Natural Language Processing (NLP) applications. The core idea is to assign a specific class or tag to each token within a given input sequence. The resulting sequence of labels can then be interpreted to extract linguistic annotations. Common examples of this technique include Part-of-Speech (POS) tagging, where each word is labeled with its grammatical role, and Named Entity Recognition (NER), where tokens are tagged to identify entities like names or locations.

Sequence Labeling

Information extraction is a fundamental task in NLP where many problems can be framed as extracting specific, structured information from unstructured text. This process involves identifying data points such as named entities, relationships, and events. The ultimate goal is to convert this raw textual data into a structured format suitable for analysis and utilization in various downstream applications.

Information Extraction

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

Reference of Foundations of Large Language Models Course

Part-of-Speech (POS) tagging is an application of sequence labeling in Natural Language Processing where every word in a sentence is assigned a label corresponding to its part of speech.

Part-of-Speech (POS) Tagging

A common architecture for sequence labeling tasks like Named Entity Recognition (NER) uses BERT. The input sequence is first tokenized, and special tokens like `[CLS]` and `[SEP]` are added to the beginning and end, respectively. This sequence is then passed through the BERT model to obtain a contextualized hidden state representation for each token. Finally, a classification layer is placed on top of each token's hidden state (excluding the special tokens) to predict its corresponding label from a predefined set, such as {B, I, O} for NER.

BERT-based Architecture for Sequence Labeling

Span prediction is an NLP task focused on identifying a continuous segment of text. A prominent example is reading comprehension, where a model is given a query (e.g., $$x_1...x_m$$) and a context passage (e.g., $$y_1...y_n$$) and must locate the span within the context that answers the query. This task can be conceptualized as a form of sequence labeling, where the objective is to predict for each token $$y_j$$ in the context whether it marks the beginning or the end of the desired span.

Span Prediction in NLP

Definition of Named Entity Recognition

A model is designed to perform a sequence labeling task by identifying organizations and locations within a text. For each word (token), it must assign one of the following labels: `O` (not an entity), `B-ORG` (beginning of an organization), `I-ORG` (inside an organization), `B-LOC` (beginning of a location), or `I-LOC` (inside a location). Given the sentence 'The United Nations headquarters in New York City is a major landmark', which of the following represents the correct sequence of labels?

Consider two distinct natural language processing tasks. Task 1 aims to categorize an entire document (e.g., a news article) into one of several predefined topics like 'Sports', 'Technology', or 'Politics'. Task 2 aims to identify every mention of a person's name within that same document. Which of these two tasks is fundamentally a sequence labeling problem? Justify your answer by explaining how the structure of the output relates to the structure of the input for the chosen task.

Applicability of Sequence Labeling

A sequence labeling model is trained for a task using a standard tagging scheme where 'B-ENTITY' marks the beginning of a multi-token entity, 'I-ENTITY' marks a token inside the same entity, and 'O' marks a token that is not part of any entity. When processing a sentence, the model outputs the label sequence below for the corresponding tokens. Analyze this output. What is the primary logical error in this predicted label sequence, and what does this error suggest about the model's understanding of how to form entities?

Analyzing a Sequence Labeling Model's Output

A sequence labeling model, such as one used for Named Entity Recognition (NER), outputs a probability distribution over the possible tag set at each position, generating a sequence of distributions $${\mathbf{p}_1, \dots, \mathbf{p}_m}$$. The training or fine-tuning of the model can be performed over these distributions. For example, if $$p_i(\mathrm{tag}_i)$$ represents the model's assigned probability for the correct tag at position $$i$$, the training loss can be defined as the negative likelihood.

Negative Likelihood Loss in Sequence Labeling

Event extraction is the task of identifying mentions of events in texts. In this case, an event mention is any expression denoting an event or state that can be assigned to a particular point, or interval, in time. 

Event extraction is generally modeled via supervised learning, detecting events via sequence models with IOB tagging, and assigning event classes and attributes with multi-class classifiers.

Event Extraction

Text summarization is the research area aiming at creating a short and condensed version of the original document, which conveys the main idea of the document in a few words.

Text Summarization

Techniques for Jointly Extracting Entities and Relations: A Survey

Named Entity Recognition (NER) in information extraction and plays an important role in many NLP downstream tasks. 

In deep learning, most of NER methods are in the sequence labeling framework. The entity information in a sentence will be transformed into the sequence of labels, and one label corresponds to one word. The model is used to predict the label of each word.

Named Entity Recognition

Large Language Models (LLMs) that have undergone instruction fine-tuning are capable of performing various information extraction tasks with relative ease. This process adapts the model to follow specific commands for extracting structured data from text.

Instruction Fine-Tuning for Information Extraction

Within the broad field of information extraction, which encompasses numerous problems, Named Entity Recognition (NER) stands out as a foundational and long-standing task. It often serves as a starting point for understanding and implementing information extraction systems due to its fundamental role in identifying key data points.

NER as a Foundational Task in Information Extraction

A financial services company wants to automate the analysis of thousands of quarterly earnings reports. Their goal is to build a structured database that tracks key metrics for each company mentioned in the reports, specifically 'Revenue', 'Net Income', and 'Earnings Per Share'. Which of the following best describes the core challenge in transforming the raw text of these reports into the desired structured database?

Based on the company's goals, identify at least three distinct categories of structured information that should be extracted from the raw review text. For each category, provide a specific example of what might be extracted from the sentence: 'The battery life on the new X-Phone is amazing, but I wish the camera had a night mode.'

Structuring Customer Feedback

Consider the task of automatically extracting the acquirer, the acquired company, the acquisition amount, and the date from a news article. Below are two text snippets describing the same event.

**Text A:** "On May 5, 2023, a press release stated that Acme Corp. acquired a smaller company, Innovate Inc., for $500 million."

**Text B:** "The deal, which was hinted at last spring and finally closed yesterday, saw the tech giant absorb the promising startup. Analysts are valuing the acquisition in the ballpark of half a billion dollars."

Analyze why an automated system would find it significantly more challenging to extract the correct, structured information from Text B compared to Text A. Identify at least two specific challenges presented in Text B.

Learn Before

Related

Learn After