Learn Before
Training an Image-to-Caption Model
Case context: You want to build a system that takes a photograph of a kitchen scene and generates a short English caption. Your team is considering several hand-built submodules, but you want to train one model end to end.
Question: What kind of training data do you need, and what will the model learn to produce directly?
Sample answer: You need matched pairs of input images and target captions. In this setup, the model learns to map each photograph directly to a full sentence, rather than predicting a single numeric label.
Key points:
- Training requires paired examples of images and captions.
- The model learns to output a complete sentence.
- The output is a structured, rich prediction instead of a single value.
Rubric: The learner must state that the training set should contain image-caption pairs and that the model is trained to generate a sentence as its direct output.
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Direct Sentiment Prediction from Raw Reviews
End-to-End Speech Recognition
Choosing a Modular Driving System
End-to-End Control Requires Many Labeled Examples
Sufficient Data and Model Capacity Can Remove Feature Bottlenecks
Learning Structured Outputs End to End
What Does End-to-End Learning Usually Replace?
Neural networks are often used in end-to-end learning pipelines.
End-to-end learning connects the _____ to the target output.
Match each model output with the kind of result it describes.
End-to-End Text Classification Flow
With the right labeled examples, what can an end-to-end model sometimes learn to produce?
End-to-end deep learning can only produce a single numerical output.
End-to-end models can learn structured outputs directly.
Match each learning setup to its definition.
Order the reasoning steps showing how end-to-end learning can produce structured outputs.
How Labeled Pairs Shape the Outputs an End-to-End Model Can Learn
Training an Image-to-Caption Model
Complex Outputs from End-to-End Models