Moving to an End-to-End Speech Model
Case context: Your team is building a speech-to-text service for customer support calls. The current design starts with hand-engineered acoustic features and an intermediate phoneme layer, but quality has stopped improving even after extensive tuning. You now have a very large collection of recorded calls paired exactly with their transcripts.
Question: Using end-to-end model design, what architecture change should you make so the new dataset can help push performance beyond the present limit?
Sample answer: Replace the staged feature-based pipeline with a single end-to-end neural network. Make the model large enough to learn the mapping from raw audio inputs directly to transcript outputs, and train it on the new paired recordings so it can learn representations that the earlier design could not capture.
Key points:
- Switch to an end-to-end training setup
- Use a sufficiently large neural network
- Learn directly from audio-to-text examples
- Stop depending on hand-built acoustic and phoneme stages
Rubric: The response must recommend replacing the manual multi-stage pipeline with a large end-to-end neural network trained on the new paired data.
0
1
Tags
Python Programming Language
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Scaling Up to Remove Hand-Crafted Features
Large End-to-End Models and Error Limits
Requirements for End-to-End Model Performance
Match each item to its role in an end-to-end speech system.
From Handcrafted Features to End-to-End Learning
How Model Size and Data Affect Feature Engineering
Moving to an End-to-End Speech Model
Requirements for an End-to-End Model
When End-to-End Learning Reduces Manual Feature Limits
Can a Small Network Always Stay Low-Bias with More Data?