Case Study

Moving to an End-to-End Speech Model

Case context: Your team is building a speech-to-text service for customer support calls. The current design starts with hand-engineered acoustic features and an intermediate phoneme layer, but quality has stopped improving even after extensive tuning. You now have a very large collection of recorded calls paired exactly with their transcripts.

Question: Using end-to-end model design, what architecture change should you make so the new dataset can help push performance beyond the present limit?

Sample answer: Replace the staged feature-based pipeline with a single end-to-end neural network. Make the model large enough to learn the mapping from raw audio inputs directly to transcript outputs, and train it on the new paired recordings so it can learn representations that the earlier design could not capture.

Key points:

  • Switch to an end-to-end training setup
  • Use a sufficiently large neural network
  • Learn directly from audio-to-text examples
  • Stop depending on hand-built acoustic and phoneme stages

Rubric: The response must recommend replacing the manual multi-stage pipeline with a large end-to-end neural network trained on the new paired data.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Python Programming Language

Data Science

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI