Concept
Preconditioning
Preconditioning is an optimization technique used to help with scale adjustment. By applying a mathematical transformation that equalizes the scales or normalizes the curvature across different dimensions, preconditioning makes the landscape of the objective function more uniform. This allows optimization algorithms to navigate high-dimensional spaces more efficiently without requiring complex, dimension-specific learning rate adjustments.
0
1
Updated 2026-05-15
Tags
D2L
Dive into Deep Learning @ D2L