Learn Before
Case Study

Evaluating Activation Function Properties

A deep neural network is being trained, but a significant number of neurons consistently output zero for any negative input they receive. This causes these neurons to stop learning, a phenomenon that hinders the model's overall performance. An engineer proposes replacing the problematic activation function with the function defined as f(x) = x / (1 + e^(-βx)), where β is a positive constant. Based on the mathematical properties of this proposed function, justify why this change is a reasonable strategy to mitigate the issue of non-learning neurons.

0

1

Updated 2025-10-06

Contributors are:

Who are from:

Tags

Data Science

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science