Activation Function

A nonlinear function applied to a neuron’s weighted sum that lets neural networks learn complex patterns. Common examples include ReLU, sigmoid, and tanh.