The activation function is a central component in every neural network.It determines how strongly a neuron in the network is activated and thus decides which structures are learned from the data.This article explores the properties of activation functions and compares commonly used functions.Tips are provided on how to choose the right activation function for a network architecture and specific use case.