Page 85 - DCAP603_DATAWARE_HOUSING_AND_DATAMINING
P. 85
Unit 4: Data Mining Classification
activation functions notes
As mentioned previously, the activation function acts as a squashing function, such that the
output of a neuron in a neural network is between certain values (usually 0 and 1, or -1 and 1). In
general, there are three types of activation functions, denoted by Φ(.) . First, there is the Threshold
Function which takes on a value of 0 if the summed input is less than a certain threshold value
(v), and the value 1 if the summed input is greater than or equal to the threshold value.
Secondly, there is the Piecewise-Linear function. This function again can take on the values of 0
or 1, but can also take on values between that depending on the amplification factor in a certain
region of linear operation.
Thirdly, there is the sigmoid function. This function can range between 0 and 1, but it is also
sometimes useful to use the -1 to 1 range. An example of the sigmoid function is the hyperbolic
tangent function.
LoveLy professionaL university 79
