tanh Activation Function

-

Tanh Activation Function




tanh Activation Function is also very similar to the Sigmoid Activation Function. But this function returns the value range(-1,1). It is Zero Centric.


Advantages:
  • It is mostly used in the output layer for binary classification.
  • It is a good example in case when input>0, the gradient we will get all negative or positive, this will lead to an exploding or vanishing gradient problem, thus tanh will work well.
Disadvantages:
  • This lead sometimes to Saturated Gradient.

Comments

Post a Comment

Popular posts from this blog

Deep Learning

-
Image
Deep Learning What is Deep Learning? Deep learning is a subset of machine learning in artificial intelligence (AI) that has networks based learning from data that is unstructured or unlabeled. Also known as deep neural learning or deep neural network. This is also similar to the Human Brain. Do you know the First Neuron created? Yes, Perceptron is the first neural network unit, that does some computations to detect features and business intelligence in input data. This was introduced by  Frank Rosenblatt in 1957. Types of Deep Learning? Artificial Neural Network Deep Brief Net Convolutional Neural Network Recurrent Neural Network Neural Network The first layer is the Input Layer, next is the Hidden layer and the final one is the Output layer. there can be any number of hidden layers in the network. How Neural Network works? Neural Network we create works similar to the human brain, we give features th
Read more

Loss & Cost Functions

-
Image
Loss Functions In neural networks, after a forward propagation, the loss function is calculated. The loss function is the result of the difference between actual output and the predicted output. Different loss functions will return different values for the same prediction, and thus have a considerable effect on model performance.\ Regression Loss Functions Mean Squared Error Loss Mean Squared Logarithmic Error Loss Mean Absolute Error Loss Binary Classification Loss Functions Binary Cross-Entropy Hinge Loss Squared Hinge Loss Multi-Class Classification Loss Functions Multi-Class Cross-Entropy Loss Sparse Multiclass Cross-Entropy Loss Kullback Leibler Divergence Loss Cost Functions Cost functions for Regression problems: Mean Error (ME) Mean Squared Error (MSE) Mean Absolute Error (MAE) Root Mean Squared Error (RMSE) Categorical Cross Entropy Cost Function. Binary Cross Entropy Cost Function. Mean Absolute Error This function returns the mean of absolute differences among predictions an
Read more

Recurrent Neural Network & LSTM

-
Image
Recurrent Neural Network RNN is a neural network, where the output of the previous step is fed as input to the current step . The reason why the previous output is required in cases likes to predict the next word in the sentence, hence remembrance of previous words. Hence RNN has loops in them, allowing information (words) to persist. Here, A is a network, A gets an input X  and leaves an output value h.  A loop allows information to be passed from one step of the network to the next. RNN is multiple copies of the same network, each passing the same information to the successor. Applications of RNN Chat Bots NLP Translator Sentence completer Stock price Predictor, etc The problem in  Simple RNN The problem is Long-term Dependencies . For example, consider a language model trying to predict the next word based on the previous word. If we are trying to predict the last word in "The clouds are in the sky," we don’t need any further context – it’s pretty obvious the next word i
Read more