Tanh

What Is Tanh?

Tanh is an activation function used by neural networks to transform the output of a neuron.

The mathematical formula is:

tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x))

You do not need to memorize this formula immediately. The important thing is to understand what Tanh does.

How Does Tanh Work?

Tanh has a simple pattern.

The output always stays between -1 and 1.

Example 1 — Negative Input

Suppose:

x = -2

Applying Tanh gives:

tanh(-2) ≈ -0.964

The output is negative and close to -1.

Example 2 — Zero

Suppose:

x = 0

Then:

tanh(0) = 0

Example 3 — Positive Input

Suppose:

x = 2

Applying Tanh gives:

tanh(2) ≈ 0.964

The output is positive and close to 1.

Example With Multiple Values

Suppose we have these inputs:

[-5, -2, -1, 0, 1, 2, 5]

Applying Tanh:

-5 → ≈ -1.000
-2 → ≈ -0.964
-1 → ≈ -0.762
 0 →    0
 1 → ≈  0.762
 2 → ≈  0.964
 5 → ≈  1.000

Tanh Examples

Why Do We Use Tanh?

One important property of Tanh is that its output is centered around zero.

This means Tanh can represent three situations:

This is different from Sigmoid, whose output is always between 0 and 1.

Tanh vs Sigmoid

The easiest way to remember the difference is:

Sigmoid → 0 to 1
Tanh    → -1 to 1

Tanh vs ReLU

Tanh Inside a Neuron

A neuron first calculates its weighted sum and adds the bias.

z = (x1 * w1) + (x2 * w2) + bias

Then Tanh is applied:

output = tanh(z)

For example:

x1 = 2
x2 = 3

w1 = 0.5
w2 = 0.4

bias = -1

z = (2 * 0.5) + (3 * 0.4) - 1

z = 1.2

Now apply Tanh:

tanh(1.2) ≈ 0.834

Another Neuron Example

Suppose the neuron produces:

z = -1.5

Apply Tanh:

tanh(-1.5) ≈ -0.905

Build Tanh With Python

Python provides Tanh through the built-in math module.

import math


def tanh(x):
    return math.tanh(x)


print(tanh(-2))
print(tanh(-1))
print(tanh(0))
print(tanh(1))
print(tanh(2))

Output:

-0.9640275800758169
-0.7615941559557649
0.0
0.7615941559557649
0.9640275800758169

Tanh With NumPy

When working with multiple values, NumPy can apply Tanh to an entire array.

import numpy as np


values = np.array([-2, -1, 0, 1, 2])

result = np.tanh(values)

print(result)

Output:

[-0.964 -0.762  0.     0.762  0.964]

Build Tanh From Scratch

We can also implement the mathematical formula ourselves.

import math


def tanh(x):

    return (
        math.exp(x) - math.exp(-x)
    ) / (
        math.exp(x) + math.exp(-x)
    )


print(tanh(-2))
print(tanh(0))
print(tanh(2))

This produces approximately:

-0.964
0.0
0.964

In real projects, using math.tanh() or numpy.tanh() is simpler and preferable.

One Limitation of Tanh

Tanh has a problem called saturation.

When the input becomes very large or very negative, Tanh gets extremely close to 1 or -1.

tanh(10)  ≈ 1
tanh(-10) ≈ -1

In these regions, the gradient becomes very small. This can make learning slower in deep networks.

Because of this, ReLU and its variants are commonly preferred for many modern hidden layers.

Where Is Tanh Used?

Tanh is especially important when learning about recurrent neural networks.

RNNs often need to represent information that can be positive, negative, or close to zero. Tanh naturally provides this range.

The Big Picture

Input
  ↓
Weights + Bias
  ↓
Linear Calculation
  ↓
z
  ↓
Tanh
  ↓
Value between -1 and 1
  ↓
Next Layer

Tanh takes the value calculated by the neuron and transforms it into a controlled range from -1 to 1.

Sigmoid vs Tanh vs ReLU