Dot Product

What Is a Dot Product?

A dot product is a calculation performed between two vectors of the same length.

The easiest way to understand it is:

1. Multiply corresponding values
2. Add the results

For example, consider these two vectors:

A = [2, 3]

B = [4, 5]

We multiply the first values together and the second values together.

2 × 4 = 8

3 × 5 = 15

Then we add the results:

8 + 15 = 23

Therefore:

A · B = 23

The important point is that two vectors produced one number.

How Does the Dot Product Work?

Let's look at the calculation step by step.

Suppose:

A = [1, 2, 3]

B = [4, 5, 6]

First, multiply corresponding values:

1 × 4 = 4

2 × 5 = 10

3 × 6 = 18

Now add the results:

4 + 10 + 18 = 32

Therefore:

[1, 2, 3] · [4, 5, 6] = 32

Dot Product Formula

We can write the same calculation using mathematical notation.

A = [a₁, a₂, a₃]

B = [b₁, b₂, b₃]

Their dot product is:

A · B
=
a₁b₁ + a₂b₂ + a₃b₃

For vectors with more values, the same pattern continues.

A · B
=
a₁b₁ + a₂b₂ + a₃b₃ + ... + aₙbₙ

You do not need to memorize the notation immediately. Remember the simple rule:

Multiply
   ↓
Multiply
   ↓
Multiply
   ↓
Add everything

Another Simple Example

Let's calculate another dot product.

A = [3, 2]

B = [5, 4]

Step 1 — Multiply corresponding values:

3 × 5 = 15

2 × 4 = 8

Step 2 — Add the results:

15 + 8 = 23

Therefore:

[3, 2] · [5, 4] = 23

Why Must the Vectors Have the Same Length?

The dot product compares values at the same positions. Therefore, both vectors need to have the same number of values.

For example:

A = [1, 2, 3]

B = [4, 5, 6]

This works because both vectors contain three values.

1 ↔ 4
2 ↔ 5
3 ↔ 6

But these vectors cannot be directly used for a dot product:

A = [1, 2, 3]

B = [4, 5]

One vector has three values while the other has only two. There is no corresponding value for the third position.

Dot Product Produces a Scalar

Remember that a vector contains multiple values, while a scalar is one single numerical value.

Vector

[2, 3, 4]


Vector

[5, 6, 7]


        ↓
    Dot Product
        ↓


Scalar

56

Let's verify it:

2 × 5 = 10

3 × 6 = 18

4 × 7 = 28

10 + 18 + 28 = 56

So:

[2, 3, 4] · [5, 6, 7] = 56

The final result is one number, so it is a scalar.

Dot Product as a Similarity Score

One important use of the dot product is comparing vectors.

Consider:

A = [1, 0]

B = [1, 0]

Their dot product is:

1 × 1 + 0 × 0

= 1 + 0

= 1

Now consider:

A = [1, 0]

C = [0, 1]

Their dot product is:

1 × 0 + 0 × 1

= 0 + 0

= 0

In this simple example, the first pair has a larger dot product because the vectors point in the same direction, while the second pair is different.

Dot Product in AI

Dot products are used heavily in AI and machine learning. They allow models to combine and compare numerical representations.

For example, an AI model can represent information using vectors:

Vector A
↓
[1, 2, 3]


Vector B
↓
[4, 5, 6]

The model can calculate:

A · B

= 1×4 + 2×5 + 3×6

= 4 + 10 + 18

= 32

The resulting number can then be used as part of a larger calculation.

Dot Product in Attention

Dot products are also used in attention mechanisms. In attention, a query is compared with keys using dot products to produce attention scores. :contentReference[oaicite:3]{index=3}

Query = [1, 0]

Key 1 = [1, 0]
Key 2 = [0, 1]
Key 3 = [1, 1]

Compare the query with each key.

[1, 0] · [1, 0] = 1

[1, 0] · [0, 1] = 0

[1, 0] · [1, 1] = 1

Therefore the attention scores are:

[1, 0, 1]

These scores can then be converted into attention weights using softmax. This is the basic idea behind part of the attention calculation. :contentReference[oaicite:4]{index=4}

Dot Product With Python

We can calculate a dot product in Python using a simple loop.

A = [1, 2, 3]

B = [4, 5, 6]

result = 0

for i in range(len(A)):
    result += A[i] * B[i]

print(result)

The result is:

32

The code performs exactly the same mathematical steps:

1 × 4
+
2 × 5
+
3 × 6
=
32

In real AI projects, libraries such as NumPy and machine learning frameworks provide optimized operations for this calculation.

Dot Product vs Element-by-Element Multiplication

Do not confuse a dot product with simply multiplying corresponding values.

Element-by-element multiplication keeps all the results:

A = [1, 2, 3]

B = [4, 5, 6]

Element-wise multiplication:

[1×4, 2×5, 3×6]

= [4, 10, 18]

The dot product goes one step further and adds those values:

4 + 10 + 18

= 32

So remember:

Element-wise multiplication
        ↓
[4, 10, 18]


Dot product
        ↓
32

Simple Mental Model

If you forget the formula, remember this simple process:

Two vectors
     ↓
Match corresponding positions
     ↓
Multiply each pair
     ↓
Add all products
     ↓
One number

That is the dot product.

Once this becomes familiar, concepts such as attention scores and matrix multiplication become much easier to understand.