What Embeddings Are

A customer support team at a SaaS company had a knowledge base with 3,000 articles. Their search was keyword-based. When a customer typed "my dashboard is not loading," the search returned articles containing the words "dashboard," "not," and "loading" -- including an article about "Loading Custom Dashboard Themes" that was completely irrelevant. Meanwhile, the article titled "Troubleshooting Blank Screens After Login" -- the exact solution to the customer's problem -- did not appear because it never used the word "loading."

They switched to semantic search powered by embeddings. The same query now returns the blank screen troubleshooting article first, because the system understands that "dashboard is not loading" and "blank screen after login" mean the same thing.

That is what embeddings do. They turn text into meaning.

What You Will Learn

1. What embeddings are and why they matter
2. How high-dimensional vectors capture semantic meaning
3. Why cosine similarity works for comparing meanings
4. The fundamental limitation of keyword search that embeddings solve

From Words to Numbers

An embedding is a list of numbers that represents the meaning of a piece of text. Not the characters, not the word count, not the sentiment score -- the meaning.

When you send the sentence "The patient has a fever" to an embedding model, you get back something like this:

[0.0123, -0.0456, 0.0789, 0.0234, -0.0567, ... 1536 numbers total]

Each number captures one dimension of the text's meaning. No single number means anything on its own -- just like no single pixel defines a photograph. But together, these 1,536 numbers create a rich representation of what the text is about.

The critical insight: texts with similar meanings produce similar numbers. "The patient has a fever" and "The sick person has an elevated temperature" generate vectors that are close together in this 1,536-dimensional space. "The stock market crashed today" generates a vector that is far away.

High-Dimensional Space

Why 1,536 dimensions? Because meaning is complex.

Consider just two dimensions. You could place "happy" and "sad" on an x-axis (emotion valence) and "calm" and "excited" on a y-axis (energy level). That gives you four quadrants:

• Happy + Calm = Content
• Happy + Excited = Ecstatic
• Sad + Calm = Melancholic
• Sad + Excited = Distressed

But two dimensions cannot capture that "nostalgic" is a bittersweet emotion that is both happy and sad. Or that "anxious" is different from "excited" even though both are high-energy. Or that "professional" and "casual" are a separate axis entirely.

Real language has thousands of meaning dimensions: topic, tone, formality, domain, intent, time reference, certainty level, audience, complexity, and hundreds more. Each of the 1,536 dimensions in an embedding captures some blend of these semantic properties.

Cosine Similarity

Once you have two vectors, you need a way to measure how similar they are. The standard metric is cosine similarity.

Cosine similarity measures the angle between two vectors, ignoring their length. It returns a value between -1 and 1:

• 1.0: Identical meaning (or very close)
• 0.7-0.9: Related, similar topic
• 0.3-0.6: Loosely related
• 0.0: Unrelated
• Negative: Opposite or very different

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Example
similarity = cosine_similarity(embedding_a, embedding_b)
print(f"Similarity: {similarity:.4f}")

Why Keyword Search Fails

Keyword search matches character patterns. "Running" matches "running" but not "jogging." "Car" matches "car" but not "automobile." This creates two failure modes:

False negatives (missed results): The user searches for "how to fix a slow computer" but the best article is titled "Optimizing PC Performance" -- no keyword overlap.

False positives (irrelevant results): The user searches for "apple stock price" and gets articles about apple farming because the word "apple" matches.

Semantic search does not replace keyword search entirely -- it complements it. The most effective search systems combine both, as we will see in Chapter 2.