Embedding Strategy: How to Choose, Test and Ship One.

An embedding strategy is your plan for turning content into numbers a machine can search by meaning. It covers four choices: how you cut up your content, which model you use, how you store the results, and how you test it all.

Get these right and your AI finds the right answer. Get them wrong and it sounds sure while quoting the wrong page.

This guide is for founders and teams who run a knowledge business. You do not need to write code to follow it. You do need to know what to ask for.

What an embedding actually is

Think of a huge library. Each book gets a spot on a map. Books about the same thing sit close together.

An embedding is that spot on the map. It is a list of numbers that captures what a piece of content means. AWS describes embeddings as numerical representations that capture relationships among real-world data. That lets a system compare meaning, not only exact words.

So a search for "cancel my plan" can find a page called "How to end your subscription". The words differ. The meaning is close.

Encoding versus embedding

People mix these up.

• Encoding turns a label into a number. "Red" becomes 1, "blue" becomes 2. Nothing about meaning is kept.
• Embedding turns something into a short list of numbers where closeness means similarity.

AWS makes the same split: encoding converts categories into numbers, while embeddings are low-dimensional vectors that show how similar things are. (A vector is just a list of numbers.)

Word embeddings and modern models

Older methods like Word2Vec gave each word one spot on the map. The trouble is that one word can mean two things. "Bank" at a river is not "bank" with your money.

Newer models such as BERT fix this. AWS notes that BERT can produce contextual word embeddings, so the same word gets different numbers in different phrases. Today's text embedding models build on that idea.

How an embedding strategy fits together

Most systems that use embeddings follow the same loop. It is like a good filing clerk.

1. Cut your content into pieces (chunks).
2. Embed each piece and store it in a vector database (a store built to find close numbers fast).
3. Embed each question the same way.
4. Rank stored pieces by how close they sit to the question.
5. Hand the best pieces to the AI so it can write the answer.

Snowflake describes this pipeline as splitting content into retrievable units, embedding and storing each one, embedding each query, and ranking by closeness.

One rule saves a lot of pain. Microsoft's guidance says to embed the user's question with the same model you used for the content. Mix models and the map no longer lines up.

This whole pattern is called RAG (retrieval-augmented generation). In kitchen terms: the AI is a cook, and retrieval fetches the right ingredients before cooking starts. Bad ingredients, bad dish.

Step 1: Start with the job, not the model

Before you compare any model, write down the job.

• Who asks questions? Clients, staff, or your own AI agents?
• What do they ask? Short lookups, or long "how do I..." questions?
• What content answers them? Courses, call transcripts, playbooks, a help centre?
• How fast must it reply? And what can you spend?

Qdrant says model choice must begin with the actual task. Models differ in how well they handle similarity, retrieval and question answering.

A model that is great at matching two similar sentences may be average at finding an answer inside a long document. Same tool, different job.

Step 2: Build a small test set first

Most teams skip this step. It matters most.

Make a "golden set". That is a list of real questions, each with the approved answer and the source page. Microsoft suggests a golden dataset with questions, approved answers, metadata, source links and alternate phrasings.

Build one like this:

• Pull 50 to 100 real questions from emails, support tickets and call notes.
• Write the correct answer for each.
• Mark which page or chunk holds that answer.
• Add two or three ways each question could be worded.

This takes a few hours. It saves weeks of guessing.

Qdrant is blunt about why. A ground-truth set built from your own data is needed for an informed decision, and public leaderboards alone cannot pick your best model.

Step 3: Choose a model

Now compare a few models against your test set. Do not test twenty. Test three or four.

What to compare

Qdrant lists the factors: language support, sequence length (how much text fits at once), model size, speed, delay and cost. Weigh each against your needs.

Factor	Plain question to ask
Language	Does it handle every language your clients use?
Input length	Will your chunks fit inside its limit?
Size	Can your servers run it, or will you call an outside service?
Speed	How long does one lookup take?
Cost	What does it cost at your real volume?
Fit	Does it score well on your questions, not just a public chart?

General or domain model?

Microsoft's view is simple. Test strong general models for general content. If your field is specialised, such as law, medicine or finance, consider a domain model or fine-tuning.

Most knowledge businesses can start with a good general model. Their content is plain language.

Do not trust one average score

Leaderboards often show one blended number. That number mixes many tasks. Vercel's production guide says retrieval work should be judged mainly on retrieval-specific results, not an overall average.

A recent academic report goes further. The July 2026 arXiv paper argues that picking an embedding model should be treated as part of the whole retrieval pipeline, including indexing, search and chunking. It is not a standalone leaderboard call.

Step 4: Cut your content well

Chunking decides what can be found at all. If the answer is split across two chunks, the system may fetch only half.

Microsoft's advanced RAG guide says chunking determines what units can be retrieved. It suggests you test several styles:

• By section
• By paragraph
• By sentence
• With overlap (each chunk repeats a bit of the last)
• Sliding window (a moving frame across the text)

The small-to-big trick

Think of a book index. You find the exact line, then read the paragraph around it.

Microsoft calls this Small2Big. You search with a small sentence-level unit, then pass the nearby sentences or whole paragraph to the AI for context. You get precise matching and full answers.

Add labels to every chunk

Snowflake recommends storing metadata with each piece: source, date, category, region and permissions. Then the app can filter eligible records before or alongside the similarity ranking.

This matters for knowledge businesses. A client on a basic plan should not get answers from material only premium members paid for. Permissions are a label, set up on day one.

Clean content in, clean answers out

If your source is a messy call transcript, tidy it first. Remove filler. Mark who is speaking. Add a title to each section.

If your content includes video or images, convert it to text first. Microsoft says to turn non-text media into a text form that the model can process before embedding.

And if you publish a lot of content on a schedule, keep quality steady. A guide on automated content generation can help you think through that workflow. Whatever gets published is what your system will later retrieve.

Step 5: Watch storage and cost

Every number in an embedding takes space. More numbers (dimensions) usually means more detail, but also more cost.

Vercel gives a clear example. A float32 number takes 4 bytes. For 10 million vectors, that is about 14.3 GiB at 384 dimensions, 28.6 GiB at 768, 57.2 GiB at 1,536, and 114.4 GiB at 3,072. Those figures show how fast size climbs.

Most knowledge businesses hold far fewer than 10 million chunks. Still, the maths shows the trade-off.

You can shrink vectors. Microsoft says dimensionality reduction can cut storage and compute and should be done after the embeddings are made. PCA is one named method.

Vercel adds two more tools: Matryoshka truncation and quantisation. Both shrink the footprint. But any cut in size or precision should be re-tested on your own content. Smaller is only better if the answers stay right.

We cover the storage side in more depth in our guide to choosing a vector database for RAG.

Step 6: Pick how the search runs

Searching every vector one by one is exact but slow at scale. Faster indexes skip some work.

Google Cloud explains the trade-off. Approximate nearest neighbour indexes improve speed but give up some recall. Brute-force search gives exact distance comparisons. (Recall means how many of the right items you actually find.)

Small collection? Exact search is often fine. Big collection with strict speed needs? Use an approximate index, and check your test set to see what you lost.

Step 7: Know when embeddings are not enough

Meaning search is strong, but it misses things.

Say a client types an error code, "E-4021". Or a product name. Or a person's name. Meaning search may rank these oddly, because the numbers care about topic, not exact spelling.

Snowflake advises testing hybrid retrieval when your content holds product names, identifiers, error codes or special terms. Keyword search keeps exact matches that meaning search can underweight.

Google Cloud's vector search supports both semantic and hybrid lexical search. Hybrid is like asking two staff members. One understands the idea. One knows the exact word. You listen to both.

A reranker can add a final check. It scores the top candidates again and puts the best first, as AWS describes for semantic search.

Snowflake sums it up well: retrieval quality is a design property. Embeddings, nearest-neighbour search, filters, keyword matching and reranking work together to decide what evidence reaches the AI.

Step 8: Fine-tune only when you have proof

Fine-tuning means training a model a little more on your own data. It sounds clever. It costs time.

Microsoft's advice has an order. Try a general model, then a domain model. If neither meets your needs, fine-tune with your data.

So the evidence comes first. Your test set shows a clear gap, such as the model missing your own terms. Then you fine-tune. Not before.

Step 9: Test before you launch

Microsoft calls retrieval performance the most practical way to judge embedding quality. Embed real content, run searches with real questions, and check that the correct items come back.

A simple routine:

1. Run every golden question through the system.
2. Check if the right chunk shows up in the top five.
3. Record the score.
4. Change one thing: model, chunk size or search type.
5. Run again. Compare.

Change one thing at a time. Change two and you will not know which one helped.

After launch, keep reviewing. Look at user feedback and check which chunks the AI actually used. Wrong answers are a map to your weak spots.

Our view: the strategy is about your expertise

We build AI systems for founders who earn $1M+ from their own methods. In our work, most embedding projects stall on content, not maths. The founder's real judgement sits in their head, in call recordings and in half-finished documents. Nobody has cut it into clean pieces with labels.

So we do the human work first. We sit with the founder and pull out how they decide. What do they check? What do they ignore? What would they never say to a client?

Then we shape that into content a system can find. The model comes second. A modest model on well-organised IP beats a fancy model on a dump of files.

And we test with the founder's real questions. If the system cannot answer like the founder would, it is not ready. Our AI performance benchmarks post shows what we track.

One more thing. Embeddings are memory, and memory needs rules. What to keep, what to drop, who may see what. We wrote about that in AI agent memory.

James Killick

Want us to build it with you?

If you run a consulting or education business above $1M, and your team keeps asking you the same questions, this is our job.

We offer a 90-day done-with-you program at $18,500. We also offer a Foundation Month at $3,500 as a first step. You can read both on our AI consulting services page.

Prefer to build on your own? Our platform costs $90 a month or $900 a year. See Platform Access.

We say what works and what does not. If your content is not ready, we will tell you.

A quick checklist

Use this before you sign off any embedding plan.

• We wrote down the job and who asks the questions.
• We built a golden set of real questions with approved answers.
• We tested three or four models on our own content.
• We tried at least two ways of chunking.
• Every chunk has labels: source, date, category, permissions.
• We used the same model for content and questions.
• We checked storage and cost at real volume.
• We tested keyword plus meaning search for exact terms.
• We have a plan to review wrong answers after launch.

Sources

• What are Embeddings in Machine Learning? (AWS)
• Develop a RAG Solution on Azure: Generate Embeddings Phase (Microsoft Learn)
• Build Advanced Retrieval-Augmented Generation Systems (Microsoft Learn)
• Embeddings for Semantic Search and RAG (Snowflake)
• How to Choose an Embedding Model: Evaluation and Tradeoffs (Qdrant)
• Choosing a Text Embedding Model: A Practical Report (arXiv)
• How to choose an embedding model for production retrieval (Vercel)
• Introduction to embeddings and vector search (Google Cloud BigQuery)