Local Queries; Cloud Quality. Voyage-4's Shared Embedding Space

The tradeoff that wasn’t

Building semantic search has always meant picking your poison.

Cloud embedding models (OpenAI, Cohere, Voyage) give you the best retrieval quality. But you pay per token, you’re dependent on network latency, and your (possibly sensitive) queries flow through someone else’s infrastructure.

Local embedding models give you speed ¹, privacy, and zero marginal cost. But retrieval quality suffers. You’re accepting worse results because you have to, not because you want to. Voyage AI’s new model family, voyage-4, breaks this tradeoff in a way I haven’t seen before.

A crash course in vector search

If you’re not familiar with vector search, here’s the basics:

• A specially trained AI model processes your documents into ‘embeddings’ — this is a multi-dimensional mathematical map of your data, represented as numbers like ‘0.1’ (they’re coordinates in a many-dimensional space… it’s fancy math).
• You store those ‘vectors’ (named because they are coordinates) in a vector database or ‘store’.
• You search them by using the same model to turn your question into embeddings, and the vector database uses your question’s embeddings to find the closest coordinates in the multi-dimensional space. It finds the texts that are most ‘semantically similar’ — the closest ones in this multi-dimensional version of a x-y graph.
• It finds text that is most similar in meaning to your question. Not similar words. Similar ideas. That’s what is different about vector search compared to traditional keyword search.

The new voyage-4 lineup

One embedding space, four models.²

The Voyage 4 series includes four models:

• voyage-4-large (cloud): Flagship cloud model, state-of-the-art accuracy
• voyage-4 (cloud): Balanced performance and efficiency
• voyage-4-lite (cloud): High throughput, lower cost
• voyage-4-nano (local): Open-weight, Apache 2.0, runs locally on CPU. Available on Hugging Face.

The key insight: all four models produce embeddings in the same mathematical space.

That’s not normal. Typically, embeddings from different models can’t be compared—they’re like measurements in different units with no conversion table.

If you index with Model A, you query with Model A. No mixing. Until now. Voyage 4 models can be mixed freely.

Embeddings from voyage-4-large can be searched using voyage-4-nano. The vectors are compatible.

Asymmetric retrieval: a different cost structure

This compatibility enables a pattern called asymmetric retrieval: using different models for documents versus queries.

Think about how retrieval systems actually run:

1. Documents get embedded once. Maybe you re-index weekly or when files change, but it’s fundamentally a batch operation. An upfront cost.
2. Queries get embedded constantly. Every search, every RAG call, every agent memory lookup. This is your ongoing operational cost.

With asymmetric retrieval, you can optimize each side independently:

• Embed documents with voyage-4-large (maximum quality, pay once)
• Embed queries with voyage-4-nano (zero cost, local inference)

You get the retrieval quality benefits of the large model baked into your document representations. But your per-query cost drops to effectively zero.

The numbers for code search

This isn’t theoretical.

Voyage published benchmark results on RTEB, a standard retrieval evaluation suite.

For code search specifically, querying against vectors generated by Voyage-4-large:

• voyage-4: 87 (vs. 86 querying against its own embeddings)
• voyage-4-lite: 86 (vs. 83 querying against its own embeddings)
• voyage-4-nano: 84 (vs. 81 querying against its own embeddings)

Two huge things about those numbers:

1. All 4 models outscore any other model available on code search tasks. Even without asymmetric retrieval, Voyage-4-nano outperforms Google Gemini text-embedding-001, which clocks in at 76 for RTEB-Code, along with every Cohere model, every OpenAI model, etc (by a wide margin) when querying its own vectors. That… is insane.
2. voyage-4-nano only loses 3 percentage points of quality versus the baseline model when querying against voyage-4-large embeddings. The result: voyage-4-nano democratizes high quality retrieval
   • No API costs at query time
   • Sub-10ms latency instead of 100ms+ network round-trips
   • fully offline capability
   • No rate limits or quota concerns
   • complete query privacy

All for the cost of a loss of 3 percent.

Real talk. People get obsessed with benchmarks, and squeezing the best performance out of models. And model training companies chase benchmarks and publicize these numbers even though you shouldn’t rationally deploy the model like that.

The practical difference in search quality between 87 and 84 or 81? Absolutely nothing. It means your very accurate results return in slightly off order — the #8 most relevant should have been #9. Considered against cost, nearly every application should choose nano over large. ³

That is a trade worth making. Every. Single. Time.

What this means for CodeWeaver

I’m building CodeWeaver as semantic code search infrastructure— an MCP server and CLI tool that helps AI agents and developers find relevant code across repositories. This new asymmetric retrieval pattern fits CodeWeaver’s architecture perfectly:

• Indexing happens once. We re-index continuously as files change, but for mature codebases, most of the cost is up front. If your codebase is newer (like CodeWeaver itself) and prone to major and frequent change, you’re probably better off sticking to voyage-4-nano altogether.
• I’m planning to add voyage-4-nano as an optional dependency that ships with CodeWeaver. At a few hundred megabytes, you can directly include it in a package. You get world class embeddings right out of the gate. No API keys. No network dependency. All yours. Right there.
• Querying, which is continuous, can be fully local and free. Always.

The 32K context window

All of these new models have a 32k token context window. Most local models? 512. Some have 4096 or 8092. 32k is something you only see in enormous models.

I don’t advise embedding documents at that length, but what it means is you can maintain context. Because of context windows, you typically need to “chunk” your documents — cut them up into pieces. How you do that can make or break search quality (this is an area where I’m most proud of how CodeWeaver does it — smartly.)

If it’s not done well, and it usually isn’t, you end up with split context — one chunk contains half of an important idea, and the next chunk has the rest. The model may not be able to infer that overall meaning from the parts. Your results suffer.

With a 32k context window, you can make very strategic choices about where to split your chunks.

For the developers

If you’re building retrieval systems, here’s a quick version of the pattern:

from collections.abc import Sequence

import numpy as np

from sentence_transformers import SentenceTransformer
from voyageai import Client
from voyageai.object import EmbeddingObject

voyage_client = Client(api_key="YOUR_SECRET_API_KEY_THAT_SHOULD_NOT_BE_HARDCODED_LIKE_THIS")

# embedding documents using voyage-4-large

def embed_documents(documents: Sequence[str]) -> EmbeddingObject:
    """Generate embeddings using the voyage remote API."""
    return voyage_client.embed(texts=documents, model="voyage-4-large", input_type="document")


# embedding a query; we'll use SentenceTransformers but you could also use Torch and several others

transformer = SentenceTransformer(model_or_path="voyageai/voyage-4-nano", trust_remote_code=True, truncate_dim=1024) # default dimensions

def embed_query(query: str) -> np.ndarray:
    """Use sentence-transformers to embed a query."""
    return transformer.encode_query(query)

# query:

query = "What is the meaning of life, the universe, and everything?"
response = embed_query(query) # 42

When to Use Asymmetric Vector Search

• Query volume is high relative to document volume
• Your corpus does not regularly change, or changes are minimal
• You need offline or air-gapped query capability
• You’re worried about query privacy

It might not be the best choice if…

• You have a large, mostly unchanging, corpus already embedded with another provider or set of models.
• Your corpus changes regularly changes significantly (use voyage-4-nano all the time!) The 3-point gap is unacceptable for you (I’d like to hear why, though…)

For code search, agent memory, documentation retrieval, and most RAG applications, the tradeoffs favor asymmetric retrieval.

Getting started

• voyage-4-nano is available on Hugging Face under Apache 2.0
• voyage-4-large and the rest of the family are available via the Voyage API and docs. First 200M tokens are free (that’s a lot).

CodeWeaver support

I hope to ship this new approach in CodeWeaver this week with Alpha 6. It has been over a month since Alpha 5, and for good reason — there’s a lot coming:

• Support for this pattern!
• Move to a modular repo structure with installable packages. If you just want CodeWeaver’s semantic intelligence and characterization, its provider support, or its indexing engine, and not the server component — you can do that.
• Replaced the existing registry pattern with dependency injection. I should have done this from the start, so this was a painful lesson. Dependency injection will enormously improve testability and maintainability. I’ll write on this later, but essentially, I felt I had to make this change to get CodeWeaver’s reliability where I want it to be.

If you’re building retrieval systems and want to compare notes, find me on Twitter @SpyToFounder or Bluesky adam.knitli.com.

---

I don’t have an affiliation or relationship with MongoDB or VoyageAI. This post gushes because they put out something genuinely awesome.

Footnotes

1. Your local setup is probably slower than a specialized cloud server, but you don’t have the latency of requesting from the cloud. Cloud embeddings are usually much faster for embedding a corpus (group of documents), but for something short like a query, that difference becomes negligible. ↩

2. As discussed in the crash course, an ‘embedding space’ is a multi-dimensional coordinate plot — a map. ↩

3. You might be left wondering why Voyage would do this. I was. It basically undermines their own paid business. My guess? Last year, MongoDB acquired VoyageAI. MongoDB’s strategy isn’t “make money from generated embeddings”. It’s “get people using our database so we can cross-sell”. This is likely part of a larger MongoDB strategy to integrate traditional and vector databases and embeddings into a single coherent capability. ↩