Python · Vector search Implementation

Python with KERNSTACK vector search

Python plus vector search is the RAG workstation pattern on KERNSTACK: Python workers do what Python does best — crawl, clean, chunk, and load the corpus into tables and buckets — while the vector index is simply declared over those sources and embeddings come from text-embed-v1 or your own endpoint. The retrieval layer needs no separate database because it is an index, maintained from the data itself.

Vector Search stack.json IaC

Talk to us Read Technical Documentation → Software Lifecycle Cloud →

Python Vector search

The division of labour is the lesson: Python owns corpus craft (parsing PDFs, normalising HTML, choosing chunk boundaries — decisions that determine retrieval quality), and the platform owns index mechanics. Declaring a text index beside the vector one costs a second entry and buys hybrid retrieval: exact identifiers match literally, prose matches semantically, one function merges the rankings.

Freshness and deletion behave the way data engineers wish vendor pipelines did. Edit a chunk and the index follows the source; delete it and its passages leave the results. When the embedding model changes, re-embedding is an index operation over the table — tokens at 3 credits per 1,000 with text-embed-v1 — evaluated against a fixed query set before cutover.

The Python SDK keeps the worker’s side typed and boring: build the client from the injected access key set, write chunk rows, and let the declaration do the rest. Corpus state — what is indexed, from when — is answerable from the table itself.

CLI

Plan Python Vector search, then deploy.

Terminal — CLI workflow kernstack cli

kernstack add worker --name api --blueprint work-python-minimal
kernstack plan
kernstack deploy

Docs for Python Vector search. Vector search on the platform.

python-vector-search.txt Python

kernstack add worker --name ingest --blueprint work-python-minimal
# ingest writes cleaned chunks into the corpus table / bucket;
# the vector index (below) is declared, not built by your code
kernstack plan && kernstack deploy

stack.json — Python + vector-search plan · deploy

{
  "resources": {
    "workers": [{ "name": "ingest", "runtime": "python" }],
    "tables": [{ "name": "corpus", "key_name": "chunk_id", "key_type": "S", "indexes": [] }],
    "search_indexes": [
      { "name": "corpus-vector", "index_type": "vector",
        "sources": [{ "type": "tables", "name": "corpus", "fields": ["title", "body"] }] },
      { "name": "corpus-text", "index_type": "text",
        "sources": [{ "type": "tables", "name": "corpus", "fields": ["title", "body"] }] }
    ]
  }
}

FAQ

Frequently asked questions

Does our Python code embed the documents?

Not necessarily. With a catalogue model the platform embeds from the declared sources; your worker focuses on preparing clean chunks. Choose a custom endpoint when a domain-specific embedding model should do the embedding instead.

How do we test a new chunking strategy?

Load the re-chunked corpus as new rows (or a parallel table), declare the candidate index, and run the same query set against both. Chunking becomes an experiment with data, not a migration with downtime.

What scale of corpus fits?

Product corpora — documentation sets, catalogues, archives, ticket histories. If retrieval becomes an extreme-scale speciality, keep the corpus here and graduate the index; the source data staying home is what makes that move cheap.

Connected in the Project Graph

Related capabilities, architectures, and comparisons

Ship your stack on the Software Lifecycle Cloud.

Closed preview on dedicated European infrastructure. Bring a repository or start from a blueprint.

Talk to us See pricing & credits →