Python · Inference Implementation

Python with KERNSTACK inference

Python is how custom models arrive on KERNSTACK: declare an inference_endpoint with "runtime": "python", point it at the bucket and prefix holding your weights, and the platform serves it as a project resource — invocable from functions, workers, and workflow ai steps exactly like a catalogue model. Fine-tunes and specialist classifiers stop needing a separate serving platform.

Model Inference stack.json IaC

Talk to us Read Technical Documentation → Software Lifecycle Cloud →

Python Inference

The endpoint declaration carries the operational truth: runtime (with an optional custom image when the stack outgrows the standard one), model location as bucket plus prefix, and sizing. Versioning is a key-path convention — churn/v11 beside churn/v12 — so promoting a model is a one-line declaration change with the old version still addressable during evaluation.

Callers cannot tell custom from catalogue, and that is the design’s payoff. A workflow ai step names an endpoint instead of a model; a Go function invokes either through the same project surface. Experiments compare a fine-tune against text-embed-v1 call-sites without an integration project in between.

Metering follows the endpoint’s compute while it works. A classifier that scores nightly spends its minutes; the catalogue alternative would have spent tokens. The choice between them becomes arithmetic a team can do in one sitting, from posted rates.

CLI

Plan Python Inference, then deploy.

Terminal — CLI workflow kernstack cli

kernstack add worker --name api --blueprint work-python-minimal
kernstack plan
kernstack deploy

Docs for Python Inference. Inference on the platform.

python-inference.txt Python

kernstack add worker --name prepare-endpoint --blueprint work-python-minimal
# 1) job writes model artefacts to the models bucket under a versioned prefix
# 2) declare the endpoint (below)  3) call it by name from any step
kernstack plan && kernstack deploy

stack.json — Python + inference plan · deploy

{
  "resources": {
    "buckets": [{ "name": "models" }],
    "inference_endpoints": [{ "name": "churn-classifier", "runtime": "python",
      "model_bucket": "models", "model_prefix": "churn/v12/",
      "memory_mb": 1024, "cpu_cores": 4 }],
    "workflows": [{ "name": "nightly-scoring", "triggers": [{ "type": "cron", "cron": "30 2 * * *" }],
      "steps": [{ "id": "score", "ai": { "endpoint": "churn-classifier",
        "prompt": "Score this account for churn risk." } }] }]
  }
}

FAQ

Frequently asked questions

How do model weights reach the endpoint?

Through your bucket: training or preparation jobs write artefacts under a versioned prefix, and the endpoint declares model_bucket and model_prefix. No registry or artefact service stands between.

Can we A/B a custom model against a catalogue model?

Call sites name the target, so splitting by a table-driven rule (or running both in a workflow and comparing outputs) is ordinary project code. The comparison happens on your data, in your zone.

What does a custom endpoint cost when idle?

Like other compute, idle endpoints trend to their storage: the model files in the bucket. Serving capacity is a declared sizing decision with a meter that follows actual work.

Connected in the Project Graph

Related capabilities, architectures, and comparisons

Ship your stack on the Software Lifecycle Cloud.

Closed preview on dedicated European infrastructure. Bring a repository or start from a blueprint.

Talk to us See pricing & credits →