Python · Workers Implementation

Python with KERNSTACK workers

Python runs on KERNSTACK as workers — "runtime": "python" — which is where the ecosystem’s strengths belong: document extraction, data pipelines, transcription, and model-adjacent processing. A Python worker declares its sizing and idle behaviour, mounts bucket volumes when files should look like a filesystem, and joins workflows as steps between human decisions.

Container Workers stack.json IaC

Talk to us Read Technical Documentation → Software Lifecycle Cloud →

Python Workers

The shape fits Python’s centre of gravity: long, library-heavy jobs that start when work exists. Upload a PDF to the bucket and the workflow starts the extraction worker; it reads from the mounted volume (a frozen snapshot of the bucket prefix at /volumes/ by default), writes structured results to tables through the typed Python SDK, and exits — min_instances: 0 means the next document pays the startup, not the calendar.

Data work composes with the rest of the stack without glue services: tables for staging and published records, search indexes over the extracted text, inference endpoints when a model should be a neighbour rather than an import. The pipeline is a stack.json a data engineer and a reviewer can read together.

Sizing honesty applies: workers declare up to 1024 MiB and 8 cores in the standard shape. Product-scale extraction and pipeline work fits comfortably; cluster-scale data engineering belongs to specialised tools, with outputs landing here for the product to serve.

CLI

Plan Python Workers, then deploy.

Terminal — CLI workflow kernstack cli

kernstack add worker --name api --blueprint work-python-minimal
kernstack plan
kernstack deploy

Docs for Python Workers. Workers on the platform.

python-workers.txt Python

kernstack add worker --name extract --blueprint work-python-minimal
# workers/extract: your Python; datasets arrive as bucket volumes or SDK reads
kernstack plan
kernstack deploy --only worker:extract

stack.json — Python + workers plan · deploy

{
  "resources": {
    "buckets": [{ "name": "documents", "volumes": [{ "name": "docs", "source_prefix": "incoming/" }] }],
    "workers": [{ "name": "extract", "runtime": "python", "memory_mb": 1024,
      "volumes": [{ "name": "docs" }], "min_instances": 0, "idle_timeout_ms": 120000 }]
  }
}

FAQ

Frequently asked questions

Can Python workers use our usual libraries?

Workers are container processes with a system-package catalog and your dependency files; the data-ecosystem staples (parsing, imaging, dataframe work) are the intended workload. What changes is only how the job is declared and metered.

How do workers read large files?

Through bucket volumes: declare a volume over a bucket prefix on the bucket, mount it by name on the worker, and read files from the mount path. Originals also remain addressable as objects through the SDK.

Do Python jobs need to run constantly?

No. Workflow triggers start them when work arrives, and min_instances: 0 with an idle timeout stops them afterwards. Nightly pipelines cost their minutes, not a standing process.

Connected in the Project Graph

Related capabilities, architectures, and comparisons

Ship your stack on the Software Lifecycle Cloud.

Closed preview on dedicated European infrastructure. Bring a repository or start from a blueprint.

Talk to us See pricing & credits →