The engineering bottleneck with assembled stacks
Marketers maintain sitemaps the way they were maintained in 2009: a plugin here, a hand-edited file there, lastmod dates of fiction. Pages ship and never enter the sitemap; removed pages stay listed for a year; and the SEO postmortem (“why did the new section take six weeks to index?”) discovers the map, filed under somebody else’s job.
Architecture on the Software Lifecycle Cloud
The generator is a worker on a cron workflow with three inputs, all inside the project: the gateway’s public routes, the published content prefix’s objects with their modification data, and an exclusions table for what must stay unlisted. It writes sitemap.xml (or a sharded index for large sites) into the site bucket and logs the run — URL count, additions, removals — so a sudden drop is a visible event, not a slow mystery.
Exclusions are data, additions are automatic
The useful asymmetry: publishing a page adds it by default, because the generator reads reality; keeping a page out is an explicit exclusion row, reviewed like other data. Launch checklists shrink by one forgotten task, and the exclusion list becomes the honest registry of what the business deliberately hides from search.
- lastmod comes from content records, so search engines get dates that mean something
- Run logs record URL-count deltas — a crawl-visible regression pages the team, not the quarter
- Large catalogues shard into a sitemap index generated by the same worker
A sitemap is hygiene, not strategy
This keeps the mechanical contract with search engines exact; it does not make thin pages worth indexing. The generator’s run log does quietly help there too — a site that can see its published count per section tends to notice the section that grew by four hundred near-duplicate pages before Google does.
Frequently asked questions
On the schedule you declare — nightly is typical, and the workflow can also run after publishing events. Each run overwrites the file in the site bucket; the gateway serves the fresh copy immediately.
Add its path to the exclusions table. Exclusion is explicit and reviewable data, while inclusion is the automatic default for anything public — the asymmetry that keeps both launches and delistings correct.
The worker shards output into multiple sitemap files under a sitemap index, the standard mechanism search engines expect. The inputs do not change: routes, published content, exclusions.
Related capabilities, architectures, and comparisons
Ship your stack on the Software Lifecycle Cloud.
Closed preview on dedicated European infrastructure. Bring a repository or start from a blueprint.