Stop bad deploys with runtime validation

Someone ran the v2.3 package against production. Production was already on v2.5. The deploy didn’t error — it just quietly started rolling the schema backward, because nobody told it not to. They caught it three tables in. That’s the expensive way to learn your pipeline had no guardrail.

Hey folks. I’m Forge Barrett, master of the Content Forge here at SchemaSmith. Last time we guarded the pull request — structural validation and a governance contract, no database in sight. Today we move the gate to the other end of the line: deploy time, against live state. These gates run inside the quench, they read the real database, and when the answer comes back wrong they stop the whole thing before a single table changes.

The first gate: am I on the right server?

ValidationScript is a required property on Product.json, and it runs first — before any template, against the server’s admin database (master, postgres, or information_schema). It’s your identity check. One question: am I on the server I think I am?

Here’s the SQL Server form — the expected dependency database exists, and the server clears a version floor:

SELECT CAST(CASE WHEN EXISTS (SELECT 1 FROM sys.databases WHERE name = '{{DependencyDb}}')
  AND CAST(SERVERPROPERTY('ProductMajorVersion') AS INT) >= 14 THEN 1 ELSE 0 END AS BIT)

Point it at the right server and the quench rolls on:

Validate Server

Now point the dependency check at a database that isn’t there — swap {{DependencyDb}} for something that doesn’t exist — and run it again:

Validate Server
Invalid server for this product

Exit code 3. The quench never reaches a target database — no connection to a tenant, no baseline, nothing. One gate, one question, and the wrong server gets turned away at the door. That’s the identity check. It’s cheap, it’s first, and it’s the easy half.

The contract every gate lives by

Before the harder half, one rule you can’t skip: a validation script must return a truthy scalar, or the gate fails. A 1, a true, a non-zero — that passes. A 0, a false, and — this one bites people — a NULL all count as fail. A script with a logic error that returns nothing doesn’t sail through; it stops the deploy.

Each engine spells the truthy value its own way, but the contract is identical:

SQL Server	PostgreSQL	MySQL
`SELECT CAST(… AS BIT)`	`SELECT EXISTS(…)` (native boolean)	`SELECT EXISTS(…)` (0/1)

Write the check in your engine’s native dialect, return something truthy, and the gate opens. Return nothing — or the wrong thing — and it holds.

The gate that stops a rollback

Here’s the one that would’ve caught our opening disaster. Two gates work as a pair, both at the template level, both running against the target database:

BaselineValidationScript runs before a database’s quench. It reads a small version registry — standing infrastructure you provision once, not something the package ships — and passes only when the stored version is one this release is allowed to run over.
VersionStampScript runs after the quench succeeds, recording this release’s version into that registry.

Version 1 of the package requires the registry to be at 1 or below. Version 2 requires 2 or below. Here’s v1’s baseline on SQL Server:

SELECT CAST(CASE WHEN COALESCE((SELECT MAX(Version) FROM dbo.SchemaVersion WHERE Product = 'Shop'), 0)
  <= 1 THEN 1 ELSE 0 END AS BIT)

And v1’s stamp — a plain upsert of the version number:

MERGE dbo.SchemaVersion AS t USING (SELECT 'Shop' AS Product, 1 AS Version) AS s
  ON t.Product = s.Product
  WHEN MATCHED THEN UPDATE SET Version = s.Version
  WHEN NOT MATCHED THEN INSERT (Product, Version) VALUES (s.Product, s.Version);

Now watch the sequence. Deploy v1 — the registry is empty, treated as 0, baseline passes, and it stamps 1. Deploy v2 — baseline sees 1, which is <= 2, passes, and stamps 2. Now do what our unlucky friend did: re-run the older v1 against a database that’s already at 2:

Validate Baseline
Invalid baseline for this release

Exit code 2. The registry stays at 2. Not one table moves. The older release can’t run over the newer database, because the gate read the version and said no — before the quench, not three tables in.

Check yourself: Your BaselineValidationScript checks the version registry, but the registry table doesn't exist yet on a brand-new database. Where should it come from?

Provision it as standing infrastructure — before the first deploy — not as a table inside the package. The baseline runs before the quench creates any tables, so a package-managed registry wouldn’t exist yet when the baseline reads it (and on all four engines the read fails at that point). Treat the version registry the way you’d treat any deployment bookkeeping: stand it up once when the database is stood up. Every release then reads it (baseline) and writes it (stamp). The lab provisions it in “Before you start” for exactly this reason.

The gate is per database

Because the baseline runs per target database, it decides independently for each one. Take a fleet where every tenant is at version 1, then push just one tenant ahead to version 2 out of band. Run v1 across all of them: the tenant that’s already at 2 aborts with Invalid baseline for this release, and the tenants still at 1 pass and re-stamp. The run reports the failure, keeps going, and exits 2. One blocked tenant doesn’t stop the others — the gate is evaluated database by database, exactly where the version lives.

Two gates, one pipeline

Put this next to Module 2 and you’ve got defense in depth. Pre-flight — --TestConnection and --PreviewTargets — reads the target from the outside: is it reachable, does it clear the version floor, which databases resolve. Validation scripts read it from the inside: does the expected dependency exist, is this the right server, is the database at a version I’m allowed to deploy over. Pre-flight catches what you can see before you knock. Validation scripts catch what only a live query against real state can tell you. Run both, and the quench only fires when every gate says go.

Per-engine notes

The gates behave identically on all four engines; only the native spelling changes.

	SQL Server	PostgreSQL	MySQL
Admin DB (ValidationScript)	`master`	`postgres`	`information_schema`
Truthy value	`CAST(… AS BIT)`	native `boolean`	`EXISTS(…)` → 0/1
Registry read/write	`dbo.SchemaVersion`	`public.schema_version`	`schema_version`
Server-not-right abort	exit 3, `Invalid server for this product`	same	same
Old-release abort	exit 2, `Invalid baseline for this release`	same	same

The abort messages and the exit codes are the same everywhere — 3 when the server validation says no, 2 when a baseline does. Write the gate once, carry it to every fire.

A good smith doesn’t quench blind. Before the metal goes in the oil, you check it’s the right piece and the fire’s the right heat — because the one time it isn’t, you want to know before the quench, not after. ValidationScript checks the server. The baseline checks the version. Both run inside the quench, both read live state, and when either one says no, the deploy stays home.

Guarding a deploy and not sure which gate to reach for? Email me at forgebarrett@schemasmith.com — I read every one.

More’s coming from the forge — packaging a patch down to only the objects that changed, and carrying it safely to one target at a time.

Until then, may every gate you set answer true, and may no bad heat ever reach good metal.

— Forge

Mark this lesson complete