The Pull Request Is Not the Finish Line

An abstract release loop connecting a pull request, monitored signals, and feedback

A green pull request proves a change cleared the checks you ran. It does not prove the change behaves safely with real traffic, real data, and the systems around it.

When AI makes implementation faster, this gap grows — more changes reach the release queue. The team needs a better answer than "merge and hope."

This part is a tutorial for building the release package — release-package.md, the final of the six documents — for Tablewait's waitlist auto-promote.

Step 1: Generate the release package from a template

Give the agent the generative prompt and the template — release-package.md is the shape to fill. A meaningful change needs a release note before it reaches production — not a long document, just the decisions that make it observable and reversible.

Draft release-package.md for this feature.
 
Use release-package.md as the exact template:
the release note (change, risk level, rollout stages, approval, rollback,
success signals, failure signals, observation window, owner, escalation),
the monitoring events schema, numbered deploy steps, numbered monitor
steps, and the feedback into the next work packet.
 
Read first:
- the test evidence in test-package.md
- how the waitlist_auto_promote flag and the notification service work
 
Base the risk level and signals on the real failure modes (double-book,
stranded guest). Flag anything you cannot observe with current tooling.

The release note it returns looks like this:

## Release note
 
- **Change:** Add automatic waitlist promotion. When a host marks a table available, the system
  offers it to the next eligible party with a 5-minute hold that auto-rolls on decline/expiry.
  Gated by feature flag `waitlist_auto_promote` (default OFF).
- **Risk level:** Medium. Touches seating correctness and sends diner notifications, but is
  flag-gated and the flag-OFF path is byte-for-byte the current manual flow (AC7).
- **Rollout stages:** `pilot-restaurant` → `region` → `all`.
- **Approval:** the engineering on-call and the floor-ops lead approve each advance to the next stage.
- **Rollback:** disable the `waitlist_auto_promote` flag. In-flight holds are drained, not
  cut: active holds remain valid until they are accepted or expire via the sweep; no **new**
  holds are created once the flag is OFF. The manual flow resumes immediately.
- **Success signals:** host "mark available" → first hold within a few seconds; accept→seat
  works; expired holds roll to the next party; `double_book_detected == 0`.
- **Failure signals:** `double_book_detected > 0` (any count); holds created but never offered;
  expiry sweep lag growing; spike in declines or notification errors.
- **Observation window:** 7 days per stage.
- **Owner:** `waitlist-on-call`.
- **Escalation:** page `waitlist-on-call`; if `double_book_detected > 0`, disable the flag
  immediately and open a Sev-2.

The release note describes the world after merge. That is the world the diner experiences.

Step 2: A feature flag is not a release plan

The waitlist_auto_promote flag limits blast radius. It does not decide what to watch, who can turn it off, or what counts as success. For this change the flag is the rollback lever — but the plan still needs stages, signals, and an owner.

The question every release package must answer:

If this change is wrong in production, how will we know, and what will we do next?

If nobody can answer it, the pull request is not ready.

Step 3: Emit change-level signals

A global error rate will not tell you whether this feature works. Emit events that answer the questions the change created.

// src/waitlist/telemetry/events.ts
type WaitlistHoldEvent =
  | { type: 'hold_created';         holdId: string; tableId: string; waitlistEntryId: string; restaurantId: string; expiresAt: string }
  | { type: 'hold_accepted';        holdId: string; tableId: string; waitlistEntryId: string; latencyMs: number }
  | { type: 'hold_declined';        holdId: string; tableId: string; waitlistEntryId: string }
  | { type: 'hold_expired';         holdId: string; tableId: string; waitlistEntryId: string; sweepLagMs: number }
  | { type: 'promotion_rolled';     tableId: string; fromEntryId: string; toEntryId: string | null }
  | { type: 'double_book_detected'; tableId: string; holdIds: string[] };   // must always be 0

Those map straight to the questions a reviewer will ask:

  • Did a table free create a hold? (hold_created)
  • Did parties accept, decline, or let it expire? (hold_accepted, hold_declined, hold_expired)
  • Did the offer roll to the next party? (promotion_rolled)
  • Did any table get booked twice? (double_book_detected — this should always be zero)

Step 4: Write the deploy steps

Make the rollout a checklist, not a memory.

  1. Merge with waitlist_auto_promote OFF in every environment; confirm the flag-OFF e2e (AC7) is green.
  2. Apply migration 0012_waitlist_hold.sql (adds table + unique partial index); verify the index exists before enabling anything.
  3. Deploy the expiry scheduler (runHoldExpirySweep, ~30s interval) in the disabled-flag state.
  4. Enable waitlist_auto_promote for the pilot-restaurant cohort only.
  5. Run the smoke path: host marks a table available → confirm one hold_created, accept → hold_accepted.
  6. Hold 7 days; if signals are clean, advance the flag to region, then to all.

Step 5: Write the monitor steps

  1. Watch double_book_detected — alert on any value > 0; auto-page waitlist-on-call.
  2. Track hold_created → hold_accepted conversion and accept latency per stage.
  3. Track hold_expired.sweepLagMs — alert if the sweep falls far behind the ~30s cadence.
  4. Verify active-hold count per table never exceeds 1 (DB invariant + telemetry cross-check).
  5. Compare notification volume/error rate against the manual-flow baseline.
  6. Review promotion_rolled with toEntryId == null (empty-waitlist no-ops) — expected, not errors.

Step 6: Give the release agent a narrow job

A deployment agent can prepare evidence without owning the decision.

Prepare the release evidence for change waitlist-auto-promote.
 
Read release-package.md.
 
Do not deploy, change feature flags, or modify production data.
 
Return:
1. The rollout stages and who approves each advance.
2. A link or query for each success and failure signal.
3. The observation-window start and end for the current stage.
4. The exact rollback action and its owner.
5. Any signal that cannot be observed with current tooling.

The agent makes the release legible. It does not get permission because it produced a confident summary.

Step 7: Feed production learning into the next work packet

A release creates new facts. Route them back into the loop instead of leaving them as hunches.

## Feedback into the next work packet
 
- If sweep lag is material, make the interval or hold duration configurable (currently fixed).
- If decline rates are high, a next packet could add a priority/VIP queue (explicitly out of
  scope here).
- Feed real accept-latency and roll-to-next counts back into `test-package.md` thresholds so
  the acceptance tests assert against observed production ranges, not guesses.

What the finished release package looks like

Put together, the reviewed release package has five sections, in this order:

# Release Package — Waitlist Auto-Promote
 
## Release note
 
## Monitoring events schema
 
## Deploy steps
 
## Monitor steps
 
## Feedback into the next work packet

Risk should decide the gate

Not every change needs the same ceremony.

Change typeSuitable gate
Copy or visual tweakPreview, browser check, normal review
Bounded feature behind a flagAutomated checks, human approval, observation window
Concurrency, permission, or data changeStaged rollout, rollback rehearsal, explicit owner
Irreversible actionHuman approval at the action boundary

Auto-promote sits in row three: it is a concurrency change that can double-book, so it earns staged rollout and a rehearsed rollback. The release note rates it medium: it touches seating correctness and sends diner notifications, but it is flag-gated and the flag-off path is the current manual flow.

The full loop and the evidence

The delivery loop is now visible end to end:

And it is no longer just a diagram. Each part produced a real document, and together they are the downloadable evidence that the method works:

Download all six documents (.zip) — requirements, solution design, implementation plan, status tracker, test package, and release package for one feature.

Code is one part of the loop. A delivery system is the whole path.