Post-deploy monitoring

Post-deploy monitoring is the practice of actively watching key production metrics in the minutes and hours after a deployment, with the goal of detecting deployment-induced regressions before they cause sustained user impact. Instead of deploying and moving on, the team treats the post-deployment window as a distinct monitoring phase: error rates, latency distributions, and business metrics are watched with heightened attention and lower alert thresholds immediately after a release.

At L2, post-deploy monitoring is a semi-manual practice. A deployment triggers a notification in Slack, the deploying developer or a designated reviewer watches the dashboard for 10-30 minutes, and if nothing alarming appears, the deployment is considered stable. The monitoring might involve watching a Grafana dashboard, checking Sentry for new error groups, or looking at a DataDog APM trace volume. The key distinction from no monitoring is intentionality: the team has defined what "healthy after deploy" looks like and is actively verifying that the new version meets that definition.

Canary deployments are the natural complement to post-deploy monitoring at this level. Rather than deploying to 100% of traffic immediately, a canary routes a small percentage (5-10%) of requests to the new version while the monitoring phase runs. If metrics stay healthy, traffic gradually shifts to 100%. If metrics degrade, the canary is rolled back before the majority of traffic is affected. Kubernetes supports canary patterns via weighted services or ingress controllers; AWS and GCP offer traffic-splitting in their deployment services. The canary reduces the blast radius of a bad deployment from "all users" to "a small percentage of users for a short window."

Health checks are the baseline mechanism for post-deploy monitoring: each service exposes an HTTP endpoint that returns 200 when healthy and a non-200 status when something is wrong. Kubernetes uses readiness and liveness probes. Load balancers use health checks to determine which instances receive traffic. At L2, these checks verify that the service started correctly and responds to requests; they do not yet verify business logic correctness. Combined with watching error rates and latency in the minutes after deploy, health checks form the basic canary evaluation criterion.

The deployment window is the highest-risk period in any service's lifecycle:

• Most production incidents are deployment-caused - industry data consistently shows that 60-80% of production incidents follow a recent deployment; monitoring this window catches the majority of incidents at the earliest possible moment
• Blast radius is smallest immediately after deploy - a regression caught in the first 5 minutes after deploying affects far fewer users than one caught hours later by a customer complaint
• Enables faster deployment cycles - teams that have reliable post-deploy monitoring deploy more frequently because they trust the safety net; without it, fear of regressions leads to batching changes into large, risky releases
• Provides deployment quality signal - tracking the percentage of deployments that required rollback or triggered an alert creates a metric for deployment quality that informs where to invest in testing
• Creates the data foundation for automated canary evaluation - the manual post-deploy monitoring practice, once instrumented, can be automated: define the healthy thresholds, run the deploy, let the system evaluate the canary and promote or roll back automatically

1. Define your post-deploy health criteria - Before the next deployment, answer: what does healthy look like for this service 10 minutes after deploy? At minimum: error rate below X%, P99 latency below Yms, no new Sentry error groups. Write these down. Vague monitoring ("watch the dashboard") is less useful than specific pass/fail criteria.
2. Add a deployment annotation to your dashboards - Configure your CI/CD system to send a deployment event to Grafana (via the Annotations API) or Datadog (via the Events API) at deploy time. This adds a vertical line to all relevant dashboards marking exactly when the deployment happened. Correlating metric changes with deployment events becomes visual and immediate.
3. Set up a dedicated post-deploy Slack notification - When a deployment completes, post a Slack message in your team channel that includes: service name, version deployed, deployer name, and links to the relevant dashboard and Sentry project. This notification is the starting gun for the post-deploy monitoring window and ensures the right person is watching.
4. Implement a basic canary pattern - For your most critical services, introduce traffic splitting. In Kubernetes, this can be as simple as deploying the new version as a separate deployment with a small replica count and using an ingress weight to route 10% of traffic to it. Verify the canary is healthy before proceeding to full rollout.
5. Create a post-deploy checklist - A simple checklist run by the deployer for the first 15 minutes: error rate stable? Latency stable? No new Sentry groups? Key business metrics (orders placed, logins, API calls) at expected levels? Checking these takes 2 minutes and catches most deployment regressions immediately.
6. Track rollback rate as a team metric - Record every deployment and whether it required rollback. A rollback rate above 5% indicates that changes are reaching production without adequate pre-deployment testing. A rollback rate at 0% for many months may indicate over-caution. The metric creates a quality signal for deployment practices.

Watching dashboards without defined pass/fail criteria. A developer who watches a dashboard for 10 minutes without knowing what "bad" looks like will usually declare the deployment healthy regardless of what they see. Define specific thresholds before the deployment. "Error rate increased by 20% from baseline" is an objective criterion; "the dashboard looks fine" is not.

Using only application health checks, ignoring business metrics. A service can be technically healthy (200 OK from the health endpoint, low error rate on HTTP requests) while a bug causes silent data corruption or incorrect business logic. Post-deploy monitoring needs to include metrics that capture whether the service is doing the right thing, not just whether it is responding.

No automatic rollback capability. Post-deploy monitoring without the ability to quickly roll back is incomplete. If a problem is detected, the team needs to revert in under 5 minutes. Ensure your deployment pipeline supports one-command rollback to the previous version, and test it before you need it in an incident.

Treating all services the same. A deployment to a low-traffic internal tool does not need the same monitoring intensity as a deployment to the primary checkout service. Define monitoring tiers by service criticality and apply proportionate post-deploy rigor. Spending 30 minutes watching a dashboard for a background job that runs once daily is waste; spending 30 minutes watching the checkout service after a price calculation change is essential.

Ending the monitoring window too early. Many deployment regressions do not appear immediately. Traffic patterns change throughout the day; a deployment at 2pm may cause problems that only appear when peak traffic hits at 6pm. For critical deployments, extend the monitoring window through the next traffic peak, not just the first 15 minutes of low-traffic observations.

How this guide changed across editions, newest first.

July 2026 (v1.4)

Followed the market shift from tokenmaxxing to efficiency with tooling that makes the spend line actionable. Local token-compression proxies run on-device and report 60-95% cost cuts, and both Anthropic and OpenAI shipped enterprise usage analytics and spend controls, so per-team and per-agent spend became a first-class dashboard alongside error rate and latency, with budget thresholds that alert the way a latency regression does.

May 2026 (v1.2)

Post-deploy monitoring now includes token-cost telemetry as table stakes. ccusage (13.2k stars on GitHub, ccusage.com) tracks per-session and per-project spend from local JSONL with cache breakdown and offline pricing. Claude-Code-Usage-Monitor adds live charts and "time-to-limit" predictions. Both /usage and /context shipped as built-in commands in April. Treat agent token spend the same way you treat error rate and p95 latency - it is a leading indicator that often spikes before the user-facing impact.

For Claude-based pipelines specifically, also watch the harness-quality signals from Stella Laurenzo's 6,852-session audit: median thinking length per turn, files read before edit. When those metrics drop sharply (Anthropic confirmed this happened in March-April 2026 due to harness changes, not the model), it is the same kind of leading indicator that error-rate change is for code deploys.

By June 2026 the market shifted from "tokenmaxxing" to efficiency (CNBC), and post-deploy cost monitoring has tooling to match. Local token-compression proxies run on-device (prompts never leave the machine) and report 60-95% token-cost cuts, which makes the spend line a metric you can act on rather than just observe. Both Anthropic and OpenAI shipped enterprise usage analytics and spend controls, so treat per-team and per-agent spend as a first-class dashboard alongside error rate and latency, with budget thresholds that alert the same way a latency regression would.