Find CloudWatch Alarms Stuck in INSUFFICIENT_DATA

Several monitoring screens showing graphs and charts in a dark operations room

Photo by McCarthy Beckan on Unsplash

To find CloudWatch alarms stuck in INSUFFICIENT_DATA, call DescribeAlarms with StateValue set to INSUFFICIENT_DATA and AlarmTypes set to both metric and composite alarms, then compare each StateUpdatedTimestamp with today. Check each alarm’s metric with ListMetrics: if it hasn’t reported in two weeks, the resource is probably gone and the alarm can be deleted.

An alarm in INSUFFICIENT_DATA for a day is normal. One that’s been there for months is watching nothing: the instance was terminated, the queue was renamed, the load balancer was replaced. It still costs money, it clutters every alarm list, and it trains people to ignore the grey state. This example is for engineers who want to find CloudWatch alarms with insufficient data, sort the dead ones from the merely quiet ones, and delete only the dead ones.

You’ll get a TypeScript script for the AWS SDK for JavaScript v3 that reports by default and deletes only with --apply. It’s one of our AWS SDK v3 cost and cleanup examples, and it pairs with the script to get this month’s AWS CloudWatch cost with Cost Explorer, which tells you how much of your CloudWatch bill alarms are.

Why do CloudWatch alarms get stuck in INSUFFICIENT_DATA?

An alarm reports INSUFFICIENT_DATA when it has just started, when its metric isn’t available, or when there aren’t enough data points to decide. How it treats gaps depends on TreatMissingData, which can be missing (the default), notBreaching, breaching or ignore. The long-stuck alarms usually fall into three groups:

  • The resource is gone. An alarm on CPUUtilization for a terminated instance keeps its InstanceId dimension forever. No new data will ever arrive.
  • The metric is sparse. Some metrics are only published when something happens, such as errors or messages. The alarm is fine; setting TreatMissingData to notBreaching usually is the right fix.
  • The alarm is wrong. A typo in a dimension value or namespace means it never matched a real metric. For custom metrics, the alarm has to name every dimension exactly as the code publishes it, as the guide to publish custom CloudWatch metrics with AWS SDK v3 explains.

The first and third groups have the same symptom the script tests for: ListMetrics doesn’t return metrics that haven’t reported data in the past two weeks, so an empty result means nothing has published that metric recently.

What do stale alarms cost?

As of September 2026, the Amazon CloudWatch pricing page and the AWS Price List give these us-east-1 rates. Alarm charges are prorated by the hour, and the free tier covers 10 standard-resolution alarm metrics.

Alarm type Price per month
Standard resolution metric alarm $0.10 per alarm metric
Anomaly detection alarm 3 alarm metrics (the metric plus two bands), so $0.30
Composite alarm $0.50 each

Worked example: 60 dead single-metric alarms cost 60 × $0.10 = $6.00 a month, or $72 a year. The money is modest. The bigger cost is attention: Google’s SRE book chapter on monitoring distributed systems argues that alerting should stay simple and every page actionable, and a list full of alarms that can never fire works against both.

What does the script do?

  1. Lists stuck alarmspaginateDescribeAlarms with StateValue: "INSUFFICIENT_DATA" and AlarmTypes set to both types. Without AlarmTypes, the API returns only metric alarms.
  2. Filters by ageKeeps alarms whose StateUpdatedTimestamp is at least --days old (default 7).
  3. Checks each metricCalls ListMetrics with the alarm’s namespace, name and dimensions. Metric math alarms are checked for every metric they read.
  4. Protects dependenciesSkips alarms whose names start with TargetTracking-, which Auto Scaling owns, and uses DescribeAlarms with ParentsOfAlarmName to skip alarms a composite alarm depends on.
  5. Deletes only on requestWith --apply, calls DeleteAlarms in batches of up to 100 names for alarms marked DELETE.

Prerequisites

  • Node.js 18 or later, npm and tsx.
  • The @aws-sdk/client-cloudwatch package.
  • A profile with a default Region, or AWS_REGION set. Alarms are Regional; run once per Region.

Which IAM permissions does it need?

cloudwatch:DescribeAlarms must be granted on *, or the API won’t return composite alarms. The delete statement is only for --apply; scope it to your account and Region (replace 123456789012 and us-east-1).

stale-alarms-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReportStaleAlarms",
      "Effect": "Allow",
      "Action": ["cloudwatch:DescribeAlarms", "cloudwatch:ListMetrics"],
      "Resource": "*"
    },
    {
      "Sid": "DeleteStaleAlarms",
      "Effect": "Allow",
      "Action": "cloudwatch:DeleteAlarms",
      "Resource": "arn:aws:cloudwatch:us-east-1:123456789012:alarm:*"
    }
  ]
}

The IAM policy generator for TypeScript SDK code drafts this from the script.

The full script to find CloudWatch alarms in INSUFFICIENT_DATA

find-cloudwatch-alarms-insufficient-data.ts

// find-cloudwatch-alarms-insufficient-data.ts
// Lists CloudWatch alarms in one Region that have been in INSUFFICIENT_DATA for more than N days,
// and checks whether the metric behind each metric alarm has reported any data in the last two weeks.
// Report-only by default. --apply deletes metric alarms whose metrics are gone and that no
// composite alarm depends on. Composite alarms are only reported.
// Usage: npx tsx find-cloudwatch-alarms-insufficient-data.ts [--days 7] [--apply]
import {
  CloudWatchClient,
  DeleteAlarmsCommand,
  DescribeAlarmsCommand,
  ListMetricsCommand,
  paginateDescribeAlarms,
  type MetricAlarm,
  type Metric,
} from "@aws-sdk/client-cloudwatch";

const args = process.argv.slice(2);
const apply = args.includes("--apply");
const minDays = Number(args[args.indexOf("--days") + 1]) || 7;
const cw = new CloudWatchClient({}); // region from AWS_REGION or your profile

interface Row { Alarm: string; Type: string; StuckDays: number; Metric: string; MetricData: string; Verdict: string }

// The metrics an alarm reads: a single metric, or every MetricStat inside a metric math alarm.
function metricsOf(a: MetricAlarm): Metric[] {
  if (a.MetricName) return [{ Namespace: a.Namespace, MetricName: a.MetricName, Dimensions: a.Dimensions }];
  return (a.Metrics ?? []).flatMap((q) => (q.MetricStat?.Metric ? [q.MetricStat.Metric] : []));
}

// ListMetrics only returns metrics that reported data in the past two weeks.
async function reportedRecently(m: Metric): Promise<boolean> {
  const res = await cw.send(new ListMetricsCommand({
    Namespace: m.Namespace,
    MetricName: m.MetricName,
    Dimensions: (m.Dimensions ?? []).map((d) => ({ Name: d.Name, Value: d.Value })),
  }));
  const want = (m.Dimensions ?? []).length;
  return (res.Metrics ?? []).some((x) => (x.Dimensions ?? []).length === want);
}

async function hasParents(name: string): Promise<boolean> {
  const res = await cw.send(new DescribeAlarmsCommand({ ParentsOfAlarmName: name }));
  return (res.CompositeAlarms ?? []).length + (res.MetricAlarms ?? []).length > 0;
}

async function main(): Promise<void> {
  const rows: Row[] = [];
  const deletable: string[] = [];
  const now = Date.now();
  for await (const page of paginateDescribeAlarms({ client: cw }, {
    StateValue: "INSUFFICIENT_DATA",
    AlarmTypes: ["MetricAlarm", "CompositeAlarm"],
  })) {
    for (const a of page.MetricAlarms ?? []) {
      const stuck = Math.floor((now - (a.StateUpdatedTimestamp?.getTime() ?? now)) / 86_400_000);
      if (stuck < minDays) continue;
      const metrics = metricsOf(a);
      const live = await Promise.all(metrics.map(reportedRecently));
      const label = metrics.map((m) => `${m.Namespace}/${m.MetricName} ${(m.Dimensions ?? []).map((d) => d.Value).join(",")}`).join(" + ");
      let verdict: string;
      if (a.AlarmName?.startsWith("TargetTracking-")) verdict = "owned by a scaling policy: leave it";
      else if (!metrics.length) verdict = "query-based alarm: check manually";
      else if (live.some(Boolean)) verdict = `metric reports: check TreatMissingData (${a.TreatMissingData ?? "missing"})`;
      else if (await hasParents(a.AlarmName ?? "")) verdict = "metric gone, but a composite alarm uses it";
      else {
        verdict = "DELETE: metric gone";
        deletable.push(a.AlarmName ?? "");
      }
      rows.push({ Alarm: a.AlarmName ?? "", Type: "metric", StuckDays: stuck, Metric: label || "(expression)", MetricData: metrics.length ? (live.some(Boolean) ? "yes" : "none in 14 days") : "n/a", Verdict: verdict });
    }
    for (const c of page.CompositeAlarms ?? []) {
      const stuck = Math.floor((now - (c.StateUpdatedTimestamp?.getTime() ?? now)) / 86_400_000);
      if (stuck >= minDays) rows.push({ Alarm: c.AlarmName ?? "", Type: "composite", StuckDays: stuck, Metric: "(rule)", MetricData: "n/a", Verdict: "review its child alarms" });
    }
  }

  rows.sort((x, y) => y.StuckDays - x.StuckDays);
  console.table(rows);
  // Standard-resolution alarms cost $0.10 per alarm metric per month in us-east-1 (price list, September 2026).
  console.log(`${rows.length} alarms stuck in INSUFFICIENT_DATA for ${minDays}+ days; ${deletable.length} watch metrics that no longer exist (about $${(deletable.length * 0.1).toFixed(2)}/month).`);

  if (!apply) {
    console.log("Report only: nothing was deleted. Re-run with --apply to delete the alarms marked DELETE.");
    return;
  }
  for (let i = 0; i < deletable.length; i += 100) {
    const batch = deletable.slice(i, i + 100); // DeleteAlarms takes up to 100 names per call
    await cw.send(new DeleteAlarmsCommand({ AlarmNames: batch }));
    console.log(`Deleted ${batch.length} alarms: ${batch.join(", ")}`);
  }
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

How do you run it?

Terminal

npm install @aws-sdk/client-cloudwatch
npm install --save-dev tsx typescript

# Report alarms stuck for 7+ days
AWS_PROFILE=readonly AWS_REGION=us-east-1 npx tsx find-cloudwatch-alarms-insufficient-data.ts

# Only alarms stuck for 30+ days, and delete the ones whose metric is gone
AWS_PROFILE=admin AWS_REGION=us-east-1 npx tsx find-cloudwatch-alarms-insufficient-data.ts --days 30 --apply

Sample output

Output

┌─────────┬────────────────────┬─────────────┬───────────┬─────────────────────────────────────────────────────────────────────────┬───────────────────┬────────────────────────────────────────────────────┐
│ (index) │ Alarm              │ Type        │ StuckDays │ Metric                                                                  │ MetricData        │ Verdict                                            │
├─────────┼────────────────────┼─────────────┼───────────┼─────────────────────────────────────────────────────────────────────────┼───────────────────┼────────────────────────────────────────────────────┤
│ 0       │ 'web-3-high-cpu'   │ 'metric'    │ 212       │ 'AWS/EC2/CPUUtilization i-0a1b2c3d4e5f60718'                            │ 'none in 14 days' │ 'DELETE: metric gone'                              │
│ 1       │ 'legacy-alb-5xx'   │ 'metric'    │ 140       │ 'AWS/ApplicationELB/HTTPCode_ELB_5XX_Count app/legacy/50dc6c495c0c9188' │ 'none in 14 days' │ 'DELETE: metric gone'                              │
│ 2       │ 'db-old-cpu'       │ 'metric'    │ 95        │ 'AWS/RDS/CPUUtilization db-old'                                         │ 'none in 14 days' │ 'metric gone, but a composite alarm uses it'       │
│ 3       │ 'orders-dlq-depth' │ 'metric'    │ 31        │ 'AWS/SQS/ApproximateNumberOfMessagesVisible orders-dlq'                 │ 'yes'             │ 'metric reports: check TreatMissingData (missing)' │
│ 4       │ 'db-tier-health'   │ 'composite' │ 95        │ '(rule)'                                                                │ 'n/a'             │ 'review its child alarms'                          │
└─────────┴────────────────────┴─────────────┴───────────┴─────────────────────────────────────────────────────────────────────────┴───────────────────┴────────────────────────────────────────────────────┘
5 alarms stuck in INSUFFICIENT_DATA for 7+ days; 2 watch metrics that no longer exist (about $0.20/month).
Report only: nothing was deleted. Re-run with --apply to delete the alarms marked DELETE.

Names and IDs are illustrative. orders-dlq-depth is the sparse-metric case: the queue still reports, so the fix is TreatMissingData, not deletion. Queues with no DLQ at all have nothing to alarm on; the script to find SQS queues without a dead-letter queue lists them. db-old-cpu watches a database that no longer publishes metrics, but a composite alarm still references it, so change the composite rule first. Dashboards go stale the same way, and the script to find unused CloudWatch dashboards flags widgets whose metrics have stopped reporting.

What should you check before you delete alarms?

  • Infrastructure as code. If a CloudFormation, CDK or Terraform stack created the alarm, delete it there, or the next deploy recreates it or reports drift.
  • Alarms on resources that come and go. A metric for a scheduled job or a spot fleet may be absent for two weeks and back next month. Raise --days for those.
  • Alarm actions. An alarm stuck in INSUFFICIENT_DATA can still have an action for that state. Check InsufficientDataActions before you assume nobody depends on it.

The same “is anything still there?” question drives the scripts to find idle RDS instances with no connections and to find EC2 instances stopped for weeks. And if logs are a bigger share of your CloudWatch bill than alarms, the script to set CloudWatch log retention for all log groups is the next cleanup. Storage that grows forever is the other classic leftover, and the script to find S3 buckets without lifecycle rules sizes it bucket by bucket.

Troubleshooting

  • Composite alarms are missing from the report. Your cloudwatch:DescribeAlarms permission is scoped to specific alarms. Grant it on *.
  • Some alarms weren’t deleted. DeleteAlarms still deletes the valid names in a batch when others fail, so run the report again to see what’s left. A batch can include at most one composite alarm, which this script never deletes.
  • AccessDenied on DeleteAlarms. The delete statement’s Region or account doesn’t match. Compare the alarm ARN in the error message with the policy’s resource.
  • A Lambda alarm shows metric reports but never leaves the grey state. Lambda metrics only appear when the function runs. If that’s expected, set TreatMissingData to notBreaching.

Ask ChatWithCloud instead

You can also ask ChatWithCloud “Which CloudWatch alarms in us-east-1 have been in INSUFFICIENT_DATA for more than 30 days?” It writes AWS SDK for JavaScript v2 code, runs it on your machine with your AWS profile and explains the result, like the checks in the guide to troubleshoot AWS infrastructure with an AI CLI. It uses one profile and Region per session and runs generated code without a confirmation step, so ask for the list and delete with the script, and connect ChatWithCloud to a read-only AWS profile. The ChatWithCloud security model explains what’s sent to the model.

Frequently asked questions

What does INSUFFICIENT_DATA mean on a CloudWatch alarm?

The alarm has just started, its metric isn’t available, or there isn’t enough data to decide between OK and ALARM. Weeks in that state usually mean the metric no longer exists.

Do alarms in INSUFFICIENT_DATA cost money?

Yes. Alarms are billed by the hour they exist, whatever their state: $0.10 per standard-resolution alarm metric per month in us-east-1 as of September 2026, after the free tier of 10.

How do I stop an alarm going to INSUFFICIENT_DATA when there’s no traffic?

Set TreatMissingData to notBreaching (or ignore) with PutMetricAlarm. Missing data points then count as within the threshold, or keep the current state.

Can I delete the TargetTracking alarms?

No. Auto Scaling creates and manages the alarms of a target tracking policy. Delete or change the scaling policy instead, and leave its alarms alone.

Related guides

Ask your AWS account in plain English

Your first 15 runs are free, with no OpenAI key needed.

npx chatwithcloud