Find S3 Buckets Without Lifecycle Rules

Tall warehouse shelving stacked with rows of identical cardboard boxes

Photo by Lance Chang on Unsplash

To find S3 buckets without a lifecycle policy, list your buckets and call GetBucketLifecycleConfiguration on each. A bucket with no rules returns the error NoSuchLifecycleConfiguration (HTTP 404) instead of an empty list. Pair the result with each bucket’s BucketSizeBytes from CloudWatch so you fix the large buckets first.

This example is for engineers who suspect their S3 bill includes data nobody asked to keep: old logs, abandoned multipart uploads and noncurrent versions that no rule ever expires. The script uses AWS SDK for JavaScript v3 to find S3 buckets without a lifecycle policy, shows which ones lack a rule that cleans up incomplete multipart uploads, and sizes every bucket from the daily storage metrics S3 publishes to CloudWatch at no additional cost (reading them with GetMetricData is an ordinary CloudWatch API call).

It deliberately stops at the one rule that’s safe for almost every bucket. Choosing transitions to colder storage classes is a cost decision covered in S3 storage class cost for backups, Glacier vs Standard-IA, so this page doesn’t repeat it.

What counts as a bucket without a lifecycle policy?

The script sorts buckets into three states:

State What the API returns Why it matters
No lifecycle configuration NoSuchLifecycleConfiguration, 404 Nothing ever expires or moves; every object and version stays in its class forever.
Rules, but no multipart cleanup Rules without AbortIncompleteMultipartUpload Parts from failed uploads stay stored and billed until someone aborts them.
Rules with multipart cleanup An enabled rule with AbortIncompleteMultipartUpload The baseline is in place; review the other rules separately.

Incomplete uploads are easy to miss because they don’t appear in object listings. They do count in the BucketSizeBytes metric, which includes the size of all parts of incomplete multipart uploads. If you want to see the individual uploads before cleaning them up, the example to upload large files and streams to S3 with SDK v3 includes a script that lists and aborts them.

Prerequisites

  • Node.js 20 or later, npm and tsx.
  • The @aws-sdk/client-s3 and @aws-sdk/client-cloudwatch packages.
  • A profile for the account. ListBuckets returns each bucket’s Region in BucketRegion, and the script calls S3 and CloudWatch in that Region, so one run covers every Region. It covers general purpose buckets; directory buckets have their own ListDirectoryBuckets API.

Which IAM permissions does it need?

Three read actions for the report. s3:PutLifecycleConfiguration is only for --add-abort-rule --apply. Note the action names: the API is GetBucketLifecycleConfiguration, but the IAM action is s3:GetLifecycleConfiguration.

find-s3-lifecycle-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucketsAndMetrics",
      "Effect": "Allow",
      "Action": ["s3:ListAllMyBuckets", "cloudwatch:GetMetricData"],
      "Resource": "*"
    },
    {
      "Sid": "ReadLifecycle",
      "Effect": "Allow",
      "Action": "s3:GetLifecycleConfiguration",
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Sid": "OptionalWriteLifecycle",
      "Effect": "Allow",
      "Action": "s3:PutLifecycleConfiguration",
      "Resource": "arn:aws:s3:::*"
    }
  ]
}

Narrow arn:aws:s3:::* to specific buckets if you only manage some of them. To check the mapping from code to actions yourself, the free IAM policy generator for TypeScript code reads the script and drafts a policy.

The script to find S3 buckets without lifecycle rules

find-s3-buckets-without-lifecycle.ts

// find-s3-buckets-without-lifecycle.ts
// Lists every general purpose S3 bucket with its lifecycle status, whether any rule aborts
// incomplete multipart uploads, and its size from CloudWatch daily storage metrics. Read-only by default.
// With --add-abort-rule it shows which buckets would get an "abort incomplete multipart uploads
// after N days" rule (dry run); add --apply to write it. Existing rules are kept.
// Usage: npx tsx find-s3-buckets-without-lifecycle.ts [--add-abort-rule [--days 7] [--apply]]
import {
  S3Client,
  GetBucketLifecycleConfigurationCommand,
  PutBucketLifecycleConfigurationCommand,
  paginateListBuckets,
  type LifecycleRule,
  type TransitionDefaultMinimumObjectSize,
} from "@aws-sdk/client-s3";
import { CloudWatchClient, GetMetricDataCommand } from "@aws-sdk/client-cloudwatch";

const addAbortRule = process.argv.includes("--add-abort-rule");
const apply = process.argv.includes("--apply");
const dayIndex = process.argv.indexOf("--days");
const abortDays = dayIndex === -1 ? 7 : Number(process.argv[dayIndex + 1]);
if (!Number.isInteger(abortDays) || abortDays < 1) throw new Error("--days must be a whole number of days");

type Lifecycle = {
  rules: LifecycleRule[];
  minSize: TransitionDefaultMinimumObjectSize | undefined;
};

type BucketRow = {
  bucket: string;
  region: string;
  rules: number;
  abortMpu: string;
  standardGB: number | string;
  objects: number | string;
};

const s3Clients = new Map<string, S3Client>();
const cwClients = new Map<string, CloudWatchClient>();

function s3For(region: string): S3Client {
  let c = s3Clients.get(region);
  if (!c) s3Clients.set(region, (c = new S3Client({ region, maxAttempts: 5 })));
  return c;
}

function cwFor(region: string): CloudWatchClient {
  let c = cwClients.get(region);
  if (!c) cwClients.set(region, (c = new CloudWatchClient({ region, maxAttempts: 5 })));
  return c;
}

async function getLifecycle(bucket: string, region: string): Promise<Lifecycle | null> {
  try {
    const out = await s3For(region).send(new GetBucketLifecycleConfigurationCommand({ Bucket: bucket }));
    return { rules: out.Rules ?? [], minSize: out.TransitionDefaultMinimumObjectSize };
  } catch (err) {
    // No lifecycle configuration is an error response (404), not an empty list.
    if (err instanceof Error && err.name === "NoSuchLifecycleConfiguration") return null;
    throw err;
  }
}

// Latest daily BucketSizeBytes (S3 Standard only) and NumberOfObjects (all classes).
async function getSize(bucket: string, region: string): Promise<{ gb?: number; objects?: number }> {
  const end = new Date();
  const start = new Date(end.getTime() - 3 * 86_400_000);
  const metric = (id: string, name: string, storageType: string) => ({
    Id: id,
    MetricStat: {
      Metric: {
        Namespace: "AWS/S3",
        MetricName: name,
        Dimensions: [
          { Name: "BucketName", Value: bucket },
          { Name: "StorageType", Value: storageType },
        ],
      },
      Period: 86_400,
      Stat: "Average",
    },
  });
  const out = await cwFor(region).send(
    new GetMetricDataCommand({
      StartTime: start,
      EndTime: end,
      ScanBy: "TimestampDescending",
      MetricDataQueries: [
        metric("size", "BucketSizeBytes", "StandardStorage"),
        metric("objects", "NumberOfObjects", "AllStorageTypes"),
      ],
    }),
  );
  const latest = (id: string) => out.MetricDataResults?.find((r) => r.Id === id)?.Values?.[0];
  const bytes = latest("size");
  return { gb: bytes === undefined ? undefined : bytes / 1024 ** 3, objects: latest("objects") };
}

async function main(): Promise<void> {
  const rows: BucketRow[] = [];
  const lifecycles = new Map<string, Lifecycle | null>();

  const lister = new S3Client({ region: process.env.AWS_REGION ?? "us-east-1" });
  for await (const page of paginateListBuckets({ client: lister, pageSize: 1000 }, {})) {
    for (const b of page.Buckets ?? []) {
      if (!b.Name) continue;
      const region = b.BucketRegion ?? "us-east-1";
      const lc = await getLifecycle(b.Name, region);
      lifecycles.set(b.Name, lc);
      const size = await getSize(b.Name, region);
      rows.push({
        bucket: b.Name,
        region,
        rules: lc?.rules.length ?? 0,
        abortMpu: lc?.rules.some((r) => r.Status === "Enabled" && r.AbortIncompleteMultipartUpload) ? "yes" : "NO",
        standardGB: size.gb === undefined ? "-" : Number(size.gb.toFixed(2)),
        objects: size.objects ?? "-",
      });
    }
  }

  const gb = (r: BucketRow) => (typeof r.standardGB === "number" ? r.standardGB : 0);
  rows.sort((a, b) => a.rules - b.rules || gb(b) - gb(a));
  console.table(rows);
  const none = rows.filter((r) => r.rules === 0);
  console.log(`${rows.length} buckets, ${none.length} without any lifecycle rule, ` +
    `${rows.filter((r) => r.abortMpu === "NO").length} without an abort-incomplete-multipart rule.`);
  if (!addAbortRule) return;

  const targets = rows.filter((r) => r.abortMpu === "NO");
  const ruleId = `abort-incomplete-mpu-${abortDays}d`;
  console.log(`\n${apply ? "Adding" : "Dry run: would add"} rule ${ruleId} to ${targets.length} buckets:`);
  for (const r of targets) {
    const current = lifecycles.get(r.bucket) ?? null;
    const existing = current?.rules ?? [];
    if (existing.some((rule) => rule.ID === ruleId)) {
      console.log(`  ${r.bucket}: a rule named ${ruleId} exists but is disabled; skipped`);
      continue;
    }
    const newRule: LifecycleRule = {
      ID: ruleId,
      Status: "Enabled",
      Filter: {}, // empty filter = every object in the bucket
      AbortIncompleteMultipartUpload: { DaysAfterInitiation: abortDays },
    };
    console.log(`  ${r.bucket} (${existing.length} existing rules kept)`);
    if (!apply) continue;
    // PutBucketLifecycleConfiguration replaces the whole configuration, so send the old rules too,
    // and keep the bucket's minimum-object-size setting for transitions.
    await s3For(r.region).send(
      new PutBucketLifecycleConfigurationCommand({
        Bucket: r.bucket,
        LifecycleConfiguration: { Rules: [...existing, newRule] },
        TransitionDefaultMinimumObjectSize: current?.minSize,
      }),
    );
  }
  if (!apply && targets.length > 0) console.log("Re-run with --add-abort-rule --apply to write the rules.");
}

main().catch((err: unknown) => {
  console.error(err);
  process.exit(1);
});

Three details make the write step safe. PutBucketLifecycleConfiguration replaces the whole configuration, so the script sends the existing rules plus the new one. It passes back the bucket’s TransitionDefaultMinimumObjectSize, so buckets that allow small objects to transition into Glacier classes keep that behavior. And the new rule has an empty filter, which applies to every object; AbortIncompleteMultipartUpload can’t be used in a rule whose filter uses object tags anyway.

How do you run it?

Terminal

npm install @aws-sdk/client-s3 @aws-sdk/client-cloudwatch
npm install --save-dev tsx typescript

# Report only
AWS_PROFILE=audit npx tsx find-s3-buckets-without-lifecycle.ts

# Dry run: which buckets would get the multipart cleanup rule
AWS_PROFILE=audit npx tsx find-s3-buckets-without-lifecycle.ts --add-abort-rule --days 7

# Write the rule
AWS_PROFILE=storage-admin npx tsx find-s3-buckets-without-lifecycle.ts --add-abort-rule --days 7 --apply

Sample output

Output (illustrative)

┌─────────┬─────────────────────┬─────────────┬───────┬──────────┬────────────┬─────────┐
│ (index) │ bucket              │ region      │ rules │ abortMpu │ standardGB │ objects │
├─────────┼─────────────────────┼─────────────┼───────┼──────────┼────────────┼─────────┤
│ 0       │ 'acme-app-logs'     │ 'us-east-1' │ 0     │ 'NO'     │ 1843.27    │ 9120455 │
│ 1       │ 'acme-data-exports' │ 'eu-west-1' │ 0     │ 'NO'     │ 612.4      │ 20318   │
│ 2       │ 'acme-static-site'  │ 'us-east-1' │ 0     │ 'NO'     │ 0.31       │ 412     │
│ 3       │ 'acme-backups'      │ 'us-east-1' │ 1     │ 'yes'    │ 95.02      │ 366     │
│ 4       │ 'acme-user-uploads' │ 'us-east-1' │ 2     │ 'NO'     │ 250.66     │ 1804223 │
└─────────┴─────────────────────┴─────────────┴───────┴──────────┴────────────┴─────────┘
5 buckets, 3 without any lifecycle rule, 4 without an abort-incomplete-multipart rule.

Dry run: would add rule abort-incomplete-mpu-7d to 4 buckets:
  acme-app-logs (0 existing rules kept)
  acme-data-exports (0 existing rules kept)
  acme-static-site (0 existing rules kept)
  acme-user-uploads (2 existing rules kept)
Re-run with --add-abort-rule --apply to write the rules.

Names and numbers are placeholders. standardGB is S3 Standard only; buckets that already use other classes show less here than their total. A - means CloudWatch had no data point in the last three days, which is normal for a bucket created today. The storage metrics are daily, so today’s uploads show up tomorrow. The example to find the size of each S3 bucket and the largest one breaks size down further.

Which lifecycle rules should each bucket get?

Once the cleanup rule is in, decide the rest per bucket. Useful rule types, from the S3 lifecycle configuration elements reference:

  • Log and export buckets: an Expiration rule after your retention period. In the example above, acme-app-logs is the biggest win.
  • Versioned buckets: NoncurrentVersionExpiration, optionally with NewerNoncurrentVersions (1 to 100) to keep a few recent versions, plus ExpiredObjectDeleteMarker to clear leftover delete markers. The script to find S3 buckets without versioning enabled tells you which buckets this applies to.
  • Data that cools off: transitions to a colder class, or Intelligent-Tiering if access is unpredictable. The comparison of S3 Standard vs Intelligent-Tiering cost helps with that choice, and lifecycle transitions are billed per request, as the guide to calculate S3 GET and PUT request costs explains.
  • Static website buckets: often nothing beyond the cleanup rule.

Some timing rules apply to all of these. S3 counts days from object creation and rounds up to the next midnight UTC. A new or changed configuration takes a few minutes to propagate. And a configuration holds at most 1,000 rules, a limit that can’t be raised.

Troubleshooting

  • AccessDenied on GetBucketLifecycleConfiguration. Check the IAM action name (s3:GetLifecycleConfiguration) and any bucket policy that denies it. The steps to troubleshoot AWS IAM access denied errors cover bucket policies and SCPs.
  • A rule named abort-incomplete-mpu-7d is skipped. A rule with that ID exists but is disabled. Enable it or delete it, then rerun.
  • Small objects never transition. By default, objects under 128 KB don’t transition to any storage class; the TransitionDefaultMinimumObjectSize setting and ObjectSizeGreaterThan filters change that.
  • The size column stays -. The bucket is empty or new, or the objects are all in classes other than S3 Standard.

Ask ChatWithCloud instead

For a quick answer from a read-only profile, ask ChatWithCloud “Which S3 buckets have no lifecycle configuration?”. It writes AWS SDK for JavaScript v2 code, runs it locally with your profile and summarizes the result, and you can follow up with “which of those are over 100 GB?”. The walkthrough to ask AI which S3 buckets are largest shows that kind of session. It runs changes without asking first, so don’t give it s3:PutLifecycleConfiguration unless you want it to be able to rewrite rules.

The same “set a retention rule everywhere” pattern appears in the scripts to set CloudWatch log retention for all log groups and to delete old ECR images with a lifecycle policy. EFS has its own version, and the script to find EFS file systems without a lifecycle policy moves cold files to cheaper storage classes. Monitoring leaves its own leftovers: the script to find CloudWatch alarms stuck in INSUFFICIENT_DATA removes alarms that watch resources that no longer exist.

Frequently asked questions

How do I check if an S3 bucket has a lifecycle policy with the AWS CLI?

Run aws s3api get-bucket-lifecycle-configuration --bucket my-bucket. If there is none, the command fails with NoSuchLifecycleConfiguration.

Does adding a lifecycle rule overwrite the existing ones?

Yes. PutBucketLifecycleConfiguration replaces the whole configuration, so always read the current rules and send them back with the new one.

Do expiration rules remove incomplete multipart uploads?

No. You need a separate AbortIncompleteMultipartUpload action with DaysAfterInitiation.

How soon does a new lifecycle rule take effect?

The configuration propagates within a few minutes, and objects become eligible at the next midnight UTC after their age passes the rule’s days.

Related guides

Ask your AWS account in plain English

Your first 15 runs are free, with no OpenAI key needed.

npx chatwithcloud